Subject: Re: [alto] Reprise of REST comments

From: Richard Alimi <[email-masked]>
Date: Mon, 3 Jan 2011 10:26:21 -0800

Hi Martin and WG,

Apologies for the delay in getting back to this.

Thank you for the details in documenting how the protocol could look.
A small part of this was included in a very early version
(pre-WG-adoption), but was removed and changed to its current form
because of a few complaints that it was too complicated than
necessary. See draft-penno-alto-protocol-01 (in particular,
http://tools.ietf.org/html/draft-penno-alto-protocol-01#section-5.1)
for the initial version.

Note that there is one comment inline below regarding ALTO
redistribution which I'm not convince that making the ALTO Protocol
REST-ful can solve.  Maybe you can convince me otherwise :)

To the WG:

Since this has been a recurring suggestion, and we are slated to
finish the protocol document soon, it would be good to get this
settled within the WG.  I think Martin has done a good job in
providing examples of an alternate (REST-ful) specification of the
protocol.

Looking at Martin's suggestions, I'm not convinced that it would
reduce the size or complexity of the document, but I do think that
there are benefits in terms of additional freedom for the server
implementation while simplifying certain parts of the client
implementation (e.g., I think Server List could be removed).

It would be good to hear opinions on whether or not you think there is
value in adopting a newer scheme for the protocol, and whether the
document should be updated to follow this.  We would like to avoid
generating another version of the document that would not even have
interest to the WG, in particular since an earlier version of the
document had some components of these suggestions (see above), but it
was removed due to a couple of comments from members of the WG.  If
people from the WG believe that the document should be updated to
follow this newer scheme, we are more than willing to update it.

Thanks!
Rich

On Tue, Nov 9, 2010 at 11:18 PM, Thomson, Martin
<[email-masked]> wrote:
> I apologise for this being a reprise of previous comments, but I still feel like the current alto protocol is causing you a lot more work than you need.
>
> You really want to spend less time on dealing with protocol construction and more time on optimizing cost maps.  By building your own protocol, I don't think you get that.
>
> The delta between what you have and something that relies more heavily on existing protocol (i.e., HTTP) work is not that large.
>
> There are two things that you should be doing:
>
>  1. describe the data that you want a client to acquire (the fun stuff)
>  2. describe how a client might find where information is (the dull and messy stuff)
>  3. describe how a client might ask for information, if necessary (the yucky stuff)
>
> If you define formats, and give them MIME types you get part 1.
>
> If you then define meta-formats you get part 2.
>
> If you define a query format for cost map filtering and anything that needs it, you get part 3.

I think this is a good summarization of the issues.  Coupled with your
examples below, this is a good illustration of an alternate
specification of the protocol.

>
> Do it right and you don't have to worry about caching, or the sorts of issues that you are trying to re-invent the wheel for (for example: the ServerA ServerB load sharing example in Rich's presentation just now; load sharing does not need to be visible to clients.  There is lots of experience in building scalable HTTP infrastructure.).
>

I'm not sure the proposed solution will handle the redistribution case
discussed in Section 8.1.1.1 of draft-ietf-alto-protocol-06 (perhaps
you can convince me otherwise).  The use case is about retrieving ALTO
information from another ALTO Client (not via HTTP).  Since it comes
from another, perhaps un-trusted, ALTO Client, the information itself
is digitally signed by the original ALTO Service.  The Service ID is
simply an identifier for the Service which might be implemented by
multiple physical servers without relying on each service being behind
a single DNS name.  Perhaps I am missing it, but perhaps you could
suggest how making the ALTO Protocol REST-ful might handle this use
case as well?

> That means that extensions the like of what we're seeing might be done without having to talk about protocols.  All you do is define the format and how that format is identified in the meta-format and discovery parts.
>
> Something like:
>
>  Server has document that lists
>   where a cost map is found
>   Where a filtered cost map might be acquired by providing filtering criteria
>   Where an endpoint map is found
>   And so forth for each of the types you have, plus all the extensions that the server might support
>
>    </alto/costmap>;title="Complete network cost map";type="application/alto-costmap"
>    </alto/cost?{ip}>;title="Filtered cost map";type="application/alto-costmap"
>    <http://other.server/alto/ep>;title="Complete endpoint map";type="application/alto-epmap"
>
>  At the end of each of those links you either find a resource that contains the desired document.
>
>  You use content negotiation and MIME for version negotiation (you don't need to specific versioning stuff that you have already)
>
>  You can also use MIME for identifying the cost types to help a client in selecting the right URL:
>
>    application/alto-costmap;type=hopcount;mode=ordinal
>
>  Clients can share links (deep links might stop working occasionally, which would necessitate re-discovery, but that's not a big deal).
>
>  HTTP caches (and CDNs...) can redistribute the data
>
> Have a look at RFC 5988 and draft-core-linking for information on how you might provide a collection of links.
>
> Cheers,
> Martin
> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto
>

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 5 Jan 2011 08:39:52 +0800

Hi Rich,

Regarding complexity, there are some things that are unavoidable.  Essential complexity exists for describing formats, as well as the filtering service (that arises because you are optimizing - one of the most common sources of complexity :)

By re-using existing work, you also gain experience in dealing with known problems rather than having to tackle them for yourself.  The linking work is a good example of that.  As is MIME+conneg for versioning.

> suggest how making the ALTO Protocol REST-ful might handle this use

I'm not sure if I'm just not understanding the problem, so let me describe what I think that you are trying to achieve.

The problem is where information travels from Server A to Client A to Client B.  Client B finds that they don't trust Client A to pass the information on faithfully.  You wish to have a way for Client B to validate that the information is correct.  Right?

Let's do away with the idea that Server A and Server B have different identities.  Logically, to the clients, there is a single authority that is providing this information.  The resource has a single identity (a URI).

Load sharing and content distribution don't need to play into this.  Any server that is reached by following a URI MUST be authoritative for that resource, whether it be load balanced through DNS, anycast or reverse proxy.  That server can choose to redirect as it chooses without compromising the integrity of the result.

The task then is in determining if a particular representation is faithful for that resource.  You are talking of an optimization that doesn't result in downloading the entire representation all over.

Digital signatures are one way of checking, but there are other options:

  - MD5 is pretty weak by modern standards, but a HEAD request to the URI could produce a Content-MD5 header.
  - Use a link header (and a HEAD request) to include a link to a digest if MD5 is no good.
  - Wait for someone else to invent some better HTTP integrity mechanism.  Mark Nottingham is interested in designing something for more general caching applications.

I'm going to advise that the latter is the course you want to follow, perhaps leaving other options open in the meantime.  No point in inheriting more complexity than absolutely necessary.

--Martin

---

From: Richard Alimi <[email-masked]>
Date: Wed, 5 Jan 2011 00:38:02 -0800

> Regarding complexity, there are some things that are unavoidable.  Essential complexity exists for describing formats, as well as the filtering service (that arises because you are optimizing - one…

Agreed.  I just meant to point out that I didn't think it would
significantly slim down the document, but it your suggestions do have
their benefits beyond that.

> By re-using existing work, you also gain experience in dealing with known problems rather than having to tackle them for yourself.  The linking work is a good example of that.  As is MIME+conneg fo…

Agreed.

> The problem is where information travels from Server A to Client A to Client B.  Client B finds that they don't trust Client A to pass the information on faithfully.  You wish to have a way for Cli…

Yes.

> I'm going to advise that the latter is the course you want to follow, perhaps leaving other options open in the meantime.  No point in inheriting more complexity than absolutely necessary.

Each of these three options requires the ALTO Client to send a message
to the HTTP server to verify, right?  One of the goals of
redistribution is to allow ALTO Clients to not contact an ALTO Server
at all in the common case. With the existing solution, ALTO Clients
need only download a certificate upon initial contact with the server,
and can then verify ALTO Responses without contacting the server at
all.  http://tools.ietf.org/html/draft-ietf-alto-protocol-06#section-12.4
has some additional discussion on this.

Rich

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 6 Jan 2011 06:56:17 +0800

On 2011-01-05 at 19:38:02, Richard Alimi wrote:
> >  - Wait for someone else to invent some better HTTP integrity
> mechanism.  Mark Nottingham is interested in designing something for
> more general caching applications.
> >
> > I'm going to advise that the latter is the course you want to follow,
> perhaps leaving other options open in the meantime.  No point in
> inheriting more complexity than absolutely necessary.
> >
> 
> all.  Section 12.4 has some additional discussion on this.

There's a fair difference between asking for data (and all that entails) and just asking for metadata in terms of denial of service.

If, however, redistribution is a fundamental part of the architecture, then there's a lot of work to be done in getting that right.  

I'm not the security guy, but that signature mechanism looks like it needs some work.  I like the fact that you have put the signature in metadata, but how does a client decide which of the many available public keys to use?  There are potentially multiple sources that can provide a public key; before redirection comes into play.

From an HTTP perspective, I think that you will find that trailers are poorly supported.  Current advice indicates that they have to be able to be ignored (see [1]), which might interfere with your use case - though this is definitely the right way to use trailers.  At a minimum, drop the PEM encoding, it doesn't play well with HTTP headers, nor does it really make sense: you just have a numeric value, so base64 alone should suffice.

On balance, I'd still encourage you to investigate this separately to the rest of the protocol.  You might find that you can get someone else to do the hard work for you.  I also suspect that this sort of thing could be a sticking point in the many reviews this work has to navigate; don't let a small thing prevent the bigger thing from happening.

--Martin

[1] http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-12#page-36

---

From: Richard Alimi <[email-masked]>
Date: Wed, 5 Jan 2011 22:32:12 -0800

Hi Martin,

> I'm not the security guy, but that signature mechanism looks like it needs some work.  I like the fact that you have put the signature in metadata, but how does a client decide which of the many av…

I think Section 8.1.3 should answer the question here. Was it still
ambiguous after reading this?  If so, we can certainly clarify it.

> From an HTTP perspective, I think that you will find that trailers are poorly supported.  Current advice indicates that they have to be able to be ignored (see [1]), which might interfere with your use case - though this is definitely the right way to use trailers.
>

That's unfortunate.  I think the requirements here are that an ALTO
Server should be able to generate and an append a signature even for a
dynamically-generated response which is streamed (as opposed to
buffered) to a client.  An alternative was to put the signature as
some trailing portion of the JSON message (e.g., the last member of
the outer-most object), but that seemed ugly since it seemed to
require signing a subset of the JSON message, thus requiring us to
define some canonical form that would be used for signature
verification.  Do you know of other alternatives beyond these two?

>  At a minimum, drop the PEM encoding, it doesn't play well with HTTP headers, nor does it really make sense: you just have a numeric value, so base64 alone should suffice.
>

Thanks - we can update that.

> On balance, I'd still encourage you to investigate this separately to the rest of the protocol.  You might find that you can get someone else to do the hard work for you.  I also suspect that this…

Splitting it out to a separate draft would be okay I think. Other
feedback from the WG here?

Rich

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 7 Jan 2011 06:03:20 +0800

On 2011-01-06 at 17:32:12, Richard Alimi wrote:
> > I'm not the security guy, but that signature mechanism looks like it
> needs some work.  I like the fact that you have put the signature in
> metadata, but how does a client decide which of the many available
> public keys to use?  There are potentially multiple sources that can
> provide a public key; before redirection comes into play.
> >
> 

These comments were based on 8.1.3.1: you present a choice, which is - as I understand it - not always a good thing when you are looking for some security property.
 
> > From an HTTP perspective, I think that you will find that trailers
> are poorly supported.  Current advice indicates that they have to be
> able to be ignored (see [1]), which might interfere with your use case
> - though this is definitely the right way to use trailers.
> 

CMS (formerly S/MIME; RFC 5652 from memory) might work, though I'm certainly not across all the complexities.  By all means avoid enveloped signatures, you really don't want to deal with canonicalization, even if JSON is easier than XML, it's still non-trivial.

---

From: Richard Alimi <[email-masked]>
Date: Fri, 7 Jan 2011 22:49:16 -0800

> These comments were based on 8.1.3.1: you present a choice, which is - as I understand it - not always a good thing when you are looking for some security property.

Perhaps it was unclear because the following sentence was broken onto
the following page:
  "To reduce requirements on the underlying transport (i.e., requiring
   SSL/TLS), the ALTO Protocol uses the latter option."
I'd agree this should be clarified to only state upfront the one that
the protocol actively uses (and either avoid mentioning the
alternative, or push the mention of the alternative elsewhere such as
additional discussion in Security Considerations).  I've made a note
of this.

> CMS (formerly S/MIME; RFC 5652 from memory) might work, though I'm certainly not across all the complexities.

Thanks for pointing this out.  Looking at it briefly, my understanding
of the encoding is that we either define an alternate query (or add a
query-string parameter to each query) to request signed ALTO
Responses, or make all ALTO Responses be encapsulated using CMS (which
still allows them to be signed or unsigned).  If we were to make use
of CMS, of those two options, the latter seems better to me. OpenSSL
appears to support CMS so there is certainly existing library support
out there already.

Between, HTTP trailers and CMS, I don't have a strong opinion.
Trailers may have issues in certain implementations/deployments, but
CMS introduces an additional piece to the puzzle (only in the
encoding, though).  Feedback from others would be useful here.

> By all means avoid enveloped signatures, you really don't want to deal with canonicalization, even if JSON is easier than XML, it's still non-trivial.
>

Agreed. Thanks for confirming that intuition :)

Rich

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 10 Jan 2011 06:17:56 +0800

On 2011-01-08 at 17:49:16, Richard Alimi wrote:
> > CMS (formerly S/MIME; RFC 5652 from memory) might work, though I'm
> certainly not across all the complexities.
> 

You could simply use conneg and include one format by default and the other if Accept headers indicated a preference for it.

--Martin