Subject: Re: [Acme] Signing HTTP Messages

From: Richard Barnes <[email-masked]>
Date: Fri, 19 Dec 2014 19:35:29 -0500

Hey Manu,

Thanks for reaching out.  Just so the context is clear, there's nothing in
ACME currently that uses an HTTP header to convey a signature structure.
It's all in the body.

The draft-cavage- document had been pointed out to me before.  It's too
ambitious :)  That draft tries to solve the problem of signing an HTTP
message.  That's a fiendishly difficult problem because it involves
headers, whose complicated, ambiguous syntax is inimical to signing.  Also,
middleboxes routinely tamper with headers.

All I want is to sign the body of the message.  HTTP treats the body as an
octet string, which means there's no c14n issues.  And middleboxes tamper
with bodies much less often.

Also, the draft-cavage- document invents its own signature syntax, when it
should just use JWS.

So my proposal was: Start with a Content-Signature header that just has a
JWS covering the body.  In some cases that's all you want, and it's a more
tractable problem than covering HTTP messages as a whole.  Then, if you
want to protect HTTP headers later, you can add a signed attribute to the
JWS (e.g., digest(canonicalized-header-info)).

Does that make sense?   Is that at all relevant to your use cases?

--Richard

On Fri, Dec 19, 2014 at 6:00 PM, Manu Sporny <[email-masked]>
wrote:
>
> > I like the idea of using a header like container for the signature.
> > It makes good architectural sense and it is easy to code. A signature
> > is logically meta-data and so it should be expressed as a header.
>
> Hey Phillip, Richard,
>
> I'm not on the ACME mailing list, nor do I have the bandwidth to follow
> the ACME discussions (even though I love what you guys are doing over
> there) but what you two are talking about here:
>
> http://www.ietf.org/mail-archive/web/acme/current/msg00125.html
>
> Sounds an awful lot like this (which has existed for years):
>
> http://tools.ietf.org/html/draft-cavage-http-signatures-03
>
> I might be missing something, but saw your conversation fly by and
> thought I'd mention it.
>
> -- manu
>
> --
> Manu Sporny (skype: msporny, twitter: manusporny, G+: +Manu Sporny)
> Founder/CEO - Digital Bazaar, Inc.
> blog: High-Stakes Credentials and Web Login
> http://manu.sporny.org/2014/identity-credentials/
>

---

From: Richard Barnes <[email-masked]>
Date: Sat, 20 Dec 2014 12:32:08 -0500

On Fri, Dec 19, 2014 at 9:44 PM, Manu Sporny <[email-masked]>
wrote:
>
> On 12/19/2014 07:35 PM, Richard Barnes wrote:
> > Thanks for reaching out.  Just so the context is clear, there's
> > nothing in ACME currently that uses an HTTP header to convey a
> > signature structure.  It's all in the body.
>
> Good to know. My assumption was that ACME is being built on top of JOSE.
>
> > The draft-cavage- document had been pointed out to me before.  It's
> > too ambitious :)
>
> Haha, we were trying to do something sensible, not ambitious. :)
>
> > That draft tries to solve the problem of signing an HTTP message.
> > That's a fiendishly difficult problem because it involves headers,
> > whose complicated, ambiguous syntax is inimical to signing. Also,
> Yes, both rabbit-holes were taken into account when designing the spec.
> Just to be clear, we're not trying to sign /all/ HTTP headers in a
> message, the sender gets to pick which headers to sign, the server gets
> to decide whether or not that's good enough for the type of request.
>
> The developer can choose specific headers that don't have ambiguous
> syntax and ones that middleboxes are most likely not to modify.
>
> I think we address these two concerns in the "Signing HTTP Messages"
> spec as best we can.
>
> > All I want is to sign the body of the message.  HTTP treats the body
> > as an octet string, which means there's no c14n issues.  And
> > middleboxes tamper with bodies much less often.
>
> I think you can do this easily with the Signing HTTP Messages spec. Just
> digitally sign the Digest header (along with a few others to prevent
> replay attacks if doing so is important to the use case).
>
> There's an example of doing this in this section:
>
> https://tools.ietf.org/html/draft-cavage-http-signatures-03#section-3.1.2
>
> > Also, the draft-cavage- document invents its own signature syntax,
> > when it should just use JWS.
>
> What do you mean by "signature syntax"? Do you mean "the way the bytes
> are constructed in order to sign them"?
>
> The normative bits of the Signing HTTP Messages spec are roughly 5 pages
> in length vs. the 50+ pages in JWS. It didn't seem necessary to pull in
> all that unnecessary bloat just for signature string construction.
>
> It's true that maybe the length of the JWS spec is due to necessary
> security precautions that the Signing HTTP Messages spec overlooks, but
> if it does overlook something, we're not aware of it.
>
> Happy to be convinced otherwise, but I'd rather not expose implementers
> to JWS or JOSE in general.
>
> > So my proposal was: Start with a Content-Signature header that just
> > has a JWS covering the body.
>
> Sounds like a good idea. I think you could simplify it further and make
> it easier to implement.
>
> > In some cases that's all you want, and it's a more tractable problem
> > than covering HTTP messages as a whole.
>
> I think you may not quite understand what we're trying to do w/ that
> spec. Or, I don't understand the nuance that you're getting at. Either
> way, we're miscommunicating. :)
>
> > Then, if you want to protect HTTP headers later, you can add a
> > signed attribute to the JWS (e.g.,
> > digest(canonicalized-header-info)).
> >
> What you're trying to do certainly makes sense, and if you want to use
> JWS to solve the problem, I guess you could do that. I still hold that
> the Signing HTTP Messages spec sounds like it could solve your problem
> in a much simpler way. For example, here's what a full solution might
> look like:
>
> ---------------------
> POST /foo HTTP/1.1
> Host: example.org
> Date: Tue, 07 Jun 2014 20:51:35 GMT
> Content-Type: application/json
> Digest: SHA-256=X48E9qOokqqrvdts8n...yWxBf7kbu9DBPE=
> Signature: keyId="Test",algorithm="rsa-sha256",
>   headers="(request-target) host date digest content-length",
>   signature="jgSqYK0yKclIHfF9zdA...NqNyAXKdxZZItOuhIs78w="
> Content-Length: 18
>
> {"hello": "world"}
> ---------------------
>

Just for context, here's the equivalent using JWS and protecting only the
body:

---------------------
POST /foo HTTP/1.1
Host: example.org
Date: Tue, 07 Jun 2014 20:51:35 GMT
Content-Type: application/json
Digest: SHA-256=X48E9qOokqqrvdts8n...yWxBf7kbu9DBPE=
Content-Signature: { "header": { "kid": "Test", "alg": "RS256" },
"signature": "jgSq...78w" }
Content-Length: 18

{"hello": "world"}
---------------------

Note that using JWS means that I can just toss the header into
JSON.parse(), whereas your construction requires custom parsing.  And if I
have a JOSE library already on hand (of which there are currently a
non-trivial number), I can just toss the whole thing in there and call
Verify().

Also, one thing I like about JWS is that you can make it self-verifying.
That is, you can replace the key ID ("kid") with the actual public key
("jwk"), so the verifier can just go ahead and verify without any
pre-provisioning of key IDs.  You can also include metadata like certs.
 (Yes, that starts to make the header a little big.  Deployment
consideration :) )

I wouldn't overestimate the complexity of JWS.  Lines of code is a better
metric than lines of spec.  It took me a couple of hours to write a JWS
library in Go; signing and verification take ~110 lines, most of which is
just algorithm mux/demux.
https://github.com/bifurcation/gose/blob/master/jws.go#L139

Finally, if you want to protect headers, you can add a protected header to
the above to bind in the header string by reference:

---------------------
{
  "header": { "kid": "Test", "alg": "RS256" },
  "protected": base64({
    "header": {
      "fields": "(request-target) host date digest content-length",
      "sha256": "KCBeV_VF5cEzgjY1cNEAkalw94mJwxtXdz874aRUbRM"
  })
  "signature": "jgSq...78w"
}
---------------------

... where the "sha256" field is the SHA-256 digest of the header string.

Is this making any more sense?  Basically: Use libraries and scale the
complexity to the problem.

--Richard

> Thoughts?
> blog: The Marathonic Dawn of Web Payments
> http://manu.sporny.org/2014/dawn-of-web-payments/
>

---

From: Richard Barnes <[email-masked]>
Date: Mon, 22 Dec 2014 22:43:20 -0500

On Mon, Dec 22, 2014 at 6:18 PM, Manu Sporny <[email-masked]>
wrote:
>
> On 12/20/2014 12:32 PM, Richard Barnes wrote:
> > Just for context, here's the equivalent using JWS and protecting only
> > the body:
> >
> > ---------------------
> > POST /foo HTTP/1.1
> > Host: example.org <http://example.org/>
> > JSON.parse(), whereas your construction requires custom parsing.
>
> True, although to be fair, it's the same kind of parsing that's required
> for most other HTTP headers (and it was specifically designed this way
> so that the HTTP header parsing mechanism can be re-used).
>
> > And if I have a JOSE library already on hand (of which there are
> > currently a non-trivial number), I can just toss the whole thing in
> > there and call Verify().
>
> Yep, that's nice, if you have already bought into the whole JWS/JOSE
> stack (which I haven't):
>
> http://manu.sporny.org/2013/sm-vs-jose/
>
> > Also, one thing I like about JWS is that you can make it
> > self-verifying.  That is, you can replace the key ID ("kid") with the
> > actual public key ("jwk"), so the verifier can just go ahead and verify
> > without any pre-provisioning of key IDs.  You can also include metadata
> > like certs.  (Yes, that starts to make the header a little big.
> > Deployment consideration :) )
>
> Yes, you could make it self-verifying that way, but that doesn't seem
> likely in practice? Maybe I'm missing something, but the server still
> needs some piece of information (like a key ID that it knows about) that
> it has access to that says that it trusts the signer in the first place,
> right?
>
> The approach we've taken w/ the Linked Data Signature stuff is to make
> kid/jwk a URL, like this:
>
> https://dev.payswarm.com/i/manu/keys/4
>
> That way, you can just fetch the public key as Linked Data and verify
> the message. Same basic process that you're talking about, but the
> approach is what we're currently using to identify keys for both Linked
> Data Signatures and Signing HTTP Messages.
>
> Here's an example of it in action (look at the Authorization HTTP header):
>
>
> http://opencreds.org/specs/source/identity-credentials/#writing-data-to-the-identity
>
> Also note how the same key is used in the "signature" at the end.
>
> > I wouldn't overestimate the complexity of JWS.  Lines of code is a
> > better metric than lines of spec.  It took me a couple of hours to write
> > a JWS library in Go; signing and verification take ~110 lines, most of
> > which is just algorithm mux/demux.
> The signing and verification implementation isn't the hard part w/ the
> JOSE stack, it's everything else you have to pull in. My point isn't
> that "it's possible", it's that "a simpler solution exists".
>
> Out of curiosity, where's the test suite for JOSE (and the
> implementations that pass the various specs)? We don't have an official
> one for the HTTP Signatures stuff yet, either, there's only this (and
> it's not that great):
>
> https://github.com/joyent/node-http-signature/tree/master/test
>
> I'm just trying to get a metric on how much I may be "overestimating the
> complexity of JWS". Maybe I am, but from a read of the specs, I think
> it's safe to say that doing HTTP Signatures requires far less code and
> technologies to be pulled in than the JOSE/JWS stack does. Happy to be
> proven wrong, because I can just drop this Signing HTTP Messages /
> Linked Data Signatures stuff if that turns out to be true. :)
>
> > Finally, if you want to protect headers, you can add a protected header
> > to the above to bind in the header string by reference:
> >
> > ---------------------
> > {
> Yes, you can protect headers like that, but aren't you just going to end
> up needing the equivalent of the Signing HTTP Messages spec at that
> point to specify how everything is serialized and SHA'd?
>
> Sure, it makes sense. As I said before, I think either approach works.
> I'm just trying to figure out which approach is better. So far, the
> arguments seem to be (warning: broad brush generalizations follow):
>
> JWS is better because it's done and there are multiple implementations
> for it. Just re-use what exists, don't invent something new.
>
> and
>
> JWS is worse because it pulls in a lot of complexity that you don't need
> to solve the particular problem you're trying to solve.
>
> I don't think either approach is going to break the Web or the Internet. :)
>

We certainly agree there :)

Without diving into the details above (holiday laziness setting in), I
actually think your broad brush outline is pretty accurate.  The main place
we differ is the complexity of JWS.

When you say "pull in complexity", what do you mean?  At least at the level
of code, all you need is:
-- A crypto library (which you need for Signing HTTP Messages)
-- A base64 encoder (which you need for Signing HTTP Messages)
-- A JSON parser
-- String concatenation (which you need for Signing HTTP Messages)

The signing process in JWS is wonky (some headers protected and some not;
base64.base64), but not rocket science.  And the key discovery has a lot of
options, but is ultimately not complicated -- you either find a key or you
don't.

And it seems like some of these things could be pinned down by using a
profile of JWS -- e.g., requiring all headers be protected and requiring
that at least one of "jwk" or "kid" be present.  That's pretty much what
I've put in for ACME (minus "kid"), and it's basically the same level of
complexity as a raw signature.

Maybe it would be a fun exercise for your winter vacation to try
implementing JWS to see how hard it really is :)

Happy holidays,
--Richard