Subject: [jose] Signing and Encryption algorithm

From: "Jim Schaad" <[email-masked]>
Date: Fri, 4 May 2012 20:36:56 -0700

<no hat>

I want to see if there is room for a discussion about the inputs used for
signature (and encryption?) computations.  The current algorithm was
designed with a specific mechanism in mind.  It is not clear that with the
new serialization methods that have been suggested that the current steps to
sign are the best.

The current code computes the signature hash as

HASH( BASE64( TEXT( header )) || "." || BASE64 ( body ))

>From what I know and can guess, there are three reasons for doing it this
way.

1.  We need to make sure that nothing can move the body to the headers and
vice versa
2.  There is a strong desire to avoid canonicalization of the inputs and
base64 is the method chosen to do this
3.  We already have the base64 values - so it makes sense to use them.

I would propose to change the hash computation to be

HASH( TEXT( header ) || body )

1.  There are no issues with the migration of data across the concatenation
operator.  Since the header is required to be a well-defined JSON object, it
is always possible to find the exact end of the JSON object in order to
separate the header and the body fields.  The only requirement would be that
the creation code would need to state that there are no trailing characters
following the closing curly brace.

2.  We do want to avoid canonicalization; however there is no effective
difference between hashing the data before or after applying the base64 data
string.  If one looks at what ALTO wants to do, they think that they do not
need to base64 encode the body of the message.  It will be transported
unchanged by http(s).  The only problem would be if the application were to
read in and then write out the JSON data.  That might change the bytes, but
the same issue would be true if you base64 encoded the byte stream as you
wrote it out.  

3.  In some cases the base64 values are not already present and would need
to be created as part of the processing.

Jim

---

From: Nat Sakimura <[email-masked]>
Date: Sat, 5 May 2012 08:09:41 -0700

Hi Jim,

Thanks for pointing it out.

I am not as confident as you are on getting the body unchanged through
all the transport and processing environment. True, it may be fine
with https, but we are not just targeting that. It may go through
various media including offline storage and by the time the JWE
processor gets to the data, it may have gone through bunch of
frameworks that may touch the whitespaces and eol chars that may end
up requiring us to do the canonicalization. Instead of risking that, I
would prefer staying on the safer side albeit it may incur a little
more overhead. The canonicalization is the primary thing that we want
to avoid.

Thoughts?

Nat Sakimura

On Fri, May 4, 2012 at 8:36 PM, Jim Schaad <[email-masked]> wrote:
> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose

-- 
Nat Sakimura (=nat)
Chairman, OpenID Foundation
http://nat.sakimura.org/
@_nat_en

---

From: "Jim Schaad" <[email-masked]>
Date: Sat, 5 May 2012 08:44:41 -0700

I am not suggesting that it should not be encoded as base64 for transport if
you are in a situation where it might be modified.  I think that how the
transport is done should be left up to the protocol using JOSE and not
specified by JOSE.  For the JWT serialization method, base64 will definitely
be used.

Jim

> -----Original Message-----
> From: Nat Sakimura [mailto:[email-masked]]
> Sent: Saturday, May 05, 2012 8:10 AM
> To: Jim Schaad
> Cc: [email-masked]
> Subject: Re: [jose] Signing and Encryption algorithm
> 
> Hi Jim,
> 
> I am not as confident as you are on getting the body unchanged through all
> the transport and processing environment. True, it may be fine with https,
> but we are not just targeting that. It may go through various media
including
> offline storage and by the time the JWE processor gets to the data, it may
> have gone through bunch of frameworks that may touch the whitespaces
> and eol chars that may end up requiring us to do the canonicalization.
Instead
> of risking that, I would prefer staying on the safer side albeit it may
incur a
> little more overhead. The canonicalization is the primary thing that we
want
> > I want to see if there is room for a discussion about the inputs used
> > for signature (and encryption?) computations.  The current algorithm
> > was designed with a specific mechanism in mind.  It is not clear that
> > with the new serialization methods that have been suggested that the
> > current steps to sign are the best.
> >
> > From what I know and can guess, there are three reasons for doing it
> > this way.
> >
> > 1.  We need to make sure that nothing can move the body to the headers
> > and vice versa 2.  There is a strong desire to avoid canonicalization
> > of the inputs and
> > base64 is the method chosen to do this 3.  We already have the base64
> > values - so it makes sense to use them.
> >
> > 1.  There are no issues with the migration of data across the
> > concatenation operator.  Since the header is required to be a
> > well-defined JSON object, it is always possible to find the exact end
> > of the JSON object in order to separate the header and the body
> > fields.  The only requirement would be that the creation code would
> > need to state that there are no trailing characters following the
closing curly
> brace.
> >
> > 2.  We do want to avoid canonicalization; however there is no
> > effective difference between hashing the data before or after applying
> > the base64 data string.  If one looks at what ALTO wants to do, they
> > think that they do not need to base64 encode the body of the message.
> > It will be transported unchanged by http(s).  The only problem would
> > be if the application were to read in and then write out the JSON
> > data.  That might change the bytes, but the same issue would be true
> > if you base64 encoded the byte stream as you wrote it out.
> >
> > 3.  In some cases the base64 values are not already present and would
> > need to be created as part of the processing.
> >
> > Jim
> >
> >

---

From: John Bradley <[email-masked]>
Date: Sat, 5 May 2012 11:24:48 -0700

Sorry I am traveling so responses may be delayed.

We have talked about this in the past, and come to the conclusion that of interoperability and simplicity of implementations calculating the hash over the encoded values was preferred.

I agree that on the face of it the hash calculated over the raw strings should produce be repeatable if the various upper layers do the right things.  

From an interoperability perspective having various transports encoding and decoding the binary seems like it will cause problems in deployments.

The other downside (Though not directly a IETF concern) is that this would be a breaking change to existing implementations of openID Connect, BrowserID and others.

I can't say I like the idea of having to do some non JSON string search to find the end of the header.

If we complicate things more than the existing practice of splitting on periods and base64 decoding we will get pushback from implementers.

I probably need to talk to more coders about this before I make a final decision on the wisdom of this.

So while not warm to the idea I will keep an open mind.

John B.

---

From: "Jim Schaad" <[email-masked]>
Date: Sat, 5 May 2012 11:59:56 -0700

> From: John Bradley [mailto:[email-masked]]
> Sent: Saturday, May 05, 2012 11:25 AM
> To: Jim Schaad
> Cc: 'Nat Sakimura'; [email-masked]
> We have talked about this in the past, and come to the conclusion that of
> interoperability and simplicity of implementations calculating the hash
over
> the encoded values was preferred.

I am not sure how much this is real - it will depend on how one creates
their library.  There is not a lot of difference between the following
pseudo codes

String hdrString = hdr.ToString();
String hdrB64String = Base64(hdrString);
Hash(hdrB64String);

And

String hdrString = hdr.ToString();
Hash(hdrString);
String hdrB64String = Base64(hdrString);

The question is where and when are the "to text" conversion and the base64
conversion done in the various libraries.

> 
> I agree that on the face of it the hash calculated over the raw strings
should
> produce be repeatable if the various upper layers do the right things.
> 
> From an interoperability perspective having various transports encoding
and
> decoding the binary seems like it will cause problems in deployments.

I might have chosen a bad word here.  When I say transport in this case I am
not talking about http or email, I am talking about the ALTO message object
or the JWT encoded object.  This is an issue that the container which is
using the JOSE work needs to deal with.  I am not saying that this is an
issue the HTTP or email needs to deal with.

> 
> The other downside (Though not directly a IETF concern) is that this would
be
> a breaking change to existing implementations of openID Connect,
> BrowserID and others.
> 
> I can't say I like the idea of having to do some non JSON string search to
find
> the end of the header.

I don't believe that I said this was ever going to be necessary.  However
there is a security issue which is almost surely not currently done in the
libraries.  When the header is parsed, the code needs to ensure that there
are no unused characters at the end of the input string.  If this is not the
case then there is a dead zone in which the signature algorithm can be
attacked.  This would also apply to a body if it is a JSON encoded object.

I expect that the header and the body are going to stay in distinct
locations and there is no need to try and separate them in any protocol.
The issue of making sure that there is no way for data to travel between the
header and the body is strictly a security consideration on using the hash
algorithm.  One needs to make sure that you are not in the situation of
having

ABCDEF	GHIJK
ABCD		EFGHIJK

If you ignore the white space, these two strings generate the same hash
value.   However the semantics are going to be different as the characters
are either part of the header or the body and that can change the meaning of
things.

Jim

> 
> If we complicate things more than the existing practice of splitting on
periods
> and base64 decoding we will get pushback from implementers.
> 
> > I am not suggesting that it should not be encoded as base64 for
> > transport if you are in a situation where it might be modified.  I
> > think that how the transport is done should be left up to the protocol
> > using JOSE and not specified by JOSE.  For the JWT serialization
> > method, base64 will definitely be used.
> >
> > Jim
> >
> >
> >> I am not as confident as you are on getting the body unchanged
> >> through all the transport and processing environment. True, it may be
> >> fine with https, but we are not just targeting that. It may go
> >> through various media
> > including
> >> offline storage and by the time the JWE processor gets to the data,
> >> it may have gone through bunch of frameworks that may touch the
> >> whitespaces and eol chars that may end up requiring us to do the
> canonicalization.
> > Instead
> >>> I want to see if there is room for a discussion about the inputs
> >>> used for signature (and encryption?) computations.  The current
> >>> algorithm was designed with a specific mechanism in mind.  It is not
> >>> clear that with the new serialization methods that have been
> >>> suggested that the current steps to sign are the best.
> >>>
> >>> 1.  We need to make sure that nothing can move the body to the
> >>> headers and vice versa 2.  There is a strong desire to avoid
> >>> canonicalization of the inputs and
> >>> base64 is the method chosen to do this 3.  We already have the
> >>> base64 values - so it makes sense to use them.
> >>>
> >>> well-defined JSON object, it is always possible to find the exact
> >>> end of the JSON object in order to separate the header and the body
> >>> effective difference between hashing the data before or after
> >>> applying the base64 data string.  If one looks at what ALTO wants to
> >>> do, they think that they do not need to base64 encode the body of the
> message.
> >>> 3.  In some cases the base64 values are not already present and
> >>> would need to be created as part of the processing.
> >>>
> >>> Jim
> >>>
> >>>

---

From: "Manger, James H" <[email-masked]>
Date: Mon, 7 May 2012 16:47:08 +1000

> following the closing curly brace.

I agree that it would be better to do the base64url-encoding outside of the MAC calculation.

Only issue 1 seems relevant. Requiring nothing after the closing "}" of the header might be the answer. That should be easy enough for signers to meet. However, it might be harder for verifiers to check (or they might "forget" to check).

I am sure many/most JSON decoders will (sensibly) ignore whitespace after the closing "}" -- it is allowed by the JSON spec. I wouldn't be surprised if some allowed C-style comments. Others might even allow arbitrary junk.

Base64url-encoding the header, but not the body, might satisfy more parties.

HASH( BASE64( TEXT( header )) || "." || body )

P.S. HMAC is not the only form of MAC. I hope the spec drops the unnecessary restriction to "Hash-based MACs".

--
James Manger