Subject: Re: [jose] Splitting headers

From: "Tschofenig, Hannes (NSN - FI/Espoo)" <[email-masked]>
Date: Mon, 20 Aug 2012 11:16:45 +0300

Here is my view: the muliple recipient case is a rare case. Having a bit of extra overhead for rare cases is IMHO acceptable to keep the core spec simpler. 

hannes

Sent from my Windows Phone

-----Original Message-----
From: ext Richard L. Barnes
Sent: 8/18/2012 12:58 AM
To: [email-masked]
Subject: [jose] Splitting headers

Hey all,

At IETF 84, I suggested that we take another look at what header information gets integrity-protected and what doesn't.  The main issue being that in multiple-recipient or multiple-signer scenarios, there's unnecessary duplication of data if recipient- or signer-specific information is in the integrity-protected header.

Following some on what Bob Wyman proposed this afternoon, I would like to propose that we split the set of header fields into some logical groupings:
1. Top-level fields that require integrity protection (e.g., cty or a digest algorithm parameter)
2. Top-level fields that do not require integrity protection (e.g., x5c)
3. Fields that are specific to a signer (e.g., jwk)
4. Fields that are specific to a recipient (e.g., epk)

In this taxonomy, only fields of type (1) need to be base64-encoded for to preserve their serialization, and fields of type (3) and (4) could be gathered together into the moral equivalent of SignerInfo and RecipientInfo objects.  All this makes the JSON serialization a lot cleaner, for example:

{
    "type": "S",
    "header": "[base64-type1]",
    "params": { /* type2 */ },
    "content": "[base64-content]",
    "signers": [
        { /* type3 */ }
    ]
}

You can also imagine a reasonable compact serialization, something like:

S.[base64-type1].[base64-type2].[base64-content].[base64-signer1].[base64-signer2]. ...

That encoding even has the benefit that the multi-signer case doesn't add any overhead over the single-signer case, except for maybe a couple of "." characters. (Obviously, you would want some structure on how the signer info gets serialized, but that's another level of detail down.  You could imagine something like "[signer-header].[public-key].[signature-value]".)

The additional structure might seem like it's more work to parse, but I don't think it's really that bad.  Given a parsed JOSE object "obj":
1. Decode obj.header and add its fields to obj.params
2. Look through "signers" / "recipients" to find one we understand
3. Verify / decrypt content using signer/recipient info and parameters

There are several details to be worked out -- most critically, which fields go in which groupings -- but I thought I would throw out the general idea for comment before diving into the details.

Cheers,
--Richard

_______________________________________________
jose mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jose

---

From: "Richard L. Barnes" <[email-masked]>
Date: Mon, 20 Aug 2012 08:21:41 -0400

Hannes,

By what metric of "rare"?  In the XMPP case, different appearances of a JID count as multiple recipients (at the crypto layer), and in XMPP, multiple appearance is very common (e.g., laptop vs. phone).

--Richard

---

From: Justin Richer <[email-masked]>
Date: Thu, 23 Aug 2012 11:17:42 -0400

I'd like to pose the question of what constitutes "a bit of extra 
overhead". Making the overhead mandatory to support in the core can lead 
to developer friction for the majority cases.

This reminds me of the discussion on the OAuth list about returning 
multiple tokens. Sometimes, yes, you want to return multiple access 
tokens. Usually it's just one. We could have made the return value an 
array instead of a string, but then you'd force every client to pull out 
"access_token[0]" instead of just "access_token", and that's an 
annoyance for the non-rare case. In the end, the group decided on 
supporting the simple case in the core and letting an extension define 
the multi-token format. That extension was never proposed, so I would 
wager that the multi-token case was at least rare enough to not force 
the standardization question.

I think the same approach should be taken here. Keep the existing, 
deployed, simple format of header.payload.signature for the simple 
single-signature case. Use a different serialization format for 
different use case, such as the JWS-JSON format that Mike's proposed or 
the multi-dot format proposed below. I would argue that they're 
fundamentally different kinds of objects and that conflating their use 
cases by forcing the overhead is a big mistake.

  -- Justin

On 08/20/2012 04:16 AM, Tschofenig, Hannes (NSN - FI/Espoo) wrote:
> Here is my view: the muliple recipient case is a rare case. Having a 
> bit of extra overhead for rare cases is IMHO acceptable to keep the 
> core spec simpler.
>
> hannes
>
> ------------------------------------------------------------------------
> At IETF 84, I suggested that we take another look at what header 
> information gets integrity-protected and what doesn't.  The main issue 
> being that in multiple-recipient or multiple-signer scenarios, there's 
> unnecessary duplication of data if recipient- or signer-specific 
> information is in the integrity-protected header.
>
> Following some on what Bob Wyman proposed this afternoon, I would like 
> to propose that we split the set of header fields into some logical 
> groupings:
> In this taxonomy, only fields of type (1) need to be base64-encoded 
> for to preserve their serialization, and fields of type (3) and (4) 
> could be gathered together into the moral equivalent of SignerInfo and 
> RecipientInfo objects.  All this makes the JSON serialization a lot 
> cleaner, for example:
>
> {
>     "type": "S",
> That encoding even has the benefit that the multi-signer case doesn't 
> add any overhead over the single-signer case, except for maybe a 
> couple of "." characters. (Obviously, you would want some structure on 
> how the signer info gets serialized, but that's another level of 
> detail down.  You could imagine something like 
> "[signer-header].[public-key].[signature-value]".)
>
> The additional structure might seem like it's more work to parse, but 
> I don't think it's really that bad.  Given a parsed JOSE object "obj":
> There are several details to be worked out -- most critically, which 
> fields go in which groupings -- but I thought I would throw out the 
> general idea for comment before diving into the details.
>
> Cheers,
> --Richard
>
>
>
>

---

From: Hannes Tschofenig <[email-masked]>
Date: Thu, 23 Aug 2012 18:54:42 +0300

I agree with you, Justin. 

My impression is that the multi-party communication and multi-party security scenario will either have to live with additional overhead (Richard, how much overhead are we talking about) or (what is more likely) there will be a different mechanism used (which is what happened with security for multi-party communication in the IETF in general). 

Ciao
Hannes

On Aug 23, 2012, at 6:17 PM, Justin Richer wrote:

> I'd like to pose the question of what constitutes "a bit of extra overhead". Making the overhead mandatory to support in the core can lead to developer friction for the majority cases. 
> 
> This reminds me of the discussion on the OAuth list about returning multiple tokens. Sometimes, yes, you want to return multiple access tokens. Usually it's just one. We could have made the return value an array instead of a string, but then you'd force every client to pull out "access_token[0]" instead of just "access_token", and that's an annoyance for the non-rare case. In       the end, the group decided on supporting the simple case in the core and letting an extension define the multi-token format. That extension was never proposed, so I would wager that the multi-token case was at least rare enough to not force the standardization question.
> 
> I think the same approach should be taken here. Keep the existing, deployed, simple format of header.payload.signature for the simple single-signature case. Use a different serialization format for different use case, such as the JWS-JSON format that Mike's proposed or the multi-dot format proposed below. I would argue that they're fundamentally different kinds of objects and that conflating their use cases by forcing the overhead is a big mistake.
> 
>  -- Justin
>

---

From: Brian Campbell <[email-masked]>
Date: Thu, 23 Aug 2012 10:01:32 -0600

I'll reiterate that I'd very much like to see the single signer and single
recipient cases remain as simple as possible.

On Thu, Aug 23, 2012 at 9:54 AM, Hannes Tschofenig <
[email-masked]> wrote:

> My impression is that the multi-party communication and multi-party
> security scenario will either have to live with additional overhead
> (Richard, how much overhead are we talking about) or (what is more likely)
> there will be a different mechanism used (which is what happened with
> security for multi-party communication in the IETF in general).
>
> Ciao
> Hannes
>
>
> overhead". Making the overhead mandatory to support in the core can lead to
> developer friction for the majority cases.
> >
> multiple tokens. Sometimes, yes, you want to return multiple access tokens.
> Usually it's just one. We could have made the return value an array instead
> of a string, but then you'd force every client to pull out
> "access_token[0]" instead of just "access_token", and that's an annoyance
> for the non-rare case. In       the end, the group decided on supporting
> the simple case in the core and letting an extension define the multi-token
> format. That extension was never proposed, so I would wager that the
> multi-token case was at least rare enough to not force the standardization
> question.
> >
> single-signature case. Use a different serialization format for different
> use case, such as the JWS-JSON format that Mike's proposed or the multi-dot
> format proposed below. I would argue that they're fundamentally different
> kinds of objects and that conflating their use cases by forcing the
> overhead is a big mistake.
> >
> >  -- Justin
> >
> bit of extra overhead for rare cases is IMHO acceptable to keep the core
> spec simpler.
> >>
> >> hannes
> >>
> >> In this taxonomy, only fields of type (1) need to be base64-encoded for
> to preserve their serialization, and fields of type (3) and (4) could be
> gathered together into the moral equivalent of SignerInfo and RecipientInfo
> objects.  All this makes the JSON serialization a lot cleaner, for example:
> >>
> >> {
> >>     "type": "S",
> add any overhead over the single-signer case, except for maybe a couple of
> "." characters. (Obviously, you would want some structure on how the signer
> info gets serialized, but that's another level of detail down.  You could
> imagine something like "[signer-header].[public-key].[signature-value]".)
> >>
> >> The additional structure might seem like it's more work to parse, but I
> don't think it's really that bad.  Given a parsed JOSE object "obj":
> fields go in which groupings -- but I thought I would throw out the general
> idea for comment before diving into the details.
> >>
> >> Cheers,
> >> --Richard
> >>
> >>
> >>
> >>

---

From: Nat Sakimura <[email-masked]>
Date: Fri, 24 Aug 2012 01:24:57 +0900

On Fri, Aug 24, 2012 at 12:17 AM, Justin Richer <[email-masked]> wrote:

> for the non-rare case. In the end, the group decided on supporting the
> simple case in the core and letting an extension define the multi-token

Actually, we decided to do it as an extension, called OpenID Connect.
Userinfo endpoint actually returns multiple tokens.

> overhead is a big mistake.

+1

Note: I have a multi-signature use case that I have to solve, but I believe
for the adoption purpose, we should be focused on the main use cases.

>  Here is my view: the muliple recipient case is a rare case. Having a bit
> of extra overhead for rare cases is IMHO acceptable to keep the core spec
> simpler.
>
> hannes
>
>  ------------------------------
> Following some on what Bob Wyman proposed this afternoon, I would like to
> propose that we split the set of header fields into some logical groupings:
> In this taxonomy, only fields of type (1) need to be base64-encoded for to
> preserve their serialization, and fields of type (3) and (4) could be
> That encoding even has the benefit that the multi-signer case doesn't add
> any overhead over the single-signer case, except for maybe a couple of "."
> characters. (Obviously, you would want some structure on how the signer
> jose mailing [email-masked]://www.ietf.org/mailman/listinfo/jose
>
>
>

-- 
Nat Sakimura (=nat)
Chairman, OpenID Foundation
http://nat.sakimura.org/
@_nat_en

---

From: Justin Richer <[email-masked]>
Date: Thu, 23 Aug 2012 13:01:33 -0400

> On Fri, Aug 24, 2012 at 12:17 AM, Justin Richer <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     That extension was never proposed, so I would wager that the
>     multi-token case was at least rare enough to not force the
>     standardization question.
>
>

Just to clarify -- this isn't quite the same thing. What folks had been 
asking about was not multiple different kinds of tokens at once, which 
the existing JSON structure makes amenable, but multiple parallel access 
tokens, each with their own values and (potentially) metadata. The two 
tokens that Connect returns have different purposes, different 
audiences, potentially very different formats and structures. This is 
why they come back as two tokens in different members, "access_token" 
and "id_token". What was discussed on the group was more the case where 
you'd have an array of "access_token" values.

>     different use case, such as the JWS-JSON format that Mike's
>     proposed or the multi-dot format proposed below. I would argue
>     that they're fundamentally different kinds of objects and that
>     conflating their use cases by forcing the overhead is a big mistake.
>
>
> +1
>
> Note: I have a multi-signature use case that I have to solve, but I 
> believe for the adoption purpose, we should be focused on the main use 
> cases.

Right, there are definitely use cases for this, but I think that it's a 
different solution that's needed here. There isn't always a nice, single 
general abstraction.

  -- Justin

>>     Here is my view: the muliple recipient case is a rare case.
>>     Having a bit of extra overhead for rare cases is IMHO acceptable
>>     to keep the core spec simpler.
>>
>>     hannes
>>
>>     To: [email-masked] <mailto:[email-masked]>
>>     information gets integrity-protected and what doesn't.  The main
>>     issue being that in multiple-recipient or multiple-signer
>>     scenarios, there's unnecessary duplication of data if recipient-
>>     or signer-specific information is in the integrity-protected header.
>>
>>     Following some on what Bob Wyman proposed this afternoon, I would
>>     like to propose that we split the set of header fields into some
>>     logical groupings:
>>     In this taxonomy, only fields of type (1) need to be
>>     base64-encoded for to preserve their serialization, and fields of
>>     type (3) and (4) could be gathered together into the moral
>>     equivalent of SignerInfo and RecipientInfo objects.  All this
>>     makes the JSON serialization a lot cleaner, for example:
>>
>>     {
>>         "type": "S",
>>     That encoding even has the benefit that the multi-signer case
>>     doesn't add any overhead over the single-signer case, except for
>>     maybe a couple of "." characters. (Obviously, you would want some
>>     structure on how the signer info gets serialized, but that's
>>     another level of detail down.  You could imagine something like
>>     The additional structure might seem like it's more work to parse,
>>     but I don't think it's really that bad.  Given a parsed JOSE
>>     object "obj":
>>     There are several details to be worked out -- most critically,
>>     which fields go in which groupings -- but I thought I would throw
>>     out the general idea for comment before diving into the details.
>>
>>     Cheers,
>>     --Richard
>>
>>
>>
>>
>>     [email-masked] <mailto:[email-masked]>
>>     [email-masked]  <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Justin Richer <[email-masked]>
Date: Thu, 23 Aug 2012 13:25:24 -0400

[pulling the jose list back on the thread]

I agree that too much simplicity can be as much as an impediment as too 
much complexity, but I don't think we've hit that here. I could of 
course be wrong, but the single-signature use case seems to be pretty 
well-established. I'm not arguing that you should never make a breaking 
change during the specification process, but that any breaking change, 
especially one so deeply fundamental as this, needs to have very strong 
support in both group consensus and running code.

So with that in mind: What's the cause for splitting payload and 
signature headers? What kind of information constitutes a "payload 
header" anyway? Just about every claim that's defined in the header 
today is for the signature or encryption. And you need to protect all of 
them with the signature in either case.

  -- Justin

On 08/23/2012 01:01 PM, Bob Wyman wrote:
> Simplicity is good, but sometimes too much simplicity can prevent us 
> from doing things that would be valuable to our eventual users. In 
> this case, I will admit that I am asking for a bit of additional 
> complexity -- but, it should be recognized as well that I am also 
> calling for additional simplicity in the structuring of the data. 
> Rather than mixing things up, I suggest separating them into logical 
> processing groups.
>
> I argue that instead of:
> <header>.<payload>.<signature>
> we should have:
> <payload_header>.<payload>.<signature_header>.<signature>
> where payload header contains only that data which is specific to the 
> payload and the signature_header contains only data relevant to the 
> signature. Given that the content of the signature_header is what is 
> currently already defined as part of the <header>, then the added 
> message size is only several bytes. I cannot see that there is much 
> added complexity in the code, since any code written will typically 
> first concern itself with the payload related data, wherever it is, 
> and then concern itself with the signature related stuff. Frankly, it 
> looks to me like while their might be more code to handle four parts 
> rather than three, the code itself would be cleaner and better 
> structured since it would inevitably tend to reflect the better 
> compartmentalization and separation of concerns in the encoded data. 
> One interesting side-effect of my "simplifying" proposal is that what 
> falls out of it is a simple way to transmit unsigned data. (i.e. 
> <payload_header>.<payload> without a signature.)...
>
> Certainly, my suggestion is motivated largely by the 
> multiple-signature use case (which is required for many applications), 
> but I am also motivated by a desire to see the data structures be 
> better normalized. And, I would like to do what can be done now to 
> simplify the process of defining extensions in the future (from a 
> standards process point of view) as well as simplicity when people 
> decide to adopt an extension.
>
> The current definition doesn't seem to me to provide much room for 
> extensions that would handle the multi-signature case. The only 
> alternative seems to be not an extension but rather a completely new 
> spec. Clearly, given the process overhead in IETF, that isn't going to 
> move along slowly. We would probably spend much more time debating the 
> extension/alternative as would be consumed in aggregate by all the 
> developers who needed to handle four parts rather than three... In any 
> case, the harsh reality is that non-core extensions are rarely 
> implemented if they aren't firmly rooted in the conventions of the 
> core specification -- no matter what the standard.
>
> How about this as a compromise?
> *Consider using the four-part format that I suggest, but stay silent 
> on the subject of multiple signatures or any other extensions.* Then, 
> i'll write up a draft that covers a variety of extensions that 
> leverage the four-part format -- including the multi-signature case. 
> If this strategy is used, the core spec will be better structured and 
> it will remain very "simple" -- but it will be simple in a way that 
> makes it vastly easier and cleaner to define extensions with minimal 
> divergence from the simple core. Developers will be served by this 
> approach since it should make it easier for them to incorporate 
> support for extensions in the future, if they choose to. And, users 
> will be served because it makes it easier for us to define extensions 
> that serve their needs.
>
> bob wyman
>
> On Thu, Aug 23, 2012 at 11:17 AM, Justin Richer <[email-masked] 
>     overhead". Making the overhead mandatory to support in the core
>     can lead to developer friction for the majority cases.
>
>     This reminds me of the discussion on the OAuth list about
>     returning multiple tokens. Sometimes, yes, you want to return
>     multiple access tokens. Usually it's just one. We could have made
>     the return value an array instead of a string, but then you'd
>     force every client to pull out "access_token[0]" instead of just
>     "access_token", and that's an annoyance for the non-rare case. In
>     the end, the group decided on supporting the simple case in the
>     core and letting an extension define the multi-token format. That
>     extension was never proposed, so I would wager that the

---

From: Bob Wyman <[email-masked]>
Date: Thu, 23 Aug 2012 15:13:57 -0400

As you point out, today's JWS_header is primarily concerned with the
signature rather than the payload. I'm suggesting that typ and cty should
be in a payload_header while all the rest of the stuff currently in
JWS_header should be in a signature_header. Thus, instead of the three
parts:
    <JWS_header>.<payload>.<signature>
we would have four parts:
    <payload_header>.<payload>.<signature_header>.<signature>
This is, I think, a cleaner separation of the various bits of metadata and
should lead to a better structured spec as well as better structured code
-- and a variety of easily defined extensions.

The multiple-signature case is the easiest one to understand. Imagine that
I have something like a document approval application that sends around
documents that need to be signed by multiple people. Given a three part
format, what I need to do is collect all three parts (including the
payload) from all signers. If the payload is large, this can be very
cumbersome. However, using the four-part format, it becomes trivial. I can
send a <payload_header>.<payload> pair out to reviewers and ask them to
send back their <signature_header>.<signature> pairs (or send back all four
parts and I'll throw away all but one <payload_header>.<payload> pair.
Then, once I've collected a few or few hundred signatures, I can then
easily and efficiently send out the signed payload with a potentially long
list of <signature_header>.<signature> pairs. This is easy, simple, etc.

Certainly, with the three part format, I can approximate the kind of
exchange I describe above by saying to signers that they should send me
their <JWS_header>.<signature> pairs and I'll then re-insert the payload to
make sense of the signatures. But, then I have to ensure that I also tell
the signers that they must not modify the typ or cty fields in the
<JWS_header> even though they would rewrite most other fields in that
header. Frankly, it would be much easier to simply say: Here is a
<payload_header>.<payload> pair. Sign it without modifying anything and
send me your <signature_header>.<signature> pair.

bob wyman

On Thu, Aug 23, 2012 at 1:25 PM, Justin Richer <[email-masked]> wrote:

> much complexity, but I don't think we've hit that here. I could of course
> be wrong, but the single-signature use case seems to be pretty
> So with that in mind: What's the cause for splitting payload and signature
> headers? What kind of information constitutes a "payload header" anyway?
> Just about every claim that's defined in the header today is for the
> signature or encryption. And you need to protect all of them with the
> signature in either case.
>
>  -- Justin
>
>
> Simplicity is good, but sometimes too much simplicity can prevent us from
> doing things that would be valuable to our eventual users. In this case, I
> will admit that I am asking for a bit of additional complexity -- but, it
> should be recognized as well that I am also calling for additional
> simplicity in the structuring of the data. Rather than mixing things up, I
> suggest separating them into logical processing groups.
>
>  I argue that instead of:
> currently already defined as part of the <header>, then the added message
> size is only several bytes. I cannot see that there is much added
> complexity in the code, since any code written will typically first concern
> itself with the payload related data, wherever it is, and then concern
> itself with the signature related stuff. Frankly, it looks to me like while
> their might be more code to handle four parts rather than three, the code
> itself would be cleaner and better structured since it would inevitably
> tend to reflect the better compartmentalization and separation of concerns
> in the encoded data. One interesting side-effect of my "simplifying"
> proposal is that what falls out of it is a simple way to transmit unsigned
> data. (i.e. <payload_header>.<payload> without a signature.)...
>
>  Certainly, my suggestion is motivated largely by the multiple-signature
> use case (which is required for many applications), but I am also motivated
> by a desire to see the data structures be better normalized. And, I would
> like to do what can be done now to simplify the process of defining
> extensions in the future (from a standards process point of view) as well
> as simplicity when people decide to adopt an extension.
>
> extensions that would handle the multi-signature case. The only alternative
> seems to be not an extension but rather a completely new spec. Clearly,
> given the process overhead in IETF, that isn't going to move along slowly.
> We would probably spend much more time debating the extension/alternative
> as would be consumed in aggregate by all the developers who needed to
> handle four parts rather than three... In any case, the harsh reality is
> that non-core extensions are rarely implemented if they aren't firmly
> rooted in the conventions of the core specification -- no matter what the
> standard.
>
> *Consider using the four-part format that I suggest, but stay silent on
> the subject of multiple signatures or any other extensions.* Then, i'll
> write up a draft that covers a variety of extensions that leverage the
> four-part format -- including the multi-signature case. If this strategy is
> used, the core spec will be better structured and it will remain very
> "simple" -- but it will be simple in a way that makes it vastly easier and
> cleaner to define extensions with minimal divergence from the simple core.
> Developers will be served by this approach since it should make it easier
> for them to incorporate support for extensions in the future, if they
> choose to. And, users will be served because it makes it easier for us to
> define extensions that serve their needs.
>
>  bob wyman
>
> On Thu, Aug 23, 2012 at 11:17 AM, Justin Richer <[email-masked]> wrote:
>