Subject: [jose] thoughts on header criticality from recent polls

From: Karen O'Donoghue <[email-masked]>
Date: Mon, 25 Feb 2013 06:25:40 -0500

Folks,

Thank you for your patience with me regarding the recent polls on header 
criticality. I realize I have taken too long to respond from a chairs 
perspective with some results and direction. I had hoped to provide 
either a decision or a clear set of choices, but after analyzing the 
poll results, rereading all the email traffic (multiple times), and 
talking to a few working group members, I find I am not quite able to do 
so. So, what next...

First, what I do believe...

I think it is clear that the working group remains divided on this issue 
with a significant number of members feeling that an approach must be 
defined that safely allows for some header fields to be considered 
non-critical. I think there is general consensus that there must be 
reasonable mechanisms for extensibility, and that the approach must be 
generic. Based on that, I would like to ask the working group to move 
forward with the strategy that all headers cannot be considered critical 
and move on to the discussion of how to approach non-critical headers.

In my review of the mailing list traffic (and my apologies if I missed 
something), I have seen a number of basic approaches proposed 
(summarized by Jim Schaad on 9 Oct 2012 with the response by James 
Manager on 10 Oct 2012). In particular, I found James' list of 
requirements to be very helpful, and I have repeated it here...

A. Want to be able to define messages with new semantics in future, 
without any risk that old implementations will misinterpret them with 
old semantics.

B. Want to be able to *redefine* (or constrain) the semantics of 
*existing* fields in future, without any risk that old implementations 
will misinterpret them with old semantics (or without the constraints).

C. Want to prevent large blobs of ignored data appearing in messages, as 
similar blobs have been used to create hash collisions between 
legitimate & malicious certificates that use weak algorithms (eg MD5).

D. Want to strongly discourage future extensions (feature-creep), under 
the philosophy that even well-intentioned extensions tend to do more 
harm (by adding complexity that is only deployed sporadically) than the 
good of any new functionality they bring.

E. Loosely coupling clients and servers is generally considered a good 
thing; but strongly coupling them is more appropriate for a secure 
message format for [insert a reason here]. A possible reason is that 
people are too likely to choose to make an extension non-critical for 
short-term ease of interoperability, instead of making it critical for 
better long-term security.

 From this list, I believe A is definitely a requirement. I am unclear 
regarding working group consensus on B, C, and E, and I believe D is not 
a requirement.

QUESTION: Is there working group consensus on the requirements we are 
trying to address? Perhaps a short discussion clarifying these would 
help drive consensus on the overall approach.

As for possible solutions, we appear to have evolved to the point where 
we are of two minds:

1) Define a header field that explicitly lists the fields that may be 
safely ignored if not understood. (Here I am assuming that we aren't 
going to define a separate header for non-critical elements.)

2) Remain silent on header criticality in the JOSE specifications and 
leave it up to the various application specifications (possibly 
providing guidance directed to those applications in the JOSE specs).

I sense that there is more support for the first approach, but I am 
concerned that there is a significant minority that favor the second 
approach.

QUESTION: Can the working group agree on whether to work on approach 1 
or approach 2?

In conclusion, I am asking three things in this missive:
1. I am asking the working group to move forward based on a non-critical 
headers approach.
2. I am asking the working group to clarify the requirements driving the 
non-critical header discussion (A, B, C, D, E, or ?).
3. I am asking the working group to decide whether the right approach is 
to develop a specific syntax (if so, what?) or to defer to the 
applications with appropriate guidance (if so, what level of guidance)? 
To address this item, I welcome concise proposals of text.

I realize this message is long and not as specific or prescriptive as 
some would like, and I apologize in advance for any mistakes and 
misunderstandings on my part.

Regards,
Karen O'Donoghue

---

From: Mike Jones <[email-masked]>
Date: Mon, 25 Feb 2013 15:41:20 +0000

Thanks for sending this, Karen.  Hopefully your analysis will help the working group reach a resolution on this issue soon.  I'm writing this note primarily as an editor to help inform the working group's discussions, rather than as an individual contributor.

First, I want to point out that while we've been discussing criticality of header parameters, the specs actually currently have the "MUST be understood" language in the JWK specification as well, as that's another potential extension point.  The JWK description says "Additional members MAY be present in the JWK.  If present, they MUST be understood by implementations using them."  The same language is present for the JWK Set.  For our resolution of the issue to be complete, it should address those uses too.  Given that a number of additional JWK fields are being discussed, I expect that extensions will occur there.

Second, if we do decide to go with your approach (1), I'll summarize for the list what I believe the two syntax proposals on the table are for doing this.  One proposal is 2A in http://www.ietf.org/mail-archive/web/jose/current/msg01353.html (which I'll call the FLAT SYNTAX) and the other is the syntax that Dick Hardt proposed in http://www.ietf.org/mail-archive/web/jose/current/msg01457.html (which I'll call the STRUCTURED SYNTAX).  I believe that it would be useful for the working group to discuss which of those they would prefer in parallel to discussing the questions that you asked, so that if we do go with approach (1) or something like it, we'll know which to use.

FLAT SYNTAX:
    {"alg":"ES256",
     "ign":["notes"],
     "notes":"This header field can be safely ignored"
    }

STRUCTURED SYNTAX:
    {"alg":"ES256",
     "ign":{
       "notes":"This header field can be safely ignored"
      }
    }

As I see it, the advantage of the flat syntax is all header fields are in the same place.  This makes processing header fields easier and is more regular.  The disadvantage is that the list of fields that may be safely ignored repeats those field names.

As I see it, the advantage of the structured syntax is that nothing is repeated, resulting in cleaner JSON syntax.  The disadvantage is that then some header fields are in one place and some are in another, making the processing rules for header fields more complicated.

Between these two, it seems to come down to a choice between cleaner syntax or cleaner semantics.  (I'd personally lean towards the simpler processing rules resulting from the flat syntax, but I would be fine with this coming out either way.)

Finally, I'll point out that I don't believe that your approaches (1) and (2) are mutually exclusive.  We could conceivably leave it up to application specifications using the JOSE specs whether they require header fields to be understood by default AND provide a syntax for declaring which fields may be safely ignored for use in those application contexts where the application spec does require that header fields be normally understood.  In fact, one might consider this to fall under the "(possibly providing guidance directed to those applications in the JOSE specs)" that you suggested below.

I hope the points above help further a productive discussion of this topic.

                                                            -- Mike

From: [email-masked] [mailto:[email-masked]] On Behalf Of Karen O'Donoghue
Sent: Monday, February 25, 2013 3:26 AM
To: [email-masked]
Subject: [jose] thoughts on header criticality from recent polls

Folks,

Thank you for your patience with me regarding the recent polls on header criticality. I realize I have taken too long to respond from a chairs perspective with some results and direction. I had hoped to provide either a decision or a clear set of choices, but after analyzing the poll results, rereading all the email traffic (multiple times), and talking to a few working group members, I find I am not quite able to do so. So, what next...

I think it is clear that the working group remains divided on this issue with a significant number of members feeling that an approach must be defined that safely allows for some header fields to be considered non-critical. I think there is general consensus that there must be reasonable mechanisms for extensibility, and that the approach must be generic. Based on that, I would like to ask the working group to move forward with the strategy that all headers cannot be considered critical and move on to the discussion of how to approach non-critical headers.

In my review of the mailing list traffic (and my apologies if I missed something), I have seen a number of basic approaches proposed (summarized by Jim Schaad on 9 Oct 2012 with the response by James Manager on 10 Oct 2012). In particular, I found James' list of requirements to be very helpful, and I have repeated it here...

A. Want to be able to define messages with new semantics in future, without any risk that old implementations will misinterpret them with old semantics.

B. Want to be able to *redefine* (or constrain) the semantics of *existing* fields in future, without any risk that old implementations will misinterpret them with old semantics (or without the constraints).

C. Want to prevent large blobs of ignored data appearing in messages, as similar blobs have been used to create hash collisions between legitimate & malicious certificates that use weak algorithms (eg MD5).

D. Want to strongly discourage future extensions (feature-creep), under the philosophy that even well-intentioned extensions tend to do more harm (by adding complexity that is only deployed sporadically) than the good of any new functionality they bring.

E. Loosely coupling clients and servers is generally considered a good thing; but strongly coupling them is more appropriate for a secure message format for [insert a reason here]. A possible reason is that people are too likely to choose to make an extension non-critical for short-term ease of interoperability, instead of making it critical for better long-term security.

>From this list, I believe A is definitely a requirement. I am unclear regarding working group consensus on B, C, and E, and I believe D is not a requirement.

QUESTION: Is there working group consensus on the requirements we are trying to address? Perhaps a short discussion clarifying these would help drive consensus on the overall approach.

As for possible solutions, we appear to have evolved to the point where we are of two minds:

1) Define a header field that explicitly lists the fields that may be safely ignored if not understood. (Here I am assuming that we aren't going to define a separate header for non-critical elements.)

2) Remain silent on header criticality in the JOSE specifications and leave it up to the various application specifications (possibly providing guidance directed to those applications in the JOSE specs).

I sense that there is more support for the first approach, but I am concerned that there is a significant minority that favor the second approach.

QUESTION: Can the working group agree on whether to work on approach 1 or approach 2?

In conclusion, I am asking three things in this missive:
1. I am asking the working group to move forward based on a non-critical headers approach.
2. I am asking the working group to clarify the requirements driving the non-critical header discussion (A, B, C, D, E, or ?).
3. I am asking the working group to decide whether the right approach is to develop a specific syntax (if so, what?) or to defer to the applications with appropriate guidance (if so, what level of guidance)? To address this item, I welcome concise proposals of text.

I realize this message is long and not as specific or prescriptive as some would like, and I apologize in advance for any mistakes and misunderstandings on my part.

Regards,
Karen O'Donoghue

---

From: Richard Barnes <[email-masked]>
Date: Mon, 25 Feb 2013 17:12:25 -0500

Hi Karen,

Thanks for these thoughts.

tl;dr: I'm OK with (1) or (2).  I would propose that we extend (1) by
saying that the non-critical headers would not be covered by the header
integrity check.

Detailed responses inline below.

--Richard

<snip>

> without any risk that old implementations will misinterpret them with old
> semantics.
>
> *existing* fields in future, without any risk that old implementations will
> misinterpret them with old semantics (or without the constraints).
>
> similar blobs have been used to create hash collisions between legitimate &
> malicious certificates that use weak algorithms (eg MD5).
>
> the philosophy that even well-intentioned extensions tend to do more harm
> (by adding complexity that is only deployed sporadically) than the good of
> any new functionality they bring.
>
> thing; but strongly coupling them is more appropriate for a secure message
> format for [insert a reason here]. A possible reason is that people are too
> likely to choose to make an extension non-critical for short-term ease of
> interoperability, instead of making it critical for better long-term
> security.
>
> regarding working group consensus on B, C, and E, and I believe D is not a
> requirement.
>
> trying to address? Perhaps a short discussion clarifying these would help
> drive consensus on the overall approach.
>

Here's my take:

A: This is a requirement.  But it is met trivially by having the new format
use new identifier names and marking them critical.

B: This is not a requirement.  It's a bad idea.  Identifiers here are
cheap, so if you want new semantics, just use a new identifier.

C: This is a requirement (and a good point), in the sense that we should
avoid providing channels that an attacker could use to control the
integrity check value on a JW* object.  However, what that argues is that
non-critical fields shouldn't be covered by the integrity check, not that
they should be ruled out altogether.

D: This is not a requirement, for the reasons you give.

E: This is not a requirement.  Couple of clients/servers is an
application-layer issue.

> leave it up to the various application specifications (possibly providing
> guidance directed to those applications in the JOSE specs).
>

I would personally be OK with either approach.  I would generalize (1)
slightly to say "Define a syntax for indicating which fields are critical /
non-critical", and, in light of (C) above, also propose that that syntax
remove non-critical fields from under the integrity check.

---

From: John Bradley <[email-masked]>
Date: Mon, 25 Feb 2013 14:16:31 -0800

Karen & Mike,

I think an important issue that Jim and others have tried to get at is some information in the header is intended for the JOSE library, while other header parameters are essentially additional associated data (meta-data about the body) and need to be passed on to other software layers for processing along with the body.   
 
An example of that was the proposed "iat" (issued at) element, which needs to be processed outside of the JOSE crypto processing to be useful.
 
I suspect a point of agreement might be having an "aad" header parameter that is a JSON object that MUST be ignored by the JOSE library and passed on with the body.  This would provide applications a way to pass meta-data through JOSE that is integrity protected, without JOSE having to parse it.
 
Once we remove a large number of the possible extension header parameters from JOSE processing, the problem gets smaller.
 
There is however, still the requirement that extensions for the header parameters JOSE processes be possible.
 
For that the list of non-critical header parameters is the best solution.  That enables a header parameter to be migrated from non-critical to critical without changing its JSON element name.
 
I think that a large number of people on the list favoured the single element with an array of element names that are non-critical.
 
My proposal is to add two new top level elements that MUST be understood:
 
1 "ign":  Ignore - an array of top level elements in the envelope that an application MUST ignore if it cannot process them.
2 "aad": Additional Associated Data - a JSON object that MUST be provided to applications along with the body if present in a JOSE header.
 
All other top level header parameters MUST be understood by a JOSE library or produce a processing fault resulting in an error.
 
I would also not want elements listed in "ign" to be passed on to applications.  I think that would cause interoperability issues.   That is what "aad" is intended to provide.
 
I know James has legitimate concerns about allowing potentially user provided data to be included in the header (Or anywhere likely).  I think this is a separate issue that needs to be addressed by security considerations to add entropy to the header if such data is included.   That may require a new "ent" Entropy element that MUST be ignored.  This should be a separate discussion.
 
Finally, for JWK, I believe that we should just state that not-understand elements should be ignored. When including a JWK header parameter this is also allows for extension to the JWK parameter, as the MUST understand in JWE/JWS only allies to the top level parameter's, and not the sub parameters they may contain.
 
My proposal doesn't neatly fit into fit into one of the poll answers, but I think it may address the majority of concerns.
 
Regards
John B.
On 2013-02-25, at 7:41 AM, Mike Jones <[email-masked]> wrote:

> Thanks for sending this, Karen.  Hopefully your analysis will help the working group reach a resolution on this issue soon.  I’m writing this note primarily as an editor to help inform the working group’s discussions, rather than as an individual contributor.
>  
> First, I want to point out that while we’ve been discussing criticality of header parameters, the specs actually currently have the “MUST be understood” language in the JWK specification as well, as that’s another potential extension point.  The JWK description says “Additional members MAY be present in the JWK.  If present, they MUST be understood by implementations using them.”  The same language is present for the JWK Set.  For our resolution of the issue to be complete, it should address those uses too.  Given that a number of additional JWK fields are being discussed, I expect that extensions will occur there.
>  
> Second, if we do decide to go with your approach (1), I’ll summarize for the list what I believe the two syntax proposals on the table are for doing this.  One proposal is 2A in http://www.ietf.org/mail-archive/web/jose/current/msg01353.html (which I’ll call the FLAT SYNTAX) and the other is the syntax that Dick Hardt proposed in http://www.ietf.org/mail-archive/web/jose/current/msg01457.html (which I’ll call the STRUCTURED SYNTAX).  I believe that it would be useful for the working group to discuss which of those they would prefer in parallel to discussing the questions that you asked, so that if we do go with approach (1) or something like it, we’ll know which to use.
>  
> FLAT SYNTAX:
>     {"alg":"ES256",
>      "ign":["notes"],
> Between these two, it seems to come down to a choice between cleaner syntax or cleaner semantics.  (I’d personally lean towards the simpler processing rules resulting from the flat syntax, but I would be fine with this coming out either way.)
>  
> Finally, I’ll point out that I don’t believe that your approaches (1) and (2) are mutually exclusive.  We could conceivably leave it up to application specifications using the JOSE specs whether they require header fields to be understood by default AND provide a syntax for declaring which fields may be safely ignored for use in those application contexts where the application spec does require that header fields be normally understood.  In fact, one might consider this to fall under the “(possibly providing guidance directed to those applications in the JOSE specs)” that you suggested below.
>  
> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose

---

From: Tim Bray <[email-masked]>
Date: Mon, 25 Feb 2013 14:17:42 -0800

One addition to Mike’s notes.  If you are going to have ignorable fields,
it’d be nice to avoid one of the (many, many) design botches that increased
the difficulty of doing SOAP; it had a thing where you could specify, in
among the SOAP headers, a MUST-UNDERSTAND rule. The effect was that you
didn’t know whether you could act on any header until you’d seen them all,
because there might be a MUST-UNDERSTAND lurking somewhere.

The "ign" : [ "notes" ] idiom has, to my eye, that drawback.  The
structured one lets you know, as soon as you see a field, whether it’s OK
to not understand it.  -T

On Mon, Feb 25, 2013 at 7:41 AM, Mike Jones <[email-masked]>wrote:

>  Thanks for sending this, Karen.  Hopefully your analysis will help the
> working group reach a resolution on this issue soon.  I’m writing this note
> primarily as an editor to help inform the working group’s discussions,
> rather than as an individual contributor.****
>
> ** **
>
> First, I want to point out that while we’ve been discussing criticality of
> header parameters, the specs actually currently have the “MUST be
> understood” language in the JWK specification as well, as that’s another
> potential extension point.  The JWK description says “Additional members
> MAY be present in the JWK.  If present, they MUST be understood by
> implementations using them.”  The same language is present for the JWK Set.
>  For our resolution of the issue to be complete, it should address those
> uses too.  Given that a number of additional JWK fields are being
> discussed, I expect that extensions will occur there.****
>
> ** **
>
> Second, if we do decide to go with your approach (1), I’ll summarize for
> the list what I believe the two syntax proposals on the table are for doing
> this.  One proposal is 2A in
> http://www.ietf.org/mail-archive/web/jose/current/msg01353.html (which
> I’ll call the FLAT SYNTAX) and the other is the syntax that Dick Hardt
> proposed in
> http://www.ietf.org/mail-archive/web/jose/current/msg01457.html (which
> I’ll call the STRUCTURED SYNTAX).  I believe that it would be useful for
> the working group to discuss which of those they would prefer in parallel
> to discussing the questions that you asked, so that if we do go with
> approach (1) or something like it, we’ll know which to use.****
>
> ** **
>
> FLAT SYNTAX:****
>
>     {"alg":"ES256",****
>
>      "ign":["notes"],****
>
>      "notes":"This header field can be safely ignored"****
>
>     }****
>
> ** **
>
> STRUCTURED SYNTAX:****
>
>     {"alg":"ES256",****
>
>      "ign":{****
>
>        "notes":"This header field can be safely ignored"****
>
>       }****
>
>     }****
>
> ** **
>
> As I see it, the advantage of the flat syntax is all header fields are in
> the same place.  This makes processing header fields easier and is more
> regular.  The disadvantage is that the list of fields that may be safely
> ignored repeats those field names.****
>
> ** **
>
> As I see it, the advantage of the structured syntax is that nothing is
> repeated, resulting in cleaner JSON syntax.  The disadvantage is that then
> some header fields are in one place and some are in another, making the
> processing rules for header fields more complicated.****
>
> ** **
>
> Between these two, it seems to come down to a choice between cleaner
> syntax or cleaner semantics.  (I’d personally lean towards the simpler
> processing rules resulting from the flat syntax, but I would be fine with
> this coming out either way.)****
>
> ** **
>
> Finally, I’ll point out that I don’t believe that your approaches (1) and
> (2) are mutually exclusive.  We could conceivably leave it up to
> application specifications using the JOSE specs whether they require header
> fields to be understood by default AND provide a syntax for declaring which
> fields may be safely ignored for use in those application contexts where
> the application spec does require that header fields be normally
> understood.  In fact, one might consider this to fall under the “(possibly
> providing guidance directed to those applications in the JOSE specs)”
> that you suggested below.****
>
> ** **
>
> ****
> ** **
>                                                             -- Mike****
> ** **
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Karen O'Donoghue
> *Sent:* Monday, February 25, 2013 3:26 AM
> *To:* [email-masked]
> *Subject:* [jose] thoughts on header criticality from recent polls****
>
> ** **
>
> Folks,
>
> perspective with some results and direction. I had hoped to provide either
> a decision or a clear set of choices, but after analyzing the poll results,
> rereading all the email traffic (multiple times), and talking to a few
> working group members, I find I am not quite able to do so. So, what
> next...
>
> something), I have seen a number of basic approaches proposed (summarized
> by Jim Schaad on 9 Oct 2012 with the response by James Manager on 10 Oct
> 2012). In particular, I found James' list of requirements to be very
> helpful, and I have repeated it here...
>
> to develop a specific syntax (if so, what?) or to defer to the applications
> with appropriate guidance (if so, what level of guidance)? To address this
> item, I welcome concise proposals of text.
>
> I realize this message is long and not as specific or prescriptive as some
> would like, and I apologize in advance for any mistakes and
> Karen O'Donoghue****

---

From: Anthony Nadalin <[email-masked]>
Date: Mon, 25 Feb 2013 22:20:44 +0000

Worse is that "MUT UNDERSTAND" was never understood in SOAP

From: [email-masked] [mailto:[email-masked]] On Behalf Of Tim Bray
Sent: Monday, February 25, 2013 2:18 PM
To: Mike Jones
Cc: [email-masked]; [email-masked]
Subject: Re: [jose] thoughts on header criticality from recent polls

One addition to Mike's notes.  If you are going to have ignorable fields, it'd be nice to avoid one of the (many, many) design botches that increased the difficulty of doing SOAP; it had a thing where you could specify, in among the SOAP headers, a MUST-UNDERSTAND rule. The effect was that you didn't know whether you could act on any header until you'd seen them all, because there might be a MUST-UNDERSTAND lurking somewhere.

The "ign" : [ "notes" ] idiom has, to my eye, that drawback.  The structured one lets you know, as soon as you see a field, whether it's OK to not understand it.  -T

On Mon, Feb 25, 2013 at 7:41 AM, Mike Jones <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for sending this, Karen.  Hopefully your analysis will help the working group reach a resolution on this issue soon.  I'm writing this note primarily as an editor to help inform the working group's discussions, rather than as an individual contributor.

FLAT SYNTAX:
    {"alg":"ES256",
     "ign":["notes"],

STRUCTURED SYNTAX:
    {"alg":"ES256",
     "ign":{

                                                            -- Mike

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Karen O'Donoghue
Sent: Monday, February 25, 2013 3:26 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [jose] thoughts on header criticality from recent polls

Folks,

Regards,
Karen O'Donoghue

_______________________________________________
jose mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jose

---

From: Richard Barnes <[email-masked]>
Date: Mon, 25 Feb 2013 17:40:06 -0500

Sorry, John, no agreement on "aad"  :)

If you want integrity checking on non-critical stuff, just put it in the
header and mark it as non-critical.  I'm willing to step back on what I
said in my response to "B"; it can be much more easily mitigated by a
signer putting something random (and not subject-controlled) before the
unknown stuff in the header.

On Mon, Feb 25, 2013 at 5:16 PM, John Bradley <[email-masked]> wrote:

> Karen & Mike,
>
> I think an important issue that Jim and others have tried to get at is
> some information in the header is intended for the JOSE library, while
> other header parameters are essentially additional associated data
> (meta-data about the body) and need to be passed on to other software
> layers for processing along with the body.   ****
> ** **
> An example of that was the proposed "iat" (issued at) element, which needs
> to be processed outside of the JOSE crypto processing to be useful.****
> ** **
> I suspect a point of agreement might be having an "aad" header parameter
> that is a JSON object that MUST be ignored by the JOSE library and passed
> on with the body.  This would provide applications a way to pass meta-data
> through JOSE that is integrity protected, without JOSE having to parse it.
> ****
> ** **
> Once we remove a large number of the possible extension header parameters
> from JOSE processing, the problem gets smaller.****
> ** **
> There is however, still the requirement that extensions for the header
> parameters JOSE processes be possible.****
> ** **
> For that the list of non-critical header parameters is the best solution.
>  That enables a header parameter to be migrated from non-critical to
> critical without changing its JSON element name.****
> ** **
> I think that a large number of people on the list favoured the single
> element with an array of element names that are non-critical.****
> ** **
> My proposal is to add two new top level elements that MUST be understood:*
> ***
> ** **
> 1 "ign":  Ignore - an array of top level elements in the envelope that an
> application MUST ignore if it cannot process them.****
> 2 "aad": Additional Associated Data - a JSON object that MUST be provided
> to applications along with the body if present in a JOSE header.****
> ** **
> All other top level header parameters MUST be understood by a JOSE library
> or produce a processing fault resulting in an error.****
> ** **
> I would also not want elements listed in "ign" to be passed on to
> applications.  I think that would cause interoperability issues.   That is
> what "aad" is intended to provide.****
> ** **
> I know James has legitimate concerns about allowing potentially user
> provided data to be included in the header (Or anywhere likely).  I think
> this is a separate issue that needs to be addressed by security
> considerations to add entropy to the header if such data is included.
> That may require a new "ent" Entropy element that MUST be ignored.  This
> should be a separate discussion.****
> ** **
> Finally, for JWK, I believe that we should just state that not-understand
> elements should be ignored. When including a JWK header parameter this is
> also allows for extension to the JWK parameter, as the MUST understand in
> JWE/JWS only allies to the top level parameter's, and not the sub
> parameters they may contain.****
> ** **
> My proposal doesn't neatly fit into fit into one of the poll answers, but
> I think it may address the majority of concerns.****
> ** **
> Regards****
> John B.
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
>  *Karen O'Donoghue
> working group members, I find I am not quite able to do so. So, what next...
>
>

---

From: John Bradley <[email-masked]>
Date: Mon, 25 Feb 2013 15:09:45 -0800

aad allows you to signal to JOSE that the included parameters be passed through and not just discarded.  

I think it helps interoperability, simply saying header x may be ignored is a problem unless there is a rule to always pass on parameters that are not understood.. 

John B.

On 2013-02-25, at 2:40 PM, Richard Barnes <[email-masked]> wrote:

> If you want integrity checking on non-critical stuff, just put it in the header and mark it as non-critical.  I'm willing to step back on what I said in my response to "B"; it can be much more easily mitigated by a signer putting something random (and not subject-controlled) before the unknown stuff in the header.
> 
> 
>

---

From: Richard Barnes <[email-masked]>
Date: Mon, 25 Feb 2013 18:20:06 -0500

You seem to be implying that an implementation is selectively stripping out
header fields.  Why would you do that, instead of just passing on
everything, or nothing?

On Mon, Feb 25, 2013 at 6:09 PM, John Bradley <[email-masked]> wrote:

> I think it helps interoperability, simply saying header x may be ignored
> is a problem unless there is a rule to always pass on parameters that are
> not understood..
>
> John B.
>
>
>> An example of that was the proposed "iat" (issued at) element, which
>> needs to be processed outside of the JOSE crypto processing to be useful.
>> ****
>> ** **
>> All other top level header parameters MUST be understood by a JOSE
>> library or produce a processing fault resulting in an error.****
>> ** **
>> First, I want to point out that while we’ve been discussing criticality
>> of header parameters, the specs actually currently have the “MUST be
>> I hope the points above help further a productive discussion of this
>> topic.****
>>
>>                                                             -- Mike****
>>

---

From: John Bradley <[email-masked]>
Date: Mon, 25 Feb 2013 15:30:31 -0800

We are interested in interoperable JOSE libraries.   I don't have any reason to believe from what we have in the spec that the entire header is passed on with the body after verification.

Is that your interpretation of the expected behaviour for a JOSE lib?   If that is the expected behaviour then dropping aad may make sense.

This is a bit more of a challenge without talking about expected API.

John B.

On 2013-02-25, at 3:20 PM, Richard Barnes <[email-masked]> wrote:

> You seem to be implying that an implementation is selectively stripping out header fields.  Why would you do that, instead of just passing on everything, or nothing?
> 
>

---

From: Richard Barnes <[email-masked]>
Date: Mon, 25 Feb 2013 18:40:08 -0500

Actually, I would expect a library not to pass on anything from the header.
 OpenSSL's CMS_decrypt function doesn't provide you with any of the crypto
information or protected attributes.  The only thing that gets passed
through to upper layers is the protected body.

ISTM that the appropriate framework here is extension, not layering.
 Suppose I've got an extended JOSE library that does some special,
non-critical stuff.  Is that going to cause the recipient to die?  Or is
the recipient just going to ignore what he doesn't understand?

--Richard

On Mon, Feb 25, 2013 at 6:30 PM, John Bradley <[email-masked]> wrote:

> We are interested in interoperable JOSE libraries.   I don't have any
> reason to believe from what we have in the spec that the entire header is
> passed on with the body after verification.
>
> Is that your interpretation of the expected behaviour for a JOSE lib?   If
> that is the expected behaviour then dropping aad may make sense.
>
> You seem to be implying that an implementation is selectively stripping
> out header fields.  Why would you do that, instead of just passing on
> everything, or nothing?
>
>
>>> Once we remove a large number of the possible extension header
>>> parameters from JOSE processing, the problem gets smaller.****
>>> ** **
>>> For that the list of non-critical header parameters is the best
>>> solution.  That enables a header parameter to be migrated from non-critical
>>> to critical without changing its JSON element name.****
>>> ** **
>>> 1 "ign":  Ignore - an array of top level elements in the envelope that
>>> an application MUST ignore if it cannot process them.****
>>> 2 "aad": Additional Associated Data - a JSON object that MUST be
>>> provided to applications along with the body if present in a JOSE header.
>>> ****
>>> ** **
>>> Finally, for JWK, I believe that we should just state that
>>> not-understand elements should be ignored. When including a JWK header
>>> parameter this is also allows for extension to the JWK parameter, as the
>>> MUST understand in JWE/JWS only allies to the top level parameter's, and
>>> not the sub parameters they may contain.****
>>> ** **
>>> My proposal doesn't neatly fit into fit into one of the poll answers,
>>> but I think it may address the majority of concerns.****
>>> ** **
>>> Regards****
>>> John B.
>>> As I see it, the advantage of the flat syntax is all header fields are
>>> in the same place.  This makes processing header fields easier and is more
>>> Finally, I’ll point out that I don’t believe that your approaches (1)
>>> and (2) are mutually exclusive.  We could conceivably leave it up to

---

From: Mike Jones <[email-masked]>
Date: Tue, 26 Feb 2013 00:36:46 +0000

Hi Richard,

Responding to your proposal "that we extend (1) by saying that the non-critical headers would not be covered by the header integrity check", I'll note that syntactically separating the critical and non-critical headers was option "B" in the third question of the poll, and would be a pre-requisite to having non-critical members not be covered by the integrity check.  By my count, only 10% of the respondents supported that position.  (I know - repeat after me - the IETF doesn't vote. :))  Possibly more important than the actual numbers, Karen wrote in her reply "I am assuming that we aren't going to define a separate header for non-critical elements."  So I, as an individual, I think that option should be off the table, because I can't see it increasing consensus.

I think it would be more productive to focus the discussion on how we want to handle the (integrity-protected) header parameters that are and aren't critical to understand, and not try to conflate that with not integrity protecting some parts of the message.

                                                            -- Mike

From: [email-masked] [mailto:[email-masked]] On Behalf Of Richard Barnes
Sent: Monday, February 25, 2013 2:12 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [jose] thoughts on header criticality from recent polls

Hi Karen,

tl;dr: I'm OK with (1) or (2).  I would propose that we extend (1) by saying that the non-critical headers would not be covered by the header integrity check.

--Richard

<snip>

Here's my take:

A: This is a requirement.  But it is met trivially by having the new format use new identifier names and marking them critical.

B: This is not a requirement.  It's a bad idea.  Identifiers here are cheap, so if you want new semantics, just use a new identifier.

C: This is a requirement (and a good point), in the sense that we should avoid providing channels that an attacker could use to control the integrity check value on a JW* object.  However, what that argues is that non-critical fields shouldn't be covered by the integrity check, not that they should be ruled out altogether.

E: This is not a requirement.  Couple of clients/servers is an application-layer issue.

I would personally be OK with either approach.  I would generalize (1) slightly to say "Define a syntax for indicating which fields are critical / non-critical", and, in light of (C) above, also propose that that syntax remove non-critical fields from under the integrity check.

---

From: Mike Jones <[email-masked]>
Date: Tue, 26 Feb 2013 00:55:31 +0000

If, as Richard writes, that the appropriate framework here is extension, not layering, then it would seem to me that we do need a mechanism for identifying which extension parameters are ignorable (such as the proposed "ign" field list), but we don't need John's proposed "aad" header.  I think I'm with Richard on this one.

                                                            -- Mike

From: Richard Barnes [mailto:[email-masked]]
Sent: Monday, February 25, 2013 3:40 PM
To: John Bradley
Cc: Mike Jones; [email-masked]; [email-masked]
Subject: Re: [jose] thoughts on header criticality from recent polls

Actually, I would expect a library not to pass on anything from the header.  OpenSSL's CMS_decrypt function doesn't provide you with any of the crypto information or protected attributes.  The only thing that gets passed through to upper layers is the protected body.

ISTM that the appropriate framework here is extension, not layering.  Suppose I've got an extended JOSE library that does some special, non-critical stuff.  Is that going to cause the recipient to die?  Or is the recipient just going to ignore what he doesn't understand?

--Richard

On Mon, Feb 25, 2013 at 6:30 PM, John Bradley <[email-masked]<mailto:[email-masked]>> wrote:
We are interested in interoperable JOSE libraries.   I don't have any reason to believe from what we have in the spec that the entire header is passed on with the body after verification.

John B.

On 2013-02-25, at 3:20 PM, Richard Barnes <[email-masked]<mailto:[email-masked]>> wrote:

On Mon, Feb 25, 2013 at 6:09 PM, John Bradley <[email-masked]<mailto:[email-masked]>> wrote:
aad allows you to signal to JOSE that the included parameters be passed through and not just discarded.

John B.

On 2013-02-25, at 2:40 PM, Richard Barnes <[email-masked]<mailto:[email-masked]>> wrote:

On Mon, Feb 25, 2013 at 5:16 PM, John Bradley <[email-masked]<mailto:[email-masked]>> wrote:
Karen & Mike,

Regards
John B.
On 2013-02-25, at 7:41 AM, Mike Jones <[email-masked]<mailto:[email-masked]>> wrote:

FLAT SYNTAX:
    {"alg":"ES256",
     "ign":["notes"],

STRUCTURED SYNTAX:
    {"alg":"ES256",
     "ign":{

                                                            -- Mike

From: [email-masked]<mailto:[email-masked]> [mailto:jose-<mailto:jose->[email-masked]<mailto:[email-masked]>] On Behalf Of Karen O'Donoghue
Sent: Monday, February 25, 2013 3:26 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [jose] thoughts on header criticality from recent polls

Folks,

Regards,
Karen O'Donoghue

---

From: "Jim Schaad" <[email-masked]>
Date: Mon, 25 Feb 2013 17:22:06 -0800

This would correspond to my expectation.  That the entire header would be
available to the application.

From: [email-masked] [mailto:[email-masked]] On Behalf Of John
Bradley
Sent: Monday, February 25, 2013 3:31 PM
To: Richard Barnes
Cc: Mike Jones; [email-masked]; [email-masked]
Subject: Re: [jose] thoughts on header criticality from recent polls

We are interested in interoperable JOSE libraries.   I don't have any reason
to believe from what we have in the spec that the entire header is passed on
with the body after verification.

John B.

aad allows you to signal to JOSE that the included parameters be passed
through and not just discarded.  

I think it helps interoperability, simply saying header x may be ignored is
a problem unless there is a rule to always pass on parameters that are not
understood.. 

John B.

If you want integrity checking on non-critical stuff, just put it in the
header and mark it as non-critical.  I'm willing to step back on what I said
in my response to "B"; it can be much more easily mitigated by a signer
putting something random (and not subject-controlled) before the unknown
stuff in the header.

Karen & Mike,

I think an important issue that Jim and others have tried to get at is some
information in the header is intended for the JOSE library, while other
header parameters are essentially additional associated data (meta-data
about the body) and need to be passed on to other software layers for
processing along with the body.   

An example of that was the proposed "iat" (issued at) element, which needs
to be processed outside of the JOSE crypto processing to be useful.

I suspect a point of agreement might be having an "aad" header parameter
that is a JSON object that MUST be ignored by the JOSE library and passed on
with the body.  This would provide applications a way to pass meta-data
through JOSE that is integrity protected, without JOSE having to parse it.

Once we remove a large number of the possible extension header parameters
from JOSE processing, the problem gets smaller.

There is however, still the requirement that extensions for the header
parameters JOSE processes be possible.

For that the list of non-critical header parameters is the best solution.
That enables a header parameter to be migrated from non-critical to critical
without changing its JSON element name.

I think that a large number of people on the list favoured the single
element with an array of element names that are non-critical.

1 "ign":  Ignore - an array of top level elements in the envelope that an
application MUST ignore if it cannot process them.

2 "aad": Additional Associated Data - a JSON object that MUST be provided to
applications along with the body if present in a JOSE header.

All other top level header parameters MUST be understood by a JOSE library
or produce a processing fault resulting in an error.

I would also not want elements listed in "ign" to be passed on to
applications.  I think that would cause interoperability issues.   That is
what "aad" is intended to provide.

I know James has legitimate concerns about allowing potentially user
provided data to be included in the header (Or anywhere likely).  I think
this is a separate issue that needs to be addressed by security
considerations to add entropy to the header if such data is included.   That
may require a new "ent" Entropy element that MUST be ignored.  This should
be a separate discussion.

Finally, for JWK, I believe that we should just state that not-understand
elements should be ignored. When including a JWK header parameter this is
also allows for extension to the JWK parameter, as the MUST understand in
JWE/JWS only allies to the top level parameter's, and not the sub parameters
they may contain.

My proposal doesn't neatly fit into fit into one of the poll answers, but I
think it may address the majority of concerns.

Regards

John B.

Thanks for sending this, Karen.  Hopefully your analysis will help the
working group reach a resolution on this issue soon.  I'm writing this note
primarily as an editor to help inform the working group's discussions,
rather than as an individual contributor.

First, I want to point out that while we've been discussing criticality of
header parameters, the specs actually currently have the "MUST be
understood" language in the JWK specification as well, as that's another
potential extension point.  The JWK description says "Additional members MAY
be present in the JWK.  If present, they MUST be understood by
implementations using them."  The same language is present for the JWK Set.
For our resolution of the issue to be complete, it should address those uses
too.  Given that a number of additional JWK fields are being discussed, I
expect that extensions will occur there.

Second, if we do decide to go with your approach (1), I'll summarize for the
list what I believe the two syntax proposals on the table are for doing
this.  One proposal is 2A in
<http://www.ietf.org/mail-archive/web/jose/current/msg01353.html>
http://www.ietf.org/mail-archive/web/jose/current/msg01353.html (which I'll
call the FLAT SYNTAX) and the other is the syntax that Dick Hardt proposed
in  <http://www.ietf.org/mail-archive/web/jose/current/msg01457.html>
http://www.ietf.org/mail-archive/web/jose/current/msg01457.html (which I'll
call the STRUCTURED SYNTAX).  I believe that it would be useful for the
working group to discuss which of those they would prefer in parallel to
discussing the questions that you asked, so that if we do go with approach
(1) or something like it, we'll know which to use.

FLAT SYNTAX:

    {"alg":"ES256",

     "ign":["notes"],

    }

STRUCTURED SYNTAX:

    {"alg":"ES256",

     "ign":{

      }

    }

As I see it, the advantage of the flat syntax is all header fields are in
the same place.  This makes processing header fields easier and is more
regular.  The disadvantage is that the list of fields that may be safely
ignored repeats those field names.

As I see it, the advantage of the structured syntax is that nothing is
repeated, resulting in cleaner JSON syntax.  The disadvantage is that then
some header fields are in one place and some are in another, making the
processing rules for header fields more complicated.

Between these two, it seems to come down to a choice between cleaner syntax
or cleaner semantics.  (I'd personally lean towards the simpler processing
rules resulting from the flat syntax, but I would be fine with this coming
out either way.)

Finally, I'll point out that I don't believe that your approaches (1) and
(2) are mutually exclusive.  We could conceivably leave it up to application
specifications using the JOSE specs whether they require header fields to be
understood by default AND provide a syntax for declaring which fields may be
safely ignored for use in those application contexts where the application
spec does require that header fields be normally understood.  In fact, one
might consider this to fall under the "(possibly providing guidance directed
to those applications in the JOSE specs)" that you suggested below.

                                                            -- Mike

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Karen O'Donoghue
Sent: Monday, February 25, 2013 3:26 AM
To: [email-masked]
Subject: [jose] thoughts on header criticality from recent polls

Folks,

Thank you for your patience with me regarding the recent polls on header
criticality. I realize I have taken too long to respond from a chairs
perspective with some results and direction. I had hoped to provide either a
decision or a clear set of choices, but after analyzing the poll results,
rereading all the email traffic (multiple times), and talking to a few
working group members, I find I am not quite able to do so. So, what next...

I think it is clear that the working group remains divided on this issue
with a significant number of members feeling that an approach must be
defined that safely allows for some header fields to be considered
non-critical. I think there is general consensus that there must be
reasonable mechanisms for extensibility, and that the approach must be
generic. Based on that, I would like to ask the working group to move
forward with the strategy that all headers cannot be considered critical and
move on to the discussion of how to approach non-critical headers. 

In my review of the mailing list traffic (and my apologies if I missed
something), I have seen a number of basic approaches proposed (summarized by
Jim Schaad on 9 Oct 2012 with the response by James Manager on 10 Oct 2012).
In particular, I found James' list of requirements to be very helpful, and I
have repeated it here...

A. Want to be able to define messages with new semantics in future, without
any risk that old implementations will misinterpret them with old semantics.

B. Want to be able to *redefine* (or constrain) the semantics of *existing*
fields in future, without any risk that old implementations will
misinterpret them with old semantics (or without the constraints).

D. Want to strongly discourage future extensions (feature-creep), under the
philosophy that even well-intentioned extensions tend to do more harm (by
adding complexity that is only deployed sporadically) than the good of any
new functionality they bring.

QUESTION: Is there working group consensus on the requirements we are trying
to address? Perhaps a short discussion clarifying these would help drive
consensus on the overall approach. 

As for possible solutions, we appear to have evolved to the point where we
are of two minds: 

1) Define a header field that explicitly lists the fields that may be safely
ignored if not understood. (Here I am assuming that we aren't going to
define a separate header for non-critical elements.) 

2) Remain silent on header criticality in the JOSE specifications and leave
it up to the various application specifications (possibly providing guidance
directed to those applications in the JOSE specs). 

QUESTION: Can the working group agree on whether to work on approach 1 or
approach 2? 

In conclusion, I am asking three things in this missive: 
1. I am asking the working group to move forward based on a non-critical
headers approach.  
2. I am asking the working group to clarify the requirements driving the
non-critical header discussion (A, B, C, D, E, or ?). 
3. I am asking the working group to decide whether the right approach is to
develop a specific syntax (if so, what?) or to defer to the applications
with appropriate guidance (if so, what level of guidance)? To address this
item, I welcome concise proposals of text. 

Regards,
Karen O'Donoghue

---

From: "Manger, James H" <[email-masked]>
Date: Tue, 26 Feb 2013 17:19:24 +1100

A problem with both the FLAT and STRUCTURED syntaxes is that there are lots of non-critical fields.

From JWE (draft 8): critical fields are “alg”, “zip” – and I think that is it. “jku”, “jwk”, “x5u”, “x5t”, “x5c”, “kid”, “typ”(?), and “cty” are non-critical. Even “apu”, “apv”, “epu”, “epv”, “enc”, and “epk” can probably be marked as non-critical (you might not be able to successfully process a JOSE message that has these fields if you ignore them, but you will not *misinterpret* the JOSE message which is the only real concern).

From JWS (draft 8): the only critical field is “alg”.

I doubt anyone wants most field names duplicated in an “ign” array [FLAT syntax].

I don’t like the STRUCTURED syntax. The prospect of writing code that has to look in 2 places for every field it is interested in is fairly appalling (eg k = header.kid !== undefined ? header.kid: header.ign.kid). Not to mention the extra complexity/corner-cases/security-holes from the possibility that a field has 2 different values: one at the top-level, and another under “ign”.

Having 1 critical field that must be given a new value if a new message could otherwise be misinterpreted seems much simpler.

--
James Manger

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mike Jones

FLAT SYNTAX:
    {"alg":"ES256",
     "ign":["notes"],

STRUCTURED SYNTAX:
    {"alg":"ES256",
     "ign":{

---

From: John Bradley <[email-masked]>
Date: Tue, 26 Feb 2013 01:10:16 -0800

There seem to be multiple interpretations of the entire header would be available to the app.

I think Richard clarified that like CMS the only thing that would be returned to the application would be the body, and the method of accessing the header would be through JOSE plugins of some sort.

> Actually, I would expect a library not to pass on anything from the header.  OpenSSL's CMS_decrypt function doesn't provide you with any of the crypto information or protected attributes.  The only…

I expect that the application could take the JWE/JWS apart and extract parameters from the header before passing it to a lib, but that is sort of a hack.  

Making the info available via extensions to the lib is also posable.

Without some common understanding of what a interoperable JOSE library must implement the specifics of criticality are awkward to discuss.

John B.

On 2013-02-25, at 5:22 PM, Jim Schaad <[email-masked]> wrote:

> This would correspond to my expectation.  That the entire header would be available to the application.
>  
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Bradley