Subject: Re: [jose] MUST understand ALL header fields

From: Mike Jones <[email-masked]>
Date: Fri, 6 Jul 2012 18:22:45 +0000

Since this is a security protocol, the way I see it, use of a not-understood parameter SHOULD break implementations if they're not prepared to handle them (or at least them cause them to reject the input).  For example, if we hypothetically added an audience restriction parameter, ignoring it would be a security error.  (The need for audience restriction is being heavily discussed in the OAuth WG right now, and real exploits that have occurred for the lack of it, so this example is real and fresh on my mind.)  If not understood, a token containing an audience restriction MUST be rejected.

On your second point, I don't think the working group wanted to recreate the whole of certificate semantics in JSON.  Hence the intentionally limited capabilities of JWK, for instance.  Those limitations even made it into the WG charter.  But if you want to propose specific certificate-related additions to the WG, have at it.

                                                                -- Mike

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of Manger, James H
Sent: Tuesday, May 15, 2012 7:56 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [jose] MUST understand ALL header fields

draft-ietf-jose-json-web-signature-02 and draft-ietf-jose-json-web-encryption-02 say:

   Implementations MUST understand the entire contents of the
   header; otherwise, the {JWS|JWE} MUST be rejected.

We should drop this restriction.

Firstly, most implementations will ignore it.
I had a quick glance at some existing implementations (magical/jwt-python, markrcote/jwt, jsontoken) and all appear to ignore this requirement.
It cannot add any security when it is ignored. It does cause massive compatibility headaches for anyone wanting to define another header parameter, since a new parameter will break some implementations.

Secondly, a couple of parameters are defined for X.509 certs, but none for OCSP responses, CRLs, timestamps etc. It is inevitable that a cert-based scheme will need these extra pieces of info. With the "MUST understand everything" rule, we have to define all the extra info that might be needed right now.

If we need to be able to break backward compatibility we should define a mechanism explicitly for that.

--
James Manger

---

From: "Richard L. Barnes" <[email-masked]>
Date: Fri, 6 Jul 2012 14:34:44 -0400

There are obviously examples on both sides.  Parameters that restrict validity have to be understood, parameters that increase availability or otherwise help out can be safely ignored.  Cf. critical vs. non-critical extensions in X.509.

ISTM that there are basically three design options:

1. Require that implementations MUST ignore unknown fields

2. Require that implementations MUST reject an object with an unknown field

3. Add an extension system like PKIX has, where individual extensions can be critical or not.  For example:
{
    ...
    "ext1": "val1",
    "ext2": "val2",
    "critical": ["ext2"],
}

Option (2) really just ensures that the current format has *no* extensibility; option (1) at least allows for "permissive" extensions.  The choice we make now depends on whether we think there's a lot of "restrictive" extensions coming.  If so, then we should do option (3).  If not, then we should do option (1) for now,  with a note that you should only do permissive extensions.  If there's a need for (3) later, it can be added in a new version of the format.

--Richard

On Jul 6, 2012, at 2:22 PM, Mike Jones wrote:

> Since this is a security protocol, the way I see it, use of a not-understood parameter SHOULD break implementations if they’re not prepared to handle them (or at least them cause them to reject the input).  For example, if we hypothetically added an audience restriction parameter, ignoring it would be a security error.  (The need for audience restriction is being heavily discussed in the OAuth WG right now, and real exploits that have occurred for the lack of it, so this example is real and fresh on my mind.)  If not understood, a token containing an audience restriction MUST be rejected.
>  
> On your second point, I don’t think the working group wanted to recreate the whole of certificate semantics in JSON.  Hence the intentionally limited capabilities of JWK, for instance.  Those limitations even made it into the WG charter.  But if you want to propose specific certificate-related additions to the WG, have at it.
>  
>                                                                 -- Mike
>  
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Manger, James H
> To: [email-masked]
> Secondly, a couple of parameters are defined for X.509 certs, but none for OCSP responses, CRLs, timestamps etc. It is inevitable that a cert-based scheme will need these extra pieces of info. With the “MUST understand everything” rule, we have to define all the extra info that might be needed right now.
>  
> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose

---

From: "Manger, James H" <[email-masked]>
Date: Mon, 9 Jul 2012 10:42:37 +1000

Mike,

Mike replied:
> Since this is a security protocol, the way I see it, use of a not-
> understood parameter SHOULD break implementations if they’re not
> prepared to handle them (or at least them cause them to reject the
> input).  For example, if we hypothetically added an audience
> restriction parameter, ignoring it would be a security error.  (The
> need for audience restriction is being heavily discussed in the OAuth
> WG right now, and real exploits that have occurred for the lack of it,
> so this example is real and fresh on my mind.)  If not understood, a
> token containing an audience restriction MUST be rejected.

Consider an ecosystem with hundreds of servers and thousands of client apps all using a protocol involving JOSE messages that has been operating for a few years. A researcher finds a vulnerability in the protocol and responsibly discloses it to selected servers. The fix is to add an audience restriction parameter "aud".

What should a server do?

If it adds the "aud" parameter immediately, every (compliant) client app will immediately break. The outage will last until every app is updated, which will takes weeks, months…? Total system shutdown to fix a flaw that is currently theoretical… heads will role.

If a server announces that it will add the "aud" parameter in, say, 3 months to give time for app updates then it has just disclosed a vulnerability and given attackers a 3 month window to exploit it. Many will clamour for an immediate fix; many will say they need longer to fix apps; it will still break for many users in 3 months… heads will role.

On the other hand, if the spec says unknown parameters MUST be ignored then the server can immediately add the "aud" parameter, and immediately tell client apps to start supporting it. Each app is protected as soon as it adds support.
If actual exploits need to be stopped the server can still shutdown the whole system, or just selected client apps, or selected users.

X.509 certificates are an interesting example for comparison. Certs are security-related, just like JOSE messages. Certs allow unrecognised additions (eg non-critical extensions). These have been very widely used to deploy new functionality.

The X.509 spec (or at least its IETF profile) has one item that MUST be understood if added: the name-constraints extension. This extension should be really useful for security. However, because adding it in the spec-compliant MUST-understand manner will break some clients, it has barely been deployed for over a decade.

Lets drop the "MUST understand everything" requirement.

--
James Manger

---

From: Breno de Medeiros <[email-masked]>
Date: Mon, 9 Jul 2012 09:44:46 -0700

At some point, the spec said that servers should reject tokens with unknown
parameters in the envelope (which describes the security context) but
tolerate them in the body.

On Sun, Jul 8, 2012 at 5:42 PM, Manger, James H <
[email-masked]> wrote:

> Consider an ecosystem with hundreds of servers and thousands of client
> apps all using a protocol involving JOSE messages that has been operating
> for a few years. A researcher finds a vulnerability in the protocol and
> responsibly discloses it to selected servers. The fix is to add an audience
> restriction parameter "aud".
>
> What should a server do?
>
> If it adds the "aud" parameter immediately, every (compliant) client app
> will immediately break. The outage will last until every app is updated,
> which will takes weeks, months…? Total system shutdown to fix a flaw that
> is currently theoretical… heads will role.
>
> If a server announces that it will add the "aud" parameter in, say, 3
> months to give time for app updates then it has just disclosed a
> vulnerability and given attackers a 3 month window to exploit it. Many will
> clamour for an immediate fix; many will say they need longer to fix apps;
> it will still break for many users in 3 months… heads will role.
>
> On the other hand, if the spec says unknown parameters MUST be ignored
> then the server can immediately add the "aud" parameter, and immediately
> tell client apps to start supporting it. Each app is protected as soon as
> it adds support.
> If actual exploits need to be stopped the server can still shutdown the
> whole system, or just selected client apps, or selected users.
>
>
> X.509 certificates are an interesting example for comparison. Certs are
> security-related, just like JOSE messages. Certs allow unrecognised
> additions (eg non-critical extensions). These have been very widely used to
> deploy new functionality.
>
> The X.509 spec (or at least its IETF profile) has one item that MUST be
> understood if added: the name-constraints extension. This extension should
> be really useful for security. However, because adding it in the
> spec-compliant MUST-understand manner will break some clients, it has
> barely been deployed for over a decade.
>
>

-- 
--Breno

---

From: Mike Jones <[email-masked]>
Date: Mon, 9 Jul 2012 16:54:41 +0000

I suspect what you're remembering is this distinction in the JWT spec.  About claims, http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-01#section-4 says:

"When used in a security-related context, implementations MUST understand and support all of the claims present; otherwise, the JWT MUST be rejected for processing."

Whereas about the headers, http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-01#section-5 says:

"Implementations MUST understand the entire contents of the header; otherwise, the JWT MUST be rejected for processing."

                                                                Best wishes,
                                                                -- Mike

From: Breno de Medeiros [mailto:[email-masked]]
Sent: Monday, July 09, 2012 9:45 AM
To: Manger, James H
Cc: Richard L. Barnes; Mike Jones; [email-masked]
Subject: Re: [jose] MUST understand ALL header fields

At some point, the spec said that servers should reject tokens with unknown parameters in the envelope (which describes the security context) but tolerate them in the body.

On Sun, Jul 8, 2012 at 5:42 PM, Manger, James H <[email-masked]<mailto:[email-masked]>> wrote:
Mike,

Mike replied:
> understood parameter SHOULD break implementations if they're not

What should a server do?

If it adds the "aud" parameter immediately, every (compliant) client app will immediately break. The outage will last until every app is updated, which will takes weeks, months...? Total system shutdown to fix a flaw that is currently theoretical... heads will role.

If a server announces that it will add the "aud" parameter in, say, 3 months to give time for app updates then it has just disclosed a vulnerability and given attackers a 3 month window to exploit it. Many will clamour for an immediate fix; many will say they need longer to fix apps; it will still break for many users in 3 months... heads will role.

--
James Manger
_______________________________________________
jose mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jose

--
--Breno

---

From: Breno de Medeiros <[email-masked]>
Date: Mon, 9 Jul 2012 10:05:24 -0700

Hmmm... "when used in a security-related context"  is quite vague. If this
wasn't a security-context, there'd be no need for crypto. I think for the
first part, it'd be more helpful to say:

"Implementations SHOULD ignore unknown claims in JWT payload unless the
security context of their application requires otherwise."

On Mon, Jul 9, 2012 at 9:54 AM, Mike Jones <[email-masked]>wrote:

>  I suspect what you’re remembering is this distinction in the JWT spec.
> About claims,
> http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-01#section-4says:
> ****
>
> ** **
>
> “When used in a security-related context, implementations MUST understand
> and support all of the claims present; otherwise, the JWT MUST be rejected
> for processing.”****
>
> ** **
>
> Whereas about the headers,
> http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-01#section-5says:
> ****
>
> ** **
>
> “Implementations MUST understand the entire contents of the header;
> otherwise, the JWT MUST be rejected for processing.”****
>
> ** **
>
>                                                                 Best
> wishes,****
>
>                                                                 -- Mike***
> *
>
> ** **
>
> *From:* Breno de Medeiros [mailto:[email-masked]]
> *Sent:* Monday, July 09, 2012 9:45 AM
> *To:* Manger, James H
> *Cc:* Richard L. Barnes; Mike Jones; [email-masked]
> *Subject:* Re: [jose] MUST understand ALL header fields****
>
> ** **
>
> At some point, the spec said that servers should reject tokens with
> unknown parameters in the envelope (which describes the security context)
> but tolerate them in the body.****
>
> ** **
>
> [email-masked]> wrote:****
>
> Mike,****
>
>
> >> We should drop this restriction.****
>
> Mike replied:****
>
> ****
> Lets drop the "MUST understand everything" requirement.****
>
>
> --
> James Manger
> https://www.ietf.org/mailman/listinfo/jose****
>
>
>
> ****
>
> ** **
>
> --
> --Breno****
>

-- 
--Breno

---

From: Mike Jones <[email-masked]>
Date: Mon, 9 Jul 2012 18:40:29 +0000

(Moving this discussion to the OAuth list, since JWT is an OAuth spec)

From: Breno de Medeiros [mailto:[email-masked]]
Sent: Monday, July 09, 2012 10:05 AM
To: Mike Jones
Cc: Manger, James H; Richard L. Barnes; [email-masked]
Subject: Re: [jose] MUST understand ALL header fields

Hmmm... "when used in a security-related context"  is quite vague. If this wasn't a security-context, there'd be no need for crypto. I think for the first part, it'd be more helpful to say:

"Implementations SHOULD ignore unknown claims in JWT payload unless the security context of their application requires otherwise."

On Mon, Jul 9, 2012 at 9:54 AM, Mike Jones <[email-masked]<mailto:[email-masked]>> wrote:
I suspect what you're remembering is this distinction in the JWT spec.  About claims, http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-01#section-4 says:

                                                                Best wishes,
                                                                -- Mike

From: Breno de Medeiros [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Monday, July 09, 2012 9:45 AM
To: Manger, James H
Cc: Richard L. Barnes; Mike Jones; [email-masked]<mailto:[email-masked]>
Subject: Re: [jose] MUST understand ALL header fields

What should a server do?

--
--Breno

--
--Breno

---

From: Brian Campbell <[email-masked]>
Date: Wed, 25 Jul 2012 09:16:29 -0600

I won't pretend to know what the *right* thing to do is. But I do think
James makes a valid point about the temporal interoperability problems that
will arise if/when new parameters get introduced down the road.

---

From: "Manger, James H" <[email-masked]>
Date: Thu, 26 Jul 2012 11:53:52 +1000

I’ll add another argument for removing the “MUST understand everything” requirement.

JOSE currently uses one bucket (the header JSON object) for different categories of info:

1.       Structure of the JOSE message (eg indicate whether the 2nd dot-separated segment is plain text, an encrypted key, etc)

2.       Algorithm details (eg name, IV, etc)

3.       Hints about keys (eg URI to get key if you don’t have it; revocation status, etc)

4.       Metadata about the content (eg content-type, last modified date, etc)

Even if it is important for security that, say, some structural info is not ignored that doesn’t mean the same importance applies to the other categories of info.

Different categories of info will be handled by quite different code. For instance, a JOSE library is unlikely to care at all about the content metadata; only the decryption code cares about the IV; etc. To enforce a “MUST understand everything” requirement all these different pieces of code from different layers in a software stack (eg app vs library) will need to cooperate. That cooperation will require more complex APIs between layers (eg library returns “content”, “headers”, and “list of headers the app MUST understand because the library ignored them (assuming they are content metadata)”).
--
James Manger

From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Campbell
Sent: Thursday, 26 July 2012 1:16 AM
To: Manger, James H
Cc: Richard L. Barnes; Mike Jones; [email-masked]
Subject: Re: [jose] MUST understand ALL header fields

I won't pretend to know what the *right* thing to do is. But I do think James makes a valid point about the temporal interoperability problems that will arise if/when new parameters get introduced down the road.
On Sun, Jul 8, 2012 at 6:42 PM, Manger, James H <[email-masked]<mailto:[email-masked]>> wrote:
Mike,

What should a server do?

---

From: Breno de Medeiros <[email-masked]>
Date: Wed, 25 Jul 2012 19:01:06 -0700

> I’ll add another argument for removing the “MUST understand everything”

The requirement of understanding everything is only about the layer that
performs signature validation, clearly.

The header is supposed to be metadata for validation only. Putting
non-validation metadata in the header is a bad idea. They payload is
arbitrary -- apps can customize it at will to specify app-level metadata
semantics there.

> ****
>
> ** **
>
> JOSE currently uses one bucket (the header JSON object) for different
> categories of info:****
>
> **1.       **Structure of the JOSE message (eg indicate whether the 2nddot-separated segment is plain text, an encrypted key, etc)
> ****
>
> **2.       **Algorithm details (eg name, IV, etc)****
>
> **3.       **Hints about keys (eg URI to get key if you don’t have it;
> revocation status, etc)****
>
> **4.       **Metadata about the content (eg content-type, last modified
> date, etc)****
>
> ** **
>
> Even if it is important for security that, say, some structural info is
> not ignored that doesn’t mean the same importance applies to the other
> categories of info.****
>
> ** **
>
> Different categories of info will be handled by quite different code. For
> instance, a JOSE library is unlikely to care at all about the content
> metadata; only the decryption code cares about the IV; etc. To enforce a
> “MUST understand everything” requirement all these different pieces of code
> from different layers in a software stack (eg app vs library) will need to
> cooperate. That cooperation will require more complex APIs between layers
> (eg library returns “content”, “headers”, and “list of headers the app MUST
> understand because the library ignored them (assuming they are content
> metadata)”).****
>
> ****
>
> --****
>
> James Manger****
>
> ** **
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Brian Campbell
> *Sent:* Thursday, 26 July 2012 1:16 AM
>
> *To:* Manger, James H
> will arise if/when new parameters get introduced down the road.****
>

-- 
--Breno

---

From: "Manger, James H" <[email-masked]>
Date: Thu, 26 Jul 2012 12:11:37 +1000

> The requirement of understanding everything is only about the layer that performs signature validation, clearly.
>
> The header is supposed to be metadata for validation only. Putting non-validation metadata in the header is a bad idea. They payload is arbitrary -- apps can customize it at will to specify app-level metadata semantics there.

So are you suggesting we disallow metadata about the content (category 4 in my list) from the header? For instance, we should drop the "typ" parameter (just keeping an indication of whether there is another layer of JOSE processing inside, or perhaps not even that)?

--
James Manger

---

From: Breno de Medeiros <[email-masked]>
Date: Wed, 25 Jul 2012 19:17:50 -0700

On Wed, Jul 25, 2012 at 7:11 PM, Manger, James H <
[email-masked]> wrote:

> So are you suggesting we disallow metadata about the content (category 4
> in my list) from the header? For instance, we should drop the "typ"
> parameter (just keeping an indication of whether there is another layer of
> JOSE processing inside, or perhaps not even that)?
>

I am suggesting that any metadata without security implications should not
be in the header, and if it is in the header, then the validator must fail
because it has implications for security that it doesn't understand.

And yes, metadata about the contents does not belong in the JWT header.

> categories of info:
> >> 1.       Structure of the JOSE message (eg indicate whether the 2nd
> dot-separated segment is plain text, an encrypted key, etc)
> categories of info.
> >> Different categories of info will be handled by quite different code.
> For instance, a JOSE library is unlikely to care at all about the content
> metadata)”).

-- 
--Breno