Subject: [jose] Use of Base64 encoding

From: "Jim Schaad" <[email-masked]>
Date: Tue, 4 Sep 2012 13:38:06 -0700

<personal>

I was struck by the questions of which base64 encoder should be used in the
different documents that the working group employed and I started going
through the different locations in the document to see where and how much it
mattered if the base64 or base64URL encoder was used.  This message
represents my conclusions and leads to some questions

1.  The simple dot encoding of the objects does require it as it will
possibly be sent as part of a URL
2.  If you are going to be in a space constrained environment then you MIGHT
want it as it will shrink the result, however doing a solution that deals
with binary data more generally would be a better solution.
3.  Joe might have an argument that only doing things one way is simpler,
however that argument can apply in both directions

The rest of the time I don't think it matters which of the encoding formats
is used.  If you are looking at the SHA-1 hash of a certificate, does it
matter if you use base64 or base64URL, not except for the minor size
increase.  The padding characters themselves are protected from the URL by
the outside base64URL encoding.

Except for the case of the dot encoding step, I think that the use of base64
URL can be dropped from a MUST to a SHOULD with the justifications being
explained.  It was stated at the F2F that the difference in the decoders is
minimal so there is no reason not to allow there and this would allow
different people to make different decisions on this issue.

Jim

---

From: Mike Jones <[email-masked]>
Date: Tue, 4 Sep 2012 20:45:42 +0000

Having multiple ways to do something never helps improve interop

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jim Schaad
Sent: Tuesday, September 04, 2012 1:38 PM
To: [email-masked]
Subject: [jose] Use of Base64 encoding

<personal>

I was struck by the questions of which base64 encoder should be used in the different documents that the working group employed and I started going through the different locations in the document to see where and how much it mattered if the base64 or base64URL encoder was used.  This message represents my conclusions and leads to some questions

1.  The simple dot encoding of the objects does require it as it will possibly be sent as part of a URL 2.  If you are going to be in a space constrained environment then you MIGHT want it as it will shrink the result, however doing a solution that deals with binary data more generally would be a better solution.
3.  Joe might have an argument that only doing things one way is simpler, however that argument can apply in both directions

The rest of the time I don't think it matters which of the encoding formats is used.  If you are looking at the SHA-1 hash of a certificate, does it matter if you use base64 or base64URL, not except for the minor size increase.  The padding characters themselves are protected from the URL by the outside base64URL encoding.

Except for the case of the dot encoding step, I think that the use of base64 URL can be dropped from a MUST to a SHOULD with the justifications being explained.  It was stated at the F2F that the difference in the decoders is minimal so there is no reason not to allow there and this would allow different people to make different decisions on this issue.

Jim

_______________________________________________
jose mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jose

---

From: Bob Wyman <[email-masked]>
Date: Tue, 4 Sep 2012 17:06:30 -0400

It seems to me that if the difference in the decoders is minimal, then one
should feel comfortable in requiring one rather than the other since any
benefit from providing individual developers choice is likely to be minimal
compared to the cost of added complexity for all developers.

bob wyman

On Tue, Sep 4, 2012 at 4:45 PM, Mike Jones <[email-masked]>wrote:

> I was struck by the questions of which base64 encoder should be used in
> the different documents that the working group employed and I started going
> through the different locations in the document to see where and how much
> it mattered if the base64 or base64URL encoder was used.  This message
> possibly be sent as part of a URL 2.  If you are going to be in a space
> constrained environment then you MIGHT want it as it will shrink the
> result, however doing a solution that deals with binary data more generally
> would be a better solution.
> The rest of the time I don't think it matters which of the encoding
> formats is used.  If you are looking at the SHA-1 hash of a certificate,
> does it matter if you use base64 or base64URL, not except for the minor
> size increase.  The padding characters themselves are protected from the
> URL by the outside base64URL encoding.
>
> Except for the case of the dot encoding step, I think that the use of
> base64 URL can be dropped from a MUST to a SHOULD with the justifications
> being explained.  It was stated at the F2F that the difference in the
> decoders is minimal so there is no reason not to allow there and this would
> allow different people to make different decisions on this issue.
>
> Jim
>
>

---

From: Breno de Medeiros <[email-masked]>
Date: Tue, 4 Sep 2012 14:09:47 -0700

+ 1 for keeping it a MUST for the reasons Mike Jones and Bob Wyman point
out.

-- 
--Breno

---

From: "Jim Schaad" <[email-masked]>
Date: Tue, 4 Sep 2012 14:10:58 -0700

I hope that you have a better response than this.  If what you say is true
then we should eliminate a large number of the cryptographic algorithms that
have been proposed as they provide multiple ways of doing things.  

Do you really believe that the difference in the receiving software is going
to be that different based on if base64 or base64URL encoding is used on a
binary value?

Jim

> From: Mike Jones [mailto:[email-masked]]
> Sent: Tuesday, September 04, 2012 1:46 PM
> To: Jim Schaad; [email-masked]
> Subject: RE: [jose] Use of Base64 encoding
> 
the
> 1.  The simple dot encoding of the objects does require it as it will
possibly be
> sent as part of a URL 2.  If you are going to be in a space constrained
> environment then you MIGHT want it as it will shrink the result, however
> doing a solution that deals with binary data more generally would be a
better
> solution.
formats is
> used.  If you are looking at the SHA-1 hash of a certificate, does it
matter if
> you use base64 or base64URL, not except for the minor size increase.  The
> padding characters themselves are protected from the URL by the outside
> base64URL encoding.
> 
base64

---

From: Daniel Holth <[email-masked]>
Date: Tue, 4 Sep 2012 17:16:24 -0400

> to be that different based on if base64 or base64URL encoding is used on a

I wrote that bug just last week. Some people will encode and compare
the encoded results, others will decode and compare the bits.

---

From: Breno de Medeiros <[email-masked]>
Date: Tue, 4 Sep 2012 14:18:14 -0700

Crypto agility is necessary because security of crypto algorithms and key
sizes change over time. So supporting multiple crypto algorithms is
required for a standard that doesn't want to become obsolete within a few
years. It's not a case of flexibility but of necessity. Also, availability
of different crypto algorithms varies widely between platforms, so being
able to support multiple algorithms is a must to ensure wide adoption --
porting a new crypto algorithm to a platform is a very large effort.

OTOH:
- Base64URL encoding is quite trivial to add to a platform that doesn't
have it
- Base64URL encoding is not likely to be deprecated in the foreseeable
future

-- 
--Breno

---

From: Bob Wyman <[email-masked]>
Date: Tue, 4 Sep 2012 17:21:49 -0400

> to be that different based on if base64 or base64URL encoding is used on a
I think the whole point of using something like either base64 or base64URL
encoding is to make the system as simple as possible. The alternative, used
in the past, was to require some kind of complex canonicalization. That
canonicalization proved, in many cases, to be so cumbersome that it
prevented developers from building solutions to real problems. The base64*
solution is the extreme opposite of the more complex canonicalization
approach. Certainly, there are all sorts of intermediate positions between
the two extremes, but it seems to me that we should avoid those mid-points
unless there is some really significant benefit for doing so.

bob wyman

---

From: "Jim Schaad" <[email-masked]>
Date: Tue, 4 Sep 2012 14:42:50 -0700

Ok - let's start with which values you are comparing - ones in the header or
the ones in the dot simple encoding.  I never said a thing about changing
the ones in the dot simple encoding just the ones in the header - such as
the SHA-1 hash of a certificate.  I doubt that you are going to compare that
in an encoded method and if you do then you are going to have a problem if
your certificate library want's it base64 encoded rather than base64URL
encoded.  

I also doubt it makes any difference for the binary values that are parts of
keys which way you encode as the library is either going to want binary
values or needs to be willing to accept it both ways - after all there is
nothing that says a different system is not going to want the same values in
a base64 encoded manner (PKCS#8 anybody).

What you are saying is perhaps a better argument for saying that you MUST
decode and compare the bytes rather than encode and compare the encoded
values.  After all, two ways of doing the same thing is bad right.

Jim

> From: Daniel Holth [mailto:[email-masked]]
> Sent: Tuesday, September 04, 2012 2:16 PM
> To: Jim Schaad
> Cc: Mike Jones; [email-masked]
> Subject: Re: [jose] Use of Base64 encoding
> 
> > I hope that you have a better response than this.  If what you say is
> > true then we should eliminate a large number of the cryptographic
> > algorithms that have been proposed as they provide multiple ways of
doing
> things.
> >
> > Do you really believe that the difference in the receiving software is
> > going to be that different based on if base64 or base64URL encoding is
> > used on a binary value?
> 
> I wrote that bug just last week. Some people will encode and compare the
> encoded results, others will decode and compare the bits.

---

From: "Jim Schaad" <[email-masked]>
Date: Tue, 4 Sep 2012 14:43:53 -0700

I am not saying that crypto agility is necessarily bad, I am saying that a
reflexive response that doing things just one way without looking at the
arguments is not a very compelling response.

Jim

From: Breno de Medeiros [mailto:[email-masked]] 
Sent: Tuesday, September 04, 2012 2:18 PM
To: Jim Schaad
Cc: Mike Jones; [email-masked]
Subject: Re: [jose] Use of Base64 encoding

Crypto agility is necessary because security of crypto algorithms and key
sizes change over time. So supporting multiple crypto algorithms is required
for a standard that doesn't want to become obsolete within a few years. It's
not a case of flexibility but of necessity. Also, availability of different
crypto algorithms varies widely between platforms, so being able to support
multiple algorithms is a must to ensure wide adoption -- porting a new
crypto algorithm to a platform is a very large effort.

OTOH:

- Base64URL encoding is quite trivial to add to a platform that doesn't have
it

Jim

the
possibly be
better
formats is
matter if
base64

 

-- 
--Breno

---

From: "Jim Schaad" <[email-masked]>
Date: Tue, 4 Sep 2012 14:47:18 -0700

I find the canonicalization argument to be very misrepresented.  The lesson
learned from the XML signature world is that you do not want both sides to
have to perform the same canonicalization.  It is not that you don't want
the sender to canonicalize.  After all, by stating that you must use
base64URL you are doing a canonicalization step - you are saying that it
must be encoded this way. 

I don't think it matters one whit to the receiving agent if the base64
decode routine is able to decode both base64 and base64url values.  It just
needs to accept the values sent to it by the sending agent.

Jim

From: [email-masked] [mailto:[email-masked]] On Behalf Of Bob Wyman
Sent: Tuesday, September 04, 2012 2:22 PM
To: Jim Schaad
Cc: Mike Jones; [email-masked]
Subject: Re: [jose] Use of Base64 encoding

bob wyman

Jim

the
possibly be
better
formats is
matter if
base64

---

From: John Bradley <[email-masked]>
Date: Tue, 4 Sep 2012 20:47:27 -0300

We need base64URL for using the tokens in OAuth and other places in a URL.

I don't see a benefit to making recipients figure out what encoding was used.  

Stick with base64URL and if someone wants to convert it to base64 on receipt that is not a huge deal.

John B.

On 2012-09-04, at 6:21 PM, Bob Wyman <[email-masked]> wrote:

> I think the whole point of using something like either base64 or base64URL encoding is to make the system as simple as possible. The alternative, used in the past, was to require some kind of complex canonicalization. That canonicalization proved, in many cases, to be so cumbersome that it prevented developers from building solutions to real problems. The base64* solution is the extreme opposite of the more complex canonicalization approach. Certainly, there are all sorts of intermediate positions between the two extremes, but it seems to me that we should avoid those mid-points unless there is some really significant benefit for doing so.
> 
> bob wyman
>  
> 
> Jim
> 
>

---

From: "Matt Miller (mamille2)" <[email-masked]>
Date: Wed, 5 Sep 2012 02:03:16 +0000

Personally, I would prefer to keep encodings as similar as possible.  Maybe it is petty of me, but I'd rather not have to wonder if the data should be base64 or base64url encoded before sending it.  I'm less worried about receiving; it's trivial to map from one to the other.

And because it's so trivial, I would argue that mandating one or the other is a better course of action; it's one less obstacle for interoperable implementations.

- m&m

Matt Miller - <[email-masked]>
Cisco Systems, Inc.

---

From: Justin Richer <[email-masked]>
Date: Wed, 5 Sep 2012 11:12:00 -0400

+1 to Bob's argument.

Just pick one and stick with it universally. I see no good reason to 
deviate from that. Below, are you suggesting that we pick normal base64 
for one value but base64url for others? Or are you suggesting that we 
allow multiple kinds of encoding for these? If the latter, why wouldn't 
you also allow hex encoding, or base32, or any number of arbitrary 
binary encodings using the same argument as below. Base64URL is well 
established, available on basically every platform, trivial to implement 
if it's not, and doesn't run into any double-encoding issues in any of 
the environments that we care about putting these values in. This isn't 
a useful kind of flexibility that I can see, especially if we don't have 
any way to signal what encoding is being used for a value. Do we want to 
do something like:

hash: {
   encoding: base64,
   value: AeyBEadf2....
}

to accommodate that structure? I hope not.

One of the biggest problems with allowing both Base64 and Base64URL is 
that in some cases they're exactly the same. This can lead you to 
getting lucky picking the wrong encoding but getting the right result 
one time, then having it blow up on you the second time. I've written 
that exact bug a few times.

And more to the point: why *not* use Base64URL everywhere for binary 
encoding? What *benefit* does it give you? I don't see any.

  -- Justin

On 09/04/2012 05:06 PM, Bob Wyman wrote:
> It seems to me that if the difference in the decoders is minimal, then 
> one should feel comfortable in requiring one rather than the other 
> since any benefit from providing individual developers choice is 
> likely to be minimal compared to the cost of added complexity for all 
> developers.
>
> bob wyman
>
> On Tue, Sep 4, 2012 at 4:45 PM, Mike Jones 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     From: [email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>] On
>     Behalf Of Jim Schaad
>     To: [email-masked] <mailto:[email-masked]>
>     I was struck by the questions of which base64 encoder should be
>     used in the different documents that the working group employed
>     and I started going through the different locations in the
>     document to see where and how much it mattered if the base64 or
>     base64URL encoder was used.  This message represents my
>     conclusions and leads to some questions
>
>     1.  The simple dot encoding of the objects does require it as it
>     will possibly be sent as part of a URL 2.  If you are going to be
>     in a space constrained environment then you MIGHT want it as it
>     will shrink the result, however doing a solution that deals with
>     binary data more generally would be a better solution.
>     3.  Joe might have an argument that only doing things one way is
>     simpler, however that argument can apply in both directions
>
>     The rest of the time I don't think it matters which of the
>     encoding formats is used.  If you are looking at the SHA-1 hash of
>     a certificate, does it matter if you use base64 or base64URL, not
>     except for the minor size increase.  The padding characters
>     themselves are protected from the URL by the outside base64URL
>     encoding.
>
>     Except for the case of the dot encoding step, I think that the use
>     of base64 URL can be dropped from a MUST to a SHOULD with the
>     justifications being explained.  It was stated at the F2F that the
>     difference in the decoders is minimal so there is no reason not to
>     allow there and this would allow different people to make
>     different decisions on this issue.
>
>     Jim
>
>
>     [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Dick Hardt <[email-masked]>
Date: Mon, 17 Sep 2012 13:09:34 -0700

My recollection of the reasons for URL safe base64 encoding 3 years ago was to minimize implementation and deployment errors when objects are including as parameters in URLs. Several implementors of other encodings had nightmare debug sessions that were caused by URL encoding / decoding issues.

Is there any reason to NOT use URL safe base64 encoding?

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 18 Sep 2012 00:11:04 +0200

> Is there any reason to NOT use URL safe base64 encoding?

No.

I think Jim's arguments are very confused about the reasons for providing choice in a protocol.
Choice is fundamentally bad for interoperability.  Always.
However, you do provide choice when there is something you win from this choice.
(E.g., crypto agility is demonstrably necessary.)
If you can't demonstrate such a win, you eliminate choice as much as possible.

Since there is zero advantage for base64 over base64url, this is a no-brainer: nail it down.

For those that would like another example for this: In the IETF, we chose network byte order.  Once.  For everything.
Other protocols have a hodgepodge, or, worse, actively negotiate byte order.  
(There are some theoretical advantages for doing this between two systems that are one of the other byte orders.
In reality, it means you have to provide two code paths and neither is optimized as well as it could be if the choice had been nailed down.
And you get a nice helping of unexpected interop failures on top.)

Again: nail it down.  Once.  For everything in this protocol.

Grüße, Carsten

---

From: Breno de Medeiros <[email-masked]>
Date: Mon, 17 Sep 2012 15:17:24 -0700

On Mon, Sep 17, 2012 at 3:11 PM, Carsten Bormann <[email-masked]> wrote:

> For those that would like another example for this: In the IETF, we chose
> network byte order.  Once.  For everything.
> In reality, it means you have to provide two code paths and neither is
> optimized as well as it could be if the choice had been nailed down.

Well said. Agree with everything in the argument above.

-- 
--Breno

---

From: John Bradley <[email-masked]>
Date: Mon, 17 Sep 2012 19:25:12 -0300

agree base64url