Subject: [TLS] PR#28: Converting cTLS to QUIC-style varints

From: Eric Rescorla <[email-masked]>
Date: Mon, 5 Oct 2020 17:34:15 -0700

Hi folks,

cTLS uses a bespoke varint format. Now that QUIC is nearly done, I propose
adopting their varint format.

https://github.com/tlswg/draft-ietf-tls-ctls/pull/28

Any objections?
-Ekr

---

From: Marten Seemann <[email-masked]>
Date: Tue, 6 Oct 2020 08:31:17 +0700

One thing that’s a bit annoying about QUIC’s variant format is that there
are multiple ways to encode a number. This has led to some complications in
the specification (e.g. QUIC requires you to use the minimal encoding for
frame types, but allows all encodings everywhere else).
It would be nice to have an unambiguous way to encode a number.

On Tue, Oct 6, 2020 at 07:35 Eric Rescorla <[email-masked]> wrote:

> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 06 Oct 2020 12:38:04 +1100

Oh, I consider that to be a feature.  One that I exploit.  It's sometimes awkward to build a structure then prepend a length that has a variable-length encoding.  If you know the maximum length, the varint encoding allows you to avoid having to move memory.

That said, cTLS won't authenticate both the value and the specific encoding of numeric values as the varint encoding is erased.  I don't consider that to be a serious problem, but it means that cTLS transcripts can't be directly compared.  I would note that in the draft: people shouldn't be processing cTLS messages that way, but they might be surprised to learn of this.

On Tue, Oct 6, 2020, at 12:31, Marten Seemann wrote:
> One thing that’s a bit annoying about QUIC’s variant format is that 
> there are multiple ways to encode a number. This has led to some 
> complications in the specification (e.g. QUIC requires you to use the 
> minimal encoding for frame types, but allows all encodings everywhere 
> else).
> > cTLS uses a bespoke varint format. Now that QUIC is nearly done, I propose adopting their varint format.
> >

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 5 Oct 2020 18:38:08 -0700

Yeah, I'm certainly sympathetic to this. TBH, from an aesthetic perspective
I prefer what's in cTLS now (though it had the same property) but I figured
that some consistency was nice.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 5 Oct 2020 18:41:44 -0700

On Mon, Oct 5, 2020 at 6:38 PM Martin Thomson <[email-masked]> wrote:

> Oh, I consider that to be a feature.  One that I exploit.  It's sometimes
> awkward to build a structure then prepend a length that has a
> variable-length encoding.  If you know the maximum length, the varint
> encoding allows you to avoid having to move memory.
>
> That said, cTLS won't authenticate both the value and the specific
> encoding of numeric values as the varint encoding is erased.  I don't
> consider that to be a serious problem, but it means that cTLS transcripts
> can't be directly compared.  I would note that in the draft: people
> shouldn't be processing cTLS messages that way, but they might be surprised
> to learn of this.
>

You could fix this by requiring the minimal encoding, right?

-Ekr

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 6 Oct 2020 02:22:49 +0000

Can you just say “QUIC rules but use the minimum possible length”?

---

From: Marten Seemann <[email-masked]>
Date: Tue, 6 Oct 2020 09:31:43 +0700

In that case, why use QUIC's encoding at all? It would just put the burden
on the receiver to check that the minimal encoding was used.
Would it instead make more sense to modify QUIC's encoding, such that the
2-byte encoding doesn't encode the numbers from 0 to 16383, but the numbers
from 64 to (16383 + 64), and equivalently for 4 and 8-byte encodings?

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 5 Oct 2020 20:08:54 -0700

I don't have a strong opinion on whether to require a minimal encoding, but
if we're not going to use QUIC's encoding as-is, then I would rather stick
with the existing scheme, which has twice as large a range for the 1 byte
encoding and is thus more compact for a range of common cases.

-Ekr

---

From: Christian Huitema <[email-masked]>
Date: Mon, 5 Oct 2020 21:58:53 -0700

1994 called. It wanted to talk about distinguished encoding rules.

On 10/5/2020 8:08 PM, Eric Rescorla wrote:
> I don't have a strong opinion on whether to require a minimal
> encoding, but if we're not going to use QUIC's encoding as-is, then I
> would rather stick with the existing scheme, which has twice as large
> a range for the 1 byte encoding and is thus more compact for a range
> of common cases.
>
> -Ekr
>
>
> On Mon, Oct 5, 2020 at 7:31 PM Marten Seemann <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     In that case, why use QUIC's encoding at all? It would just put
>     the burden on the receiver to check that the minimal encoding was
>     used.
>     Would it instead make more sense to modify QUIC's encoding, such
>     that the 2-byte encoding doesn't encode the numbers from 0 to
>     16383, but the numbers from 64 to (16383 + 64), and equivalently
>     for 4 and 8-byte encodings?
>
>     On Tue, Oct 6, 2020 at 9:22 AM Salz, Rich <[email-masked]
>     <mailto:[email-masked]>> wrote:
>

---

From: Rob Sayre <[email-masked]>
Date: Mon, 5 Oct 2020 22:11:26 -0700

> 1994 called. It wanted to talk about distinguished encoding rules.

Could you expand on this idea?

I am not sure what you mean, because most things from circa 1994 are pretty
naive, to my eye.

thanks,
Rob

> I don't have a strong opinion on whether to require a minimal encoding,
> but if we're not going to use QUIC's encoding as-is, then I would rather
> stick with the existing scheme, which has twice as large a range for the 1
> byte encoding and is thus more compact for a range of common cases.
>
> -Ekr
>
>
>> In that case, why use QUIC's encoding at all? It would just put the
>> burden on the receiver to check that the minimal encoding was used.
> TLS mailing [email-masked]://www.ietf.org/mailman/listinfo/tls
>

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 6 Oct 2020 06:36:50 +0000

Hi Ekr,

I had a chat with Richard about this and this change makes a lot of sense (particularly since the current cTLS draft only defines the encoding of varints up to 3 bytes).

In the work on QUIC did you discuss the ability to make the encoding such that there are no ways to express a number in two different ways, as shown in your example with the single byte 25 decoding to 37 and the two byte sequence 40 25?

Ciao
Hannes

From: TLS <[email-masked]> On Behalf Of Eric Rescorla
Sent: Tuesday, October 6, 2020 3:38 AM
To: Marten Seemann <[email-masked]>
Cc: <[email-masked]> <[email-masked]>
Subject: Re: [TLS] PR#28: Converting cTLS to QUIC-style varints

Yeah, I'm certainly sympathetic to this. TBH, from an aesthetic perspective I prefer what's in cTLS now (though it had the same property) but I figured that some consistency was nice.

-Ekr

On Mon, Oct 5, 2020 at 6:31 PM Marten Seemann <[email-masked]<mailto:[email-masked]>> wrote:
One thing that’s a bit annoying about QUIC’s variant format is that there are multiple ways to encode a number. This has led to some complications in the specification (e.g. QUIC requires you to use the minimal encoding for frame types, but allows all encodings everywhere else).
It would be nice to have an unambiguous way to encode a number.

On Tue, Oct 6, 2020 at 07:35 Eric Rescorla <[email-masked]<mailto:[email-masked]>> wrote:
Hi folks,

Any objections?
-Ekr

_______________________________________________
TLS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tls
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 06 Oct 2020 17:52:15 +1100

On Tue, Oct 6, 2020, at 17:36, Hannes Tschofenig wrote:
> In the work on QUIC did you discuss the ability to make the encoding 
> such that there are no ways to express a number in two different ways, 
> as shown in your example with the single byte 25 decoding to 37 and the 
> two byte sequence 40 25? 

For QUIC, where the performance impact of serialization and parsing was considered important, the additional cost involved was a little higher than I think some people wanted.  It also makes the encoding more complex.

A "distinguished" or canonical encoding wasn't critical there.  It's probably not critical here either.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 6 Oct 2020 14:25:45 +0000

*   I don't have a strong opinion on whether to require a minimal encoding, but if we're not going to use QUIC's encoding as-is, then I would rather stick with the existing scheme, which has twice as large a range for the 1 byte encoding and is thus more compact for a range of common cases.

I was only thinking code re-use, but it’s only a couple of instructions so this is fine with me.  Leave it as-is.

---

From: Michael D'Errico <[email-masked]>
Date: Tue, 6 Oct 2020 10:28:30 -0400

> 1994 called. It wanted to talk about distinguished encoding rules.

Yes, DER vs. BER.  ASN.1 is actually pretty
cool (bits on the wire in X.690, not X.680).

The encoding also reminds me of UTF-8 which is
a variable length:

     0xxxxxxx
     110xxxxx 10yyyyyy
     1110xxxx 10yyyyyy 10zzzzzz
     11110xxx 10yyyyyy 10zzzzzz 10wwwwww

UTF-8 requires using the fewest number of bytes
to represent a Unicode code point.

Mike

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Tue, 6 Oct 2020 14:45:42 +0000

I would prefer the minimum encoding length.

From: TLS <[email-masked]> on behalf of "Salz, Rich" <rsalz=[email-masked]>
Date: Monday, October 5, 2020 at 22:23
To: Eric Rescorla <[email-masked]>, Marten Seemann <[email-masked]>
Cc: "<[email-masked]>" <[email-masked]>
Subject: Re: [TLS] PR#28: Converting cTLS to QUIC-style varints

---

From: Michael D'Errico <[email-masked]>
Date: Tue, 6 Oct 2020 12:20:32 -0400

> I would prefer the minimum encoding length.

But if the spec says MUST be minimum length,
I would feel compelled to check every one.

Please don't require minimum length unless
security would be weaker without it.

Mike

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 6 Oct 2020 16:24:27 +0000

"Be liberal in what you accept" no?

If the fields are used in digests/hashing/signatures, then it matters.  (The DER/BER comment upthread)

---

From: Michael D'Errico <[email-masked]>
Date: Tue, 6 Oct 2020 13:00:32 -0400

> "Be liberal in what you accept" no?

A requirement of MUST in a security protocol usually
means (or at least should mean) that there's a good
reason for the thing, whatever it is, so if you are
not checking it, then bad things can happen.  I try
to heed Postel's advice though, and can sometimes be
more liberal than the spec says if I really know
what the trade-offs are.  With crypto though, I don't
pretend to be expert in it, I'm kind of afraid of it,
and I'm not sure how anyone can claim to understand
it well enough to say, for example, that ECC is so
much better than everything else.  What if it's all
broken?  We're getting rid of everything else that
we know works....

> If the fields are used in digests/hashing/signatures, then it matters.  (The DER/BER comment upthread)

It matters in X.509 certificates because the basic
encoding rules (BER) allow you to specify the same
thing in different ways.  With DER, there is only
one way to encode every element, so everybody will
come up with the same string of bytes and hashes of
those strings will be the same, signatures will
verify, etc.

I haven't read whatever spec cTLS is, but know that
if I decided to implement it someday and there's a
"varints MUST be minimum length" requirement, I'd
assume that it's important and would check for it.
If this is a waste of CPU cycles, then please don't
make it a requirement.

Mike

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 6 Oct 2020 17:07:19 +0000

Okay, we disagree about things.  Not sure it's worth proceeding.

>    I haven't read whatever spec cTLS is

Got it.

---

From: Michael D'Errico <[email-masked]>
Date: Tue, 6 Oct 2020 13:10:09 -0400

>>     I haven't read whatever spec cTLS is

My advice is universally applicable, to whatever
spec you are working on.  cTLS, QUIC, whatever-TF.

Mike

---

From: Christian Huitema <[email-masked]>
Date: Tue, 6 Oct 2020 10:12:39 -0700

> those strings will be the same, signatures will

Well, we have learned a few things since 1994. The DER rules are defined
to allow a "re-encoding" workflow:

* Sender side: prepare the message, encode with DER, sign the result

* Receiver side: receive the message, parser with generic ASN.1 decoder,
process the message using the "parsed" representation, re-encode with
DER, check the signature.

Experience showed that this workflow is very problematic, because the
parse/reencode process may introduce subtle changes and the signature
will fail.  One may argue that these changes are due to implementation
bugs, but fact it that this is a rich environment for growing bugs.
Based on experience, the receiver side is better done as:

* Receiver side: receive the message, save it, parse and process, and
when it is time to verify the signature go back to the original message
and check the signature.

If we do that, then there is no reason to mandate minimal length
encoding. And TLS already does that. For example, we do not reorder
extensions according to some canonical rules before placing them in the
transcript.

-- Christian Huitema

---

From: Michael D'Errico <[email-masked]>
Date: Tue, 6 Oct 2020 14:37:01 -0400

I think we are in agreement.

> DER, check the signature.

I recall that at least one root certificate had a
SEQUENCE encoded using BER-but-not-DER (?)  Yeah if
your software re-encoded that, it would no longer
be the same sequence of bytes.

> when it is time to verify the signature go back to the original message

This is how I did X.509 verification, though I was
late to the game and the advice was already there
to accept a BER-encoded certificate.  Not sure if
I would have done the DER re-encoding bit if that
was the current advice at the time since it seems
like the wrong thing to do, but maybe I would have.

> extensions according to some canonical rules before placing them in the

I was disappointed to see that the TLS 1.3 spec now
has a requirement to put one of the ClientHello
extensions in a specific place (last in the list).

We discussed this at length during the development
of either TLS 1.2 or one of the extensions (maybe
renegotiation-info?) and we ultimately came to what
I believe was the correct decision never to require
any ordering of the extensions.  Sad to see the
group capitulated to whomever said it would make
their software easier to write (which I doubt).

Mike

---

From: Nick Harper <[email-masked]>
Date: Tue, 6 Oct 2020 16:18:39 -0700

> their software easier to write (which I doubt).

Hopefully https://tools.ietf.org/html/rfc8446#section-4.2.11.2 makes it
clear why the pre_shared_key extension must be at the end of the list.

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 07 Oct 2020 11:23:48 +1100

> when it is time to verify the signature go back to the original message

I think that you mean:

receive the message, check the signature, then parse and process if that passes

> extensions according to some canonical rules before placing them in the

This I agree with.  But cTLS doesn't work that way because the signature - such as it is - applies at the next layer, which appears after the encoding is erased.  And that is important here.  The encoding we're talking about is a compression function only.  Not having a canonical form means adding an inefficiency, but it has little bearing on the process you describe, which would be modified to:

receive the message, decompress the message, check the signature, then parse and process if that passes

In TLS we don't follow that ordering either because we all routinely process tons of stuff before we get to the Finished/CertificateVerify.  Having those at the end makes a ton of sense, for a variety of reasons, but it does mean that we build a protocol on credit.  And we have plenty of experience, I hope, in dealing with bad credit in TLS.

---

From: Nick Harper <[email-masked]>
Date: Tue, 6 Oct 2020 17:33:32 -0700

I have no strong opinion on how this is formatted. I'd base my decision on
what the maximum value cTLS needs to encode: If 2^22-1 is sufficient, let's
keep it as is, otherwise let's change it to the QUIC format (or some other
change to increase the max value). I do like that the existing scheme,
compared to QUIC varints, is more efficient for values 64-127 and just as
efficient for the rest.

---

From: Christian Huitema <[email-masked]>
Date: Tue, 6 Oct 2020 17:46:22 -0700

> receive the message, check the signature, then parse and process if that passes
You would think that, but the X.509 design meant that the application
would received all its data in parsed form, and then re-serialize the
parts that it needed to check, such as for example certificates. And if
you do that, you must absolutely guarantee that parse and serialize are
bijections between the application representation and the byte
sequences. Which leads to requiring "distinguished encoding rules",
including not just the coding of integers but also ordering of elements
in sets or representations of character strings. And yes, this turned
out to be a really bad idea.
> In TLS we don't follow that ordering either because we all routinely process tons of stuff before we get to the Finished/CertificateVerify.  Having those at the end makes a ton of sense, for a vari…

Yes. And I do also agree with your statement that *not* requiring
minimal length encoding makes some of the code paths much simpler. I use
that for example to encode the length field of Quic packets.

-- Christian Huitema

---

From: Peter Gutmann <[email-masked]>
Date: Wed, 7 Oct 2020 01:13:10 +0000

Christian Huitema <[email-masked]> writes:

>Receiver side: receive the message, parser with generic ASN.1 decoder,
>process the message using the "parsed" representation, re-encode with DER,
>check the signature.

Except that no true Scots... uhh, sane person ever even tried that.  I've
heard that there was one implementation, done in Europe, but have never seen
it.  Everyone else just treated what arrived as a blob and went with that.
That's why, years ago, on the PKIX list I said "there is only one encoding
rule and that is memcpy()".

>Well, we have learned a few things since 1994.

Except for the people who did secure XML, who not only ignored what we've
learned so far but made it worse by making it active content rather than flat
byte strings.

But that seems to be the rule for XML design in any case.

Peter.

---

From: Watson Ladd <[email-masked]>
Date: Tue, 6 Oct 2020 21:30:43 -0700

> when it is time to verify the signature go back to the original message

Examining signed data ahead of verification, particularly to determine
what should have signed it, is fraught with issues. In particular
parser errors that would only be exploitable by trusted parties become
exploitable by anyone. Furthermore, you should know the expected
signer ahead of time. That this isn't possible in X509 due to
inordinate flexibility is one of many pitfalls for implementors. This
sort of issue periodically leads to authentication bypasses and other
such fun.

Sincerely,
Watson Ladd

--
Astra mortemque praestare gradatim

---

From: Michael D'Errico <[email-masked]>
Date: Wed, 7 Oct 2020 13:15:19 -0400

On 10/6/20 10:28, I wrote:
> ASN.1 is actually pretty cool ....

My favorite sentence in any RFC is:

    In ASN.1, EXPLICIT tagging is implicit
    unless IMPLICIT is explicitly specified.

(Not brown-nosing....)

Mike

---

From: Mohit Sethi M <[email-masked]>
Date: Wed, 7 Oct 2020 17:47:12 +0000

A strong +1 on the security issues of decode -> extract -> re-encode -> 
verify signature flow. The lack of canonical encoding can also mean that 
the resulting bytes can be different in different encoder/decoder 
implementations.

It would have been nice to have canonical JSON. Some implementations 
support it with flags such as "JSON_PRESERVE_ORDER" 
(https://jansson.readthedocs.io/en/2.8/apiref.html). Some specs try to 
specify things like keys should be in alphabetical order but fail to 
realize the complication with nested structures.

CBOR has thankfully specified canonical representation (called as 
deterministic encoding in 7049bis). Some CBOR libraries have added 
support: https://github.com/agronholm/cbor2/issues/6.

--Mohit

> sort of issue periodically leads to authentication bypasses and other
A strong +1 to this.

---

From: Anders Rundgren <[email-masked]>
Date: Wed, 7 Oct 2020 21:14:33 +0200

> realize the complication with nested structures.

Hi Mohit,

Since a few months back there is an RFC (8785) which specifies a
scheme for canonicalizing JSON, albeit limited to the JSON subset
supported by ECMAScript and browsers.

It can be tested in an on-line application where it is combined with
JWS: https://mobilepki.org/jws-jcs/home

Anders

---

From: Mohit Sethi M <[email-masked]>
Date: Wed, 7 Oct 2020 19:57:19 +0000

Hi Anders,

> JWS: https://mobilepki.org/jws-jcs/home

Cool ! Thanks for informing and thanks for doing the work!

--Mohit

---

From: "Michael D'Errico" <[email-masked]>
Date: Sat, 10 Oct 2020 14:41:11 -0400

> Hopefully https://tools.ietf.org/html/rfc8446#section-4.2.11.2
> makes it clear why the pre_shared_key extension must be at
> the end of the list. 

I see what was done, but it still makes me a bit
sad that whatever security property was desired
couldn't have been done differently to avoid this
requirement.  Hopefully we never discover a new
extension that should be "last" ....

Mike