Subject: [TLS] A flags extension

From: Yoav Nir <[email-masked]>
Date: Mon, 25 Mar 2019 22:09:35 +0100

Hi.  Today at the TLS meeting, there was a discussion at the mic about 1-bit extensions that only serve to indicate support for an optional feature. EKR commented that such extensions take 4 bytes each and that maybe we need to replace them with a flags extension.

So I threw together a quick -00 draft with an extension that does just that [1].

Comments are welcome.

Yoav

[1] https://datatracker.ietf.org/doc/draft-nir-tls-tlsflags/ <https://datatracker.ietf.org/doc/draft-nir-tls-tlsflags/>

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 25 Mar 2019 23:21:30 +0000

That was FAST.

Looks good.

From: Yoav Nir <[email-masked]>
Date: Monday, March 25, 2019 at 10:10 PM
To: "[email-masked]" <[email-masked]>
Subject: [TLS] A flags extension

Comments are welcome.

Yoav

[1] https://datatracker.ietf.org/doc/draft-nir-tls-tlsflags/<https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dnir-2Dtls-2Dtlsflags_&d=DwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=4LM0GbR0h9Fvx86FtsKI-w&m=vI7f_lTRNmPJU_afvRwCfhhfrX-GjDMCcpRfUZF254A&s=rcJFjdA-91NCh1SnDreFMvQpvRnUwUJYHXgYOz05wzI&e=>

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 26 Mar 2019 04:06:03 -0400

This needs more space for each flag.  8 bits is a pretty small space.  If you are concerned with the size of the result, we have some variable-length integer encodings you could try.

On Tue, Mar 26, 2019, at 00:22, Salz, Rich wrote:
>  
> That was FAST.
> 
> 
> Looks good.
> 
> 
> *From: *Yoav Nir <[email-masked]>
> *Date: *Monday, March 25, 2019 at 10:10 PM
> *To: *"[email-masked]" <[email-masked]>
> *Subject: *[TLS] A flags extension
> 
> 
> Hi. Today at the TLS meeting, there was a discussion at the mic about 
> 1-bit extensions that only serve to indicate support for an optional 
> feature. EKR commented that such extensions take 4 bytes each and that 
> maybe we need to replace them with a flags extension. 
> 
> 
> [1] https://datatracker.ietf.org/doc/draft-nir-tls-tlsflags/ 
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dnir-2Dtls-2Dtlsflags_&d=DwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=4LM0GbR0h9Fvx86FtsKI-w&m=vI7f_lTRNmPJU_afvRwCfhhfrX-GjDMCcpRfUZF254A&s=rcJFjdA-91NCh1SnDreFMvQpvRnUwUJYHXgYOz05wzI&e=>
> 
> 
> 
> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 26 Mar 2019 08:18:13 +0000

>    This needs more space for each flag.  8 bits is a pretty small space.  If you are concerned with the size of the result, we have some variable-length integer encodings you could try.
  
Enh, just take 16 bits.  Nobody will ever need more than 640k

---

From: Yoav Nir <[email-masked]>
Date: Tue, 26 Mar 2019 10:11:32 +0100

> On 26 Mar 2019, at 9:06, Martin Thomson <[email-masked]> wrote:
> 

Ah, the beautiful variable length encodings.  Like:

 - 0x00 - 0xBF - for standards-action allocations
 - 0xC0,0x00 - 0xEF,0xFF - for non-standards track
- 0xF0,0x00 - 0xFF,0xFF - for private use among consenting parties.

Are we really worried that we’re going to have more than 255 optional features for TLS?

---

From: Tom Ritter <[email-masked]>
Date: Tue, 26 Mar 2019 09:24:14 +0000

> Are we really worried that we’re going to have more than 255 optional features for TLS?

Probably not; but what about an optional feature that needs 3 bits? Is
it better to kick them off into 4 bytes?

-tom

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Tue, 26 Mar 2019 10:35:54 +0100

On Tue, 2019-03-26 at 10:11 +0100, Yoav Nir wrote:
> > This needs more space for each flag.  8 bits is a pretty small
> > space.  If you are concerned with the size of the result, we have
> > some variable-length integer encodings you could try.
> 

Given that adding an extended flags extension which can hold even more
flags is also easy, the 255-optional features do not seem so limited. 

Going into the extension itself, the array FlagExtensionType seems to
be the TLS-way to define such variable, but flags are easier, more
efficient to parse, and take less space if they are bits on an integer
value (32-bit or 64-bit). Have you considered a simpler structure like
that?

struct {
   uint64 flags;
   uint64 ext_flags1;
   uint64 ext_flags2;
   uint24 ext_flags3;
   uint16 priv_flags;
} Flags;

The advantage is that it can carry the same information with much less
bytes on the wire and it is easier to parse in low level languages.

The disadvantage is that an extension flag would now need to specify
the bit it occupies _and_ the particular element it is set to.

regards,
Nikos

---

From: Yoav Nir <[email-masked]>
Date: Tue, 26 Mar 2019 10:49:33 +0100

> On 26 Mar 2019, at 10:35, Nikos Mavrogiannopoulos <[email-masked]> wrote:
> 

I thought about that. I guess it depends on how many of these optional features we expect to be declared at the same time. 

With the current way the draft is written, if the client supports 12 such extensions, the extension takes 16 bytes.  With a bitstring, it’s always the same length. so we’d need 36 bytes for a 256-bit space. If the client supports 100 extensions, my encoding takes 104 bytes while the bitstring is still 36 bytes.

The question is which is more likely?

Yoav

---

From: Nick Harper <[email-masked]>
Date: Tue, 26 Mar 2019 10:58:27 +0100

One option for a bitstring could be to encode it in little endian and
truncate 0 bytes at the end of the encoding.

On Tue, Mar 26, 2019 at 10:50 AM Yoav Nir <[email-masked]> wrote:

> I thought about that. I guess it depends on how many of these optional
> features we expect to be declared at the same time.
>
> With the current way the draft is written, if the client supports 12 such
> extensions, the extension takes 16 bytes.  With a bitstring, it’s always
> the same length. so we’d need 36 bytes for a 256-bit space. If the client
> supports 100 extensions, my encoding takes 104 bytes while the bitstring is
> still 36 bytes.
>

---

From: Peter Gutmann <[email-masked]>
Date: Tue, 26 Mar 2019 11:05:30 +0000

>Are we really worried that we’re going to have more than 255 optional

You're new here aren't you?

Peter.

---

From: Yoav Nir <[email-masked]>
Date: Tue, 26 Mar 2019 12:20:08 +0100

> On 26 Mar 2019, at 12:05, Peter Gutmann <[email-masked]> wrote:
> 

No, but I’m looking at the TLS registries ( https://www.iana.org/assignments/tls-parameters/tls-parameters.xml <https://www.iana.org/assignments/tls-parameters/tls-parameters.xml> ) and the only one that has that many entires is the ciphersuites.

---

From: Hubert Kario <[email-masked]>
Date: Tue, 26 Mar 2019 14:45:43 +0100

On Monday, 25 March 2019 22:09:35 CET Yoav Nir wrote:
> Hi.  Today at the TLS meeting, there was a discussion at the mic about 1-bit
> extensions that only serve to indicate support for an optional feature. EKR
> commented that such extensions take 4 bytes each and that maybe we need to
> replace them with a flags extension.
> 

I don't think that "penny-pinching" the 4 bytes necessary to send a flag is 
worth the interoperability problems, and increased complexing of parsing 
Client Hello. Especially if we go the route of actual bit flags.

I think the likelihood of bugs in that code over the possible bytes saved 
makes it a net negative.

yes, TLS is quite chatty protocol, it could encode values much more tightly, 
but I think we all remember the bugs related to ASN.1 parsing from inside of 
PKCS#1 v1.5 signatures
-- 
Regards,
Hubert Kario
Senior Quality Engineer, QE BaseOS Security team
Web: www.cz.redhat.com
Red Hat Czech s.r.o., Purkyňova 115, 612 00  Brno, Czech Republic

---

From: Yoav Nir <[email-masked]>
Date: Tue, 26 Mar 2019 16:38:11 +0100

> On 26 Mar 2019, at 14:45, Hubert Kario <[email-masked]> wrote:
> 

Right. Which is why I went with a 1-byte encoding rather than a bitstring.

> I think the likelihood of bugs in that code over the possible bytes saved 

I don’t think so. My encoding is not all that complex.

> but I think we all remember the bugs related to ASN.1 parsing from inside of 

Complexity is on a spectrum.  DER encoding is pretty far on this spectrum.  A list of 1-octet identifiers is on the other end. A bitstring is more complex than the identifier list, but not anywhere near DER.

I don’t think we should project the failings of DER parsing to the parsing of much simpler structures.

Yoav

---

From: Hubert Kario <[email-masked]>
Date: Tue, 26 Mar 2019 18:49:50 +0100

On Tuesday, 26 March 2019 16:38:11 CET Yoav Nir wrote:
> >> So I threw together a quick -00 draft with an extension that does just
> >> that
> >> [1].
> >> 
> >> Comments are welcome.
> > 
> > yes, TLS is quite chatty protocol, it could encode values much more
> > tightly, but I think we all remember the bugs related to ASN.1 parsing
> > from inside of PKCS#1 v1.5 signatures
> 
> Complexity is on a spectrum.  DER encoding is pretty far on this spectrum. 
> A list of 1-octet identifiers is on the other end. A bitstring is more
> complex than the identifier list, but not anywhere near DER.

1-octet identifiers may not be considered extensible enough
(yes, you can add another extension, but the first extension to use it will be 
paying an additional price of 2 bytes on top of the extension overhead; same 
if you just need to use only one flag, then you are paying the same price for 
every connection)

2-octet identifiers asymptotically approach 2-octet saved per flag, which is 
about 50% saved per flag, I don't see it as much

to approach it from another way: while I think we will, sometime in the 
future, reach a situation when we have few hundred flag extensions *defined* , 
I do not see a future in which we will need to *use* more than few dozen flag 
extensions in any real world client. So we are talking about a possible saving 
of around 100 bytes in ClientHello (36 extensions * 3 bytes saved) in this 
proposal

won't this be completely erased by any post-quantum key share?

> of much simpler structures.

yes, that was an extreme example; I was replying to all sent messages so far

---

From: Yoav Nir <[email-masked]>
Date: Tue, 26 Mar 2019 21:18:06 +0100

> On 26 Mar 2019, at 18:49, Hubert Kario <[email-masked]> wrote:
> 

A few dozen?  I was thinking under 10 in a typical client.

> won't this be completely erased by any post-quantum key share?

I think implementing (or not) draft-ietf-tls-certificate-compression would have a much more significant effect than anything we save here.

Yoav

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 26 Mar 2019 16:53:55 -0500

> I don’t think so. My encoding is not all that complex.

It would be pretty easy to forget to sort the values (which, btw, maybe we
want to require, to make duplicate detection easier).

-Ben

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Wed, 27 Mar 2019 10:52:20 +0100

On Tue, 2019-03-26 at 10:49 +0100, Yoav Nir wrote:
> > > Are we really worried that we’re going to have more than 255
> > > optional
> > > features for TLS?
> > 
> I thought about that. I guess it depends on how many of these
> optional features we expect to be declared at the same time. 
> 
> With the current way the draft is written, if the client supports 12
> such extensions, the extension takes 16 bytes.  With a bitstring,
> it’s always the same length. so we’d need 36 bytes for a 256-bit
> space. If the client supports 100 extensions, my encoding takes 104
> bytes while the bitstring is still 36 bytes.

Right. What about defining a set of extensions (e.g., 2 extensions) of
flags as:

struct {
   uint64 flags;
} Flags;

That way there are 128 different flags available for assignment with a
cost is 4+8-bytes for every 64 flags.

regards,
Nikos

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 27 Mar 2019 12:26:47 +0100

> Right. What about defining a set of extensions (e.g., 2 extensions) of

If we're going to be doing this kind of bit-shaving, this is the way to
go, starting with a single CommonFlags extension -- and maybe even a
uint32 or uint16, with the bitfield registry under tight WG control.  If
we exhaust that space, then we just define a CommonFlags2 extension.

If someone wants an experimental boolean extension to play with, they
can always use an empty extension.  They can apply for a bit in
CommonFlags if they find that the compactness is warranted.

Keep it simple.

     --dkg

---

From: Yoav Nir <[email-masked]>
Date: Wed, 27 Mar 2019 13:08:32 +0100

> On 27 Mar 2019, at 12:26, Daniel Kahn Gillmor <[email-masked]> wrote:
> 

OK. You got me convinced.

In the spirit of revising quickly and revising often, I’ve uploaded version -01:

HTML: https://datatracker.ietf.org/doc/html/draft-nir-tls-tlsflags <https://datatracker.ietf.org/doc/html/draft-nir-tls-tlsflags>
DIFF: https://www.ietf.org/rfcdiff?url2=draft-nir-tls-tlsflags-01 <https://www.ietf.org/rfcdiff?url2=draft-nir-tls-tlsflags-01>

Yoav

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 27 Mar 2019 09:42:49 -0400

Why not go all in - make this a byte string and start from 0x80 in the first byte.  When we define the 9th flag, we add another byte.  Then you have up to 2040 flags (though it might pay to split the space before that).

struct {
  opaque<1..255> flags;
} Flags;

Otherwise, the first adopter of this pays 10 bytes where they would previously have paid 4.  Obviously there is a network effect at the third.  Since I'm writing a draft that will aim to depend on this, I have a vested interest in using this.

If you wanted to make it more attractive to me, then maybe porting some of the existing flags across might make it more appealing.

On Wed, Mar 27, 2019, at 13:08, Yoav Nir wrote:
> HTML: https://datatracker.ietf.org/doc/html/draft-nir-tls-tlsflags
> DIFF: https://www.ietf.org/rfcdiff?url2=draft-nir-tls-tlsflags-01
> 
> Yoav
>

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 27 Mar 2019 06:46:23 -0700

Of course, at some point it starts to make sense to do RLE.

-Ekr

On Wed, Mar 27, 2019 at 6:43 AM Martin Thomson <[email-masked]> wrote:

> Why not go all in - make this a byte string and start from 0x80 in the
> first byte.  When we define the 9th flag, we add another byte.  Then you
> have up to 2040 flags (though it might pay to split the space before that).
>
> struct {
>   opaque<1..255> flags;
> } Flags;
>
> Otherwise, the first adopter of this pays 10 bytes where they would
> previously have paid 4.  Obviously there is a network effect at the third.
> Since I'm writing a draft that will aim to depend on this, I have a vested
> interest in using this.
>
> If you wanted to make it more attractive to me, then maybe porting some of
> the existing flags across might make it more appealing.
>

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 27 Mar 2019 09:50:13 -0400

> Of course, at some point it starts to make sense to do RLE.

I considered that, but I don't see this being sufficiently sparsely populated.

---

From: Hubert Kario <[email-masked]>
Date: Thu, 28 Mar 2019 14:54:02 +0100

On Wednesday, 27 March 2019 14:42:49 CET Martin Thomson wrote:
> Why not go all in - make this a byte string and start from 0x80 in the first
> byte.  When we define the 9th flag, we add another byte.  Then you have up
> to 2040 flags (though it might pay to split the space before that).
> 
> struct {
>   opaque<1..255> flags;
> } Flags;
> 

this will be the only place with a bit mask in TLS, I have serious doubts 
about the correctness of code that will handle this stuff
 
> the existing flags across might make it more appealing.

what about making sure that the legacy and flags remain in-sync? we will have 
to send the legacy encoding for many years to come, so only thing it would 
possibly reduce the size of is ServerHello or EncryptedExtensions

> > > On 27 Mar 2019, at 12:26, Daniel Kahn Gillmor <[email-masked]>
> > > wrote:> >

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 29 Mar 2019 05:23:51 -0400

> possibly reduce the size of is ServerHello or EncryptedExtensions

Those are messages where we have size pressure.