Subject: Connection ID Space

From: Martin Duke <[email-masked]>
Date: Fri, 26 Apr 2019 14:42:59 -0700

Although it is quite late in the day for this objection, I find that we
already bumping up against the limits of the 18 byte connection ID.

Inevitably, this touches on some issues we're exploring in the
quic-load-balancers
draft
<https://github.com/martinduke/draft-duke-quic-load-balancers/blob/master/draft-duke-quic-load-balancers.md>,
which isn't adopted, and many of you are no doubt happily ignoring.

There are basically three uses of the 144 bits of CID entropy in QUIC-LB:

1) Server ID encoding -- as there are only so many servers one might have,
16 bits is usually enough for this.
2) "Private" server entropy -- the load balancer doesn't care what the bits
are, but they're protected by crypto. The server created and encrypted the
CID, so it can use this for its local purposes, but no one else can. The
amount of this depends, but if you're using a 256-bit block cipher it's
something like 240 bits -- plenty
3)  "Public" entropy -- not encrypted, these bits are readable by anyone
who is familiar with the encoding scheme. With a 16 B encryption block, we
have only 16 bits to play with here.

QUIC-LB is already taking 2 of these last bits for config rotation, which
allows server pools to incrementally deploy new CID keys or encoding
schemes. So we're down to 14 bits.

The public entropy is useful for a growing pile of trusted hardware devices
that are presumably not going to perform CID decryption. On f5 boxes, we
have a hardware disaggregator for the many dataplane processors and threads
that are operating. Today, we'd use perhaps 8 of the bits, but in future
architectures even 14 might not quite be enough.

And now, having chatting with Manasi Deval from Intel after her Prague talk
about hardware offload, I'm fairly sure we're going to need 4 to cheaply
encode the connection ID length, at least for servers that want to use this
service.

If we ever wanted a solution that allowed multiple tiers of load balancing,
which is not currently in scope for QUIC-LB, we don't have nearly enough
space.

I see three options:
1) Ignore the problem -- this may restrict the number of use cases of QUIC
in the future, especially as Connection ID lengths are part of the
invariants.
2) Redefine the DCIL and SCIL fields to be half the connection ID length.
Thus CIDs would always have even lengths but could run up to 30 Bytes. For
what it's worth, this also enables 2B CIDs. This is heavyweight as a diff
to the current spec but is the most futureproof. There are, of course,
other mappings with different properties but similar bureaucratic costs.
3) There are two reserved bits in the relevant header types. If we allowed
them to not resolve to zero post-protection, and therefore opened them to
encode QUIC-LB config rotation in the clear, this would mitigate the
problem and I am fairly confident that these bits would vary enough to not
ossify. This is a lightweight protocol change but may just partly relieve
the pressure we're feeling now.

What are people's initial attitudes? Unless there is strong consensus for
Option #1 I can file an issue and we can get it on an agenda to discuss in
person.

Martin

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 26 Apr 2019 15:37:01 -0700

Do I understand correctly that this would be a change to the invariants?

On Fri, Apr 26, 2019 at 2:43 PM Martin Duke <[email-masked]> wrote:

> Inevitably, this touches on some issues we're exploring in the quic-load-balancers
> draft
> 2) "Private" server entropy -- the load balancer doesn't care what the
> bits are, but they're protected by crypto. The server created and encrypted
> the CID, so it can use this for its local purposes, but no one else can.
> The amount of this depends, but if you're using a 256-bit block cipher it's
> schemes. So we're down to 14 bits..
>
> The public entropy is useful for a growing pile of trusted hardware
> devices that are presumably not going to perform CID decryption. On f5
> boxes, we have a hardware disaggregator for the many dataplane processors
> and threads that are operating. Today, we'd use perhaps 8 of the bits, but
> in future architectures even 14 might not quite be enough.
>
>
> And now, having chatting with Manasi Deval from Intel after her Prague
> talk about hardware offload, I'm fairly sure we're going to need 4 to
> cheaply encode the connection ID length, at least for servers that want to
> use this service.
>
> If we ever wanted a solution that allowed multiple tiers of load
> balancing, which is not currently in scope for QUIC-LB, we don't have
> nearly enough space.
>
> I see three options:

---

From: Martin Duke <[email-masked]>
Date: Fri, 26 Apr 2019 21:02:55 -0700

Option 2 would be a change to the invariants.

On Fri, Apr 26, 2019, 15:37 Ryan Hamilton <[email-masked]> wrote:

>> 1) Server ID encoding -- as there are only so many servers one might
>> have, 16 bits is usually enough for this.
>> 1) Ignore the problem -- this may restrict the number of use cases of
>> QUIC in the future, especially as Connection ID lengths are part of the
>> invariants.
>> 3) There are two reserved bits in the relevant header types. If we
>> allowed them to not resolve to zero post-protection, and therefore opened
>> them to encode QUIC-LB config rotation in the clear, this would mitigate
>> the problem and I am fairly confident that these bits would vary enough to
>> not ossify. This is a lightweight protocol change but may just partly
>> relieve the pressure we're feeling now.
>>

---

From: Mike Bishop <[email-masked]>
Date: Tue, 30 Apr 2019 17:50:21 +0000

I’m in the semi-ignoring QUIC-LB camp, as I believe it’s an important problem for other people to solve right now.  (Thanks, Martin!)

I take no particular position on whether the 18 byte maximum is too small; it feels exceptionally large to me, but if we’re getting actual implementation experience that shows otherwise, I’m happy to have the data.  If it needs to change, I agree that Option 2 appears the most futureproof, and if something needs to change in the invariants, this is almost the latest we could possibly do it.  Now or never.

I think Option 1a is to note that the invariants don’t limit the length of CIDs in a given version in short-header packets; a future version could define much larger CIDs to be valid post-handshake.  Coupled with a way to encourage/require clients to move off of the handshake CID immediately post-handshake, this might be sufficient for the real world.

From: QUIC <[email-masked]> On Behalf Of Martin Duke
Sent: Friday, April 26, 2019 9:03 PM
To: Ryan Hamilton <[email-masked]>
Cc: IETF QUIC WG <[email-masked]>
Subject: Re: Connection ID Space

On Fri, Apr 26, 2019, 15:37 Ryan Hamilton <[email-masked]<mailto:[email-masked]>> wrote:

   On Fri, Apr 26, 2019 at 2:43 PM Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:

      Although it is quite late in the day for this objection, I find that we already bumping up against the limits of the 18 byte connection ID.

      Inevitably, this touches on some issues we're exploring in the quic-load-balancers draft<https://github.com/martinduke/draft-duke-quic-load-balancers/blob/master/draft-duke-quic-load-balancers.md>, which isn't adopted, and many of you are no doubt happily ignoring.

      1) Server ID encoding -- as there are only so many servers one might have, 16 bits is usually enough for this.

      2) "Private" server entropy -- the load balancer doesn't care what the bits are, but they're protected by crypto. The server created and encrypted the CID, so it can use this for its local purposes, but no one else can. The amount of this depends, but if you're using a 256-bit block cipher it's something like 240 bits -- plenty

      3)  "Public" entropy -- not encrypted, these bits are readable by anyone who is familiar with the encoding scheme. With a 16 B encryption block, we have only 16 bits to play with here.

      QUIC-LB is already taking 2 of these last bits for config rotation, which allows server pools to incrementally deploy new CID keys or encoding schemes. So we're down to 14 bits..

      The public entropy is useful for a growing pile of trusted hardware devices that are presumably not going to perform CID decryption. On f5 boxes, we have a hardware disaggregator for the many dataplane processors and threads that are operating. Today, we'd use perhaps 8 of the bits, but in future architectures even 14 might not quite be enough.

      And now, having chatting with Manasi Deval from Intel after her Prague talk about hardware offload, I'm fairly sure we're going to need 4 to cheaply encode the connection ID length, at least for servers that want to use this service.

      If we ever wanted a solution that allowed multiple tiers of load balancing, which is not currently in scope for QUIC-LB, we don't have nearly enough space.

      I see three options:

      1) Ignore the problem -- this may restrict the number of use cases of QUIC in the future, especially as Connection ID lengths are part of the invariants.

      2) Redefine the DCIL and SCIL fields to be half the connection ID length. Thus CIDs would always have even lengths but could run up to 30 Bytes. For what it's worth, this also enables 2B CIDs. This is heavyweight as a diff to the current spec but is the most futureproof. There are, of course, other mappings with different properties but similar bureaucratic costs.

      3) There are two reserved bits in the relevant header types. If we allowed them to not resolve to zero post-protection, and therefore opened them to encode QUIC-LB config rotation in the clear, this would mitigate the problem and I am fairly confident that these bits would vary enough to not ossify. This is a lightweight protocol change but may just partly relieve the pressure we're feeling now.

      What are people's initial attitudes? Unless there is strong consensus for Option #1 I can file an issue and we can get it on an agenda to discuss in person.

      Martin

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 30 Apr 2019 20:31:47 -0400

On Wed, May 1, 2019, at 03:50, Mike Bishop wrote:
> I think Option 1a is to note that the invariants don’t limit the length 
> of CIDs in a given version in short-header packets; a future version 
> could define much larger CIDs to be valid post-handshake. Coupled with 
> a way to encourage/require clients to move off of the handshake CID 
> immediately post-handshake, this might be sufficient for the real world.

That's on the TODO list anyway, as we agreed to remove the 0/4-18 restriction for the short packet header a while back.