Subject: Stream vs. connection

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 5 Dec 2016 18:03:07 +0100

Hi again,

here comes further information on point one regarding the questions how to treat stream-based information vs. connection-based information. This is currently my main point about the transport design of quic. To be more concrete, quic has the concept of packets and frames. While one regular packet can carry multiple frames there are also other packets that do not carry any frames/stream-based information but are related to the state management of the whole connection (version negotiation and public reset in this case). I think this is all fine. My confusion comes when I look at the different kinds of frames. While most of the frames carry information that is clearly assigned to one stream, there are some frames that again carry connection-wide information, namely the ACK frame, the STOP_WAITING Frame, the CONNECTION_CLOSE frame, the GOWAY frame and probably also the PING frame (not talking about the PADDING frame here because its use is fully unclear to me…). I think these information should be part of an encrypted QUIC per-packet header instead. Some more concrete comments on each frame:

- ACK frame: Here the decision was made that data is ack’ed on a per-packet basis (instead on per frame-basis). I believe this decision is fine (didn’t actually further think about it so far). However, as these information are clearly aligned to packets, I think they should be part of the QUIC packet header. Here I assume that it does not make sense to send multiple ACK frames in one packet because they could always be combined to one (to save space). In this case it would be sufficient to have one flag in the general QUIC packet header that indicates if these information are present or not.

- STOP_WAITING frame: At the first reading I didn’t really understand what this is needed for because I didn’t fully internalize that packets are not retransmitted (btw. that could be stated more clearly in the intro!). I understand that this is inevitable connection to how ack'ing is done and therefore I really don’t understand why this is a separate frame at all. For me that could simply be one field in the ACK frame basically always indicating the last ACK information received. 

One more general comment here: I’m also not convince that the ACK scheme chosen (and basically aligned to the TCP ACK scheme is the right choice here). Given that you don’t retransmit, wouldn't a NACK scheme make more sense…? Meaning that you indicate the highest (un-ordered) packet number seen as well as all packet numbers not seen since the packet number announced by the last STOP_WAITING value. Did you consider that previously?

- PING frame: Having a whole frame for this really, really seems like overhead, while you only need on bit in the packet header and no frames/steam payload data are send at all… Btw. the reaction to this signal is not specified: is the receiver supposed to send a ping back or a different packet or nothing at all?

- CONNECTION_CLOSE and GO_AWAY: These two information bits are clearly per-connection. I also don’t really see a need for a CONNECTION_CLOSE and a PUBLIC_RESET. I think there can be a way to combine these signals given we have a mechanism for integrity checking.

Mirja

---

From: Mike Bishop <[email-masked]>
Date: Mon, 5 Dec 2016 19:13:13 +0000

I opened issue #70 for the suggestion to move ACK into a fixed per-packet location.  Ian has already replied there noting that there isn't an (encrypted) packet header, so we'd need to create one to hold the ACK data and anything else we wanted to consolidate there.  STOP_WAITING is already on the chopping block in another issue (#66).

The PING frame handling actually is specified:  "The receiver of a PING frame simply needs to ACK the packet containing this frame."  I can see where it seems like it serves little purpose -- there's basically no difference between PING and PADDING, except that PING is retransmittable and PADDING is not.

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Mirja Kühlewind
Sent: Monday, December 5, 2016 9:03 AM
To: [email-masked]
Subject: Stream vs. connection

Hi again,

Mirja

---

From: Ian Swett <[email-masked]>
Date: Mon, 5 Dec 2016 17:02:29 -0500

On Mon, Dec 5, 2016 at 2:13 PM, Mike Bishop <[email-masked]>
wrote:

> I opened issue #70 for the suggestion to move ACK into a fixed per-packet
> location.  Ian has already replied there noting that there isn't an
> (encrypted) packet header, so we'd need to create one to hold the ACK data
> and anything else we wanted to consolidate there.  STOP_WAITING is already
> on the chopping block in another issue (#66).
>
> The PING frame handling actually is specified:  "The receiver of a PING
> frame simply needs to ACK the packet containing this frame."  I can see
> where it seems like it serves little purpose -- there's basically no
> difference between PING and PADDING, except that PING is retransmittable
> and PADDING is not.
>
> Because it must be acked, it serves as a liveness/connectivity check,
which can be very useful on longer lived connections.

> here comes further information on point one regarding the questions how to
> treat stream-based information vs. connection-based information. This is
> currently my main point about the transport design of quic. To be more
> concrete, quic has the concept of packets and frames. While one regular
> packet can carry multiple frames there are also other packets that do not
> carry any frames/stream-based information but are related to the state
> management of the whole connection (version negotiation and public reset in
> this case). I think this is all fine. My confusion comes when I look at the
> different kinds of frames. While most of the frames carry information that
> is clearly assigned to one stream, there are some frames that again carry
> connection-wide information, namely the ACK frame, the STOP_WAITING Frame,
> the CONNECTION_CLOSE frame, the GOWAY frame and probably also the PING
> frame (not talking about the PADDING frame here because its use is fully
> unclear to me…). I think these information should be part of an encrypted
> QUIC per-packet header instead. Some more concrete comments on each frame:
>
>

> - ACK frame: Here the decision was made that data is ack’ed on a
> per-packet basis (instead on per frame-basis). I believe this decision is
> fine (didn’t actually further think about it so far). However, as these
> information are clearly aligned to packets, I think they should be part of
> the QUIC packet header. Here I assume that it does not make sense to send
> multiple ACK frames in one packet because they could always be combined to
> one (to save space). In this case it would be sufficient to have one flag
> in the general QUIC packet header that indicates if these information are
> present or not.
>

As noted in the Issue Mike opened, there is currently no encrypted QUIC
packet header, but it could be added.

>
> - STOP_WAITING frame: At the first reading I didn’t really understand what
> this is needed for because I didn’t fully internalize that packets are not
> retransmitted (btw. that could be stated more clearly in the intro!). I
> understand that this is inevitable connection to how ack'ing is done and
> therefore I really don’t understand why this is a separate frame at all.
> For me that could simply be one field in the ACK frame basically always
> indicating the last ACK information received.
>

As noted in Issue #66(https://github.com/quicwg/base-drafts/issues/66), it
is not really necessary anymore.  If not, merging into the ack frame as you
mentioned is a nice simplification, but I'd rather try removing it first if
possible.

> One more general comment here: I’m also not convince that the ACK scheme
> chosen (and basically aligned to the TCP ACK scheme is the right choice
> here). Given that you don’t retransmit, wouldn't a NACK scheme make more
> sense…? Meaning that you indicate the highest (un-ordered) packet number
> seen as well as all packet numbers not seen since the packet number
> announced by the last STOP_WAITING value. Did you consider that previously?
>

QUIC@Google originally implemented that.  In many ways they're mostly
equivalent, but it's critical to have the ack contain the smallest packet
being acked, otherwise the receiver can't unilaterally stop acking old
packets without receiving a stop waiting that raises the least unacked.
Once you have a largest acked and a set of nack ranges and a smallest
acked, you have something that looks more like ack ranges than nack ranges,
but one could word it as nack ranges.

The new ack block approach is much simpler to implement, and removed quite
a few edge cases.  I wrote a longer doc on why this change was made, but
I'm unable to find it at the moment.  I'll try to find it as background
information.

>
> - PING frame: Having a whole frame for this really, really seems like
> overhead, while you only need on bit in the packet header and no
> frames/steam payload data are send at all… Btw. the reaction to this signal
> is not specified: is the receiver supposed to send a ping back or a
> different packet or nothing at all?
>

The overhead is pretty small, since it's just a byte.  The reaction
specified in 6.8 is: "The receiver of a PING frame simply needs to ACK the
   packet containing this frame."  Would it be clearer as "only" instead of
simply?

> - CONNECTION_CLOSE and GO_AWAY: These two information bits are clearly
> per-connection. I also don’t really see a need for a CONNECTION_CLOSE and a
> PUBLIC_RESET. I think there can be a way to combine these signals given we
> have a mechanism for integrity checking.
>

A connection close is encrypted and a public reset is not.  If we have
non-spoofable public resets and we end up wanting to notify the path of the
close, I can imagine merging them into one packet, though I would want to
continue to encrypt connection close reason and details.  In which case,
we'd still need both.

Mirja

---

From: Jim Roskind <[email-masked]>
Date: Sat, 10 Dec 2016 14:31:40 -0800

Adding to Ian's comment:

On Mon, Dec 5, 2016 at 2:02 PM, Ian Swett <[email-masked]> wrote:

>> ...
Even more important/common that a "liveness check," a PING frame in a
packet often serves to revitalize a connection.  It is often the case that
a NATing router can drop the NAT association, either due to an expiration
timer firing, or plausibly due to the NAT table exhaustion and eviction.
As a result, it is routinely the case that a client sends a PING frame (in
a packet) when the client is still waiting for a response, and it is
fearful that a router has dropped the NAT association (which would make it
impossible for a server response to reach the client!).  The PING then
either assures the NAT router that "traffic is still using the route" (and
hence the NAT association should not be dropped anytime soon), or else it
causes the NAT router to establish a new association (potentially new
egress port for the externally visible router IP address).

On the receiving side, the server is therefore effectively supplied with an
explanation for why a packet was sent, containing a PING frame.  For some
servers, there is a fear that a DOS attack (intentional or accidental)
could be mounted by a series of PING-containing packets (which consume
bandwidth, but provide no significant data-payload transmission). As a
result, servers may wish to restrict the rate, or conditions, for which
clients may reasonably send PING frames, and at the same time, be "more
willing' to accept a change in the connection's client-side egress port
when a PING frame is observed.

PADDING frames serve different functions, and should not interact with
limitations/meanings on PING frames.  The cost of distinguishing these
similar frames seems small, and the value of handling PINGs differently
seems significant.

Hope that helps,

Jim

Opinions expressed are mine, and not that of my employer.