Subject: Re: Unidirectional streams PR

From: Kazuho Oku <[email-masked]>
Date: Wed, 5 Jul 2017 14:41:26 +0900

2017-06-29 10:23 GMT+09:00 Ian Swett <[email-masked]>:
> I updated my PR(#656) today, sorry for the delay.  I attempted to address
> the issues identified with text.  Some may prefer more tweaks to the state
> diagram, which are also possible, so I'm open to suggestions.

After all, I think that the approach proposed here is the most
well-balanced one.

As much as I think that having first-class support for unidirectional
streams is preferable, I also think that we should keep the overall
architecture (i.e. sum of the complexity in the transport layer and
the application layer) as simple as possible.

IMO having support for bidirectional streams (along with support for
unidirectional streams) aligns with such intention.

Most of the application protocols that will be deployed over the QUIC
transport will be in request-response style, at least to some extent.
Hence it is preferable for the transport layer to provide a framework
that fits such style.

Bidirectional streams provide the necessary features. FIN provides a
way to indicate the end of the message. RST provides a way to cancel
the exchange of the message in both directions.

IMO the biggest issue with the unidirectional-only approach is that
you cannot have the two features together. If a client needs a way to
reset the server's response before observing the first frame, it
cannot use FIN as an indicator of the end of the request.

It is true that the issue can be evaded by doing one's own framing in
the application layer. #643 does that by introducing a CANCEL_REQUEST
frame.

But I do not think that we should require every application protocol
built above QUIC to do its own framing. In addition to that, need to
keep two uni-directional streams open would consume more resources
than being able to close one direction of a bi-directional stream at
an earlier moment.

> In the meantime, I've been considering other alternatives, including
> variations on Mike's direction and a variation of the "Do Nothing" option
> which involved the application signaling to the transport that streams of a
> certain sort(ie: server to client for server push) were unidirectional, as
> GQUIC does today.  Overall, I think the approach I've outlined does a good
> job of iterating on existing deployment experience and adding explicit
> signaling for unidirectional streams instead of implicit signaling at the
> application layer.  There are pros and cons to both explicit and implicit
> signaling, but I think explicit signaling is more complete and less prone to
> application error.
>
> Thanks, Ian
>
>
> On Wed, Jun 28, 2017 at 8:14 PM, Lubashev, Igor <[email-masked]> wrote:
>>
>> > unless what the transport provides is a perfect fit for application
>> > semantics, you end up building those semantics into the application anyway.
>>
>> I agree with this. We should avoid adding complexity into transport for
>> rare use cases, since it goes against KISS principle.
>>
>> On the other hand, adding support for a by far the most common use case
>> makes a lot of sense.  This helps apps avoid screwing up implementing that
>> common case and lets us optimize that common case in the lower layer. BiDi
>> streams are such common cases. Uni streams are likely to be the
>> second-most-common cases (hence you offered this PR to optimize them).
>>
>>
>> The Associated Streams proposal offers extra semantic flexibility at a
>> cost of some semantic complexity (someone would need to verify that the
>> associated stream numbers make sense -- api? apps?) and a few extra bytes.
>>
>> I'd like to wait to see Ian's revised proposal.  The initial proposal
>> offered to do only one thing -- offer a choice of uni/bi-directional streams
>> -- but it did it in a very simple way, which is nice.
>>
>> - Igor
>>
>>
>> -----Original Message-----
>> From: Martin Thomson [mailto:[email-masked]]
>> Sent: Wednesday, June 28, 2017 7:28 PM
>> To: Mike Bishop <[email-masked]>
>> Cc: Swindells, Thomas (Nokia - GB/Cambridge, UK)
>> <[email-masked]>; QUIC WG <[email-masked]>; Mikkel Fahnøe
>> Jørgensen <[email-masked]>; Dmitri Tikhonov
>> <[email-masked]>; Jo Kulik <[email-masked]>
>> Subject: Re: Unidirectional streams PR
>>
>> There is probably a simpler approach here, take a bit (as Ian did) and say
>> that if that bit is set, then the stream is in response to another and the
>> stream ID of the stream to which this is responding follows immediately
>> after the stream ID of the stream itself.  You could then include that only
>> at the start of the stream, or in multiple frames (or as we decide).
>>
>> The problem with this, as with several of the other issues we're
>> discussing, is that unless what the transport provides is a perfect fit for
>> application semantics, you end up building those semantics into the
>> application anyway.  HTTP certainly can't survive without its own
>> association semantics for pushes.  That suggests to me that having
>> bidirectional semantics in the transport creates more duplication than
>> otherwise.  Hence my proposal.
>>
>> On 28 June 2017 at 15:26, Mike Bishop <[email-masked]>
>> wrote:
>> > As promised, a PR for adding “associated streams” is at
>> > https://github.com/quicwg/base-drafts/pull/672.  This very
>> > deliberately builds on top of MT’s PR – it’s adding a primitive which
>> > can be used to construct various abstractions atop unidirectional
>> > streams, but the lifecycle is still fundamentally unidirectional.
>> >
>> >
>> >
>> > Copying my notes here for list discussion purposes.
>> >
>> > Major changes
>> >
>> > Leveraging @igorlord's insight that OO=00 only occurs on the first
>> > STREAM frame of a stream, I used that as the trigger for a Stream
>> > Properties byte.
>> > Two bits of that byte describe the directionality of the stream:
>> >
>> > Unidirectional (no response expected)
>> > Initial bidirectional (one response expected) Initial multi-response
>> > (one or more responses expected; needs a better name) Response
>> >
>> > If the type is Response, there's an Associated Stream ID field, length
>> > given by two more bits following the same pattern as the SS bits in
>> > the STREAM frame ID.
>> >
>> > Personal Opinion
>> >
>> > On the plus side, these stream types seem to cover the abstractions I
>> > can envision for most applications. You can unilaterally send
>> > something (unidirectional), do request/response (bidirectional), or
>> > pub/sub (single subscription stream, series of update streams).
>> >
>> > I don't care for the fact that I still need the stream type header in
>> > HTTP after putting this in the transport. That will be ameliorated if
>> > we go back to one stream per request, since all unidirectional streams
>> > will be push streams. (As a side-note, I considered using the
>> > multiple-response option in the HTTP mapping, but then I need a stream
>> > header again to indicate which is the response and which the pushes.)
>> >
>> > I particularly don't like that you now have to look at the frame type
>> > header to find out whether a field exists which tells you the length
>> > of something else in the header. I'd like to simplify that. I went
>> > with this model over a CREATE_STREAM frame because of @mikkelfj's
>> > use-case of very small messages
>> > -- this adds only one byte to the first frame on a stream in one
>> > direction and 2-5 bytes to the first frame of response streams. A
>> > separate frame type would be somewhat larger, but could be cleaner in
>> > that respect.
>> >
>> >
>> >
>> >
>> >
>> > From: QUIC [mailto:[email-masked]] On Behalf Of Swindells,
>> > Thomas (Nokia - GB/Cambridge, UK)
>> > Sent: Wednesday, June 28, 2017 7:56 AM
>> > To: Jo Kulik <[email-masked]>; Mikkel Fahnøe Jørgensen
>> > <[email-masked]>
>> > Cc: QUIC WG <[email-masked]>; Dmitri Tikhonov
>> > <[email-masked]>
>> > Subject: RE: Unidirectional streams PR
>> >
>> >
>> >
>> > I agree that looking at the layers of abstraction is useful. In
>> > principle having the wire protocol just have constructs for
>> > unidirectional streams does not in itself limit creating
>> > bi-directional communication flows, supported at either the library or
>> > application layer.
>> >
>> >
>> >
>> > However, there need to be a standard way of doing bi-directional
>> > communication for migrating applications implemented using a socket
>> > style api. It needs to be easy to move an existing application from TCP
>> > to QUIC.
>> > This move may be attractive in many situations as QUIC gives improved
>> > security and may allow greater throughput due to the more modern (and
>> > customizable) congestion control algorithms compared to the OS TCP
>> > stack.
>> >
>> >
>> >
>> > For migrating standard socket api applications I don’t think it would
>> > be appropriate to leave the work to the application to do correlation,
>> > at least the library should be providing this service using the wire
>> > protocol as appropriate. Clearly we want a client written with one
>> > library to be able to communicate successfully with a server written
>> > using a different library.
>> > This needs some form of standardization of the signalling. This could
>> > either be a building block overlay on top of QUIC, or implemented at
>> > the wire protocol level.
>> >
>> >
>> >
>> > In terms of patterns I think the following may be some of the most
>> > common patterns (with potential to be provided at the library and or
>> > wire protocol level).
>> >
>> > I/O pattern  : Example
>> >
>> > 1/0   : An input only flow, perhaps a data logger like syslog with no
>> > confirmation/feedback
>> >
>> > 0/1  : an output only flow, perhaps a topic message bus service with
>> > no confirmation/feedback
>> >
>> > 1/1 : standard TCP applications with a single flow per connection
>> >
>> > 1/* : single input, many output, modelling STDIN/STDOUT+STDERR
>> >
>> > (1/1)* : multiplexed pairs of flows – supporting multiple sockets
>> > muxed onto a single QUIC connection
>> >
>> >
>> >
>> > Obviously, an application would always have the option to combine any
>> > single direction flows with application level correlators to construct
>> > more complex flows if desired.
>> >
>> >
>> >
>> > At the moment my gut says the 1/1 use-case is common enough that the
>> > wire protocol should provide a standard mechanism to support it as a
>> > standard overlay would probably end up being treated as part of the
>> > wire format anyway.
>> >
>> >
>> >
>> > Perhaps streams should be explicitly created with a CREATE_STREAM
>> > frame which would be capable of defining multiple related streams?
>> >
>> > There is the option of whether only (1/1) pairs can be created this
>> > way, or
>> > (1/n) combinations could be supported (with an application defined way
>> > to identify the use of each of the output streams). A step further may
>> > be that there is a transport parameter that defines whether the server
>> > is allowed to create additional streams, or if stream creation is
>> > purely client driven (like TCP). I don’t know if either of these would
>> > simplify how to handle stream accounting, and in particular only
>> > creating a flow when all parties have sufficient allowances left.
>> >
>> >
>> >
>> > Thomas
>> >
>> >
>> >
>> > From: QUIC [mailto:[email-masked]] On Behalf Of Jo Kulik
>> > Sent: 28 June 2017 15:09
>> > To: Mikkel Fahnøe Jørgensen <[email-masked]>
>> > Cc: QUIC WG <[email-masked]>; Dmitri Tikhonov
>> > <[email-masked]>
>> > Subject: Re: Unidirectional streams PR
>> >
>> >
>> >
>> > I'd like to pop back up to a comment Igor made last week, because I
>> > find it helpful in thinking about the design space:
>> >
>> >
>> >
>> > I think of three layers of abstraction:
>> >
>> > 1.       QUIC Wire Protocol (the thing described by the QUIC Transport
>> > RFC)
>> > 2.       QUIC Library API (a library exposing some useful abstractions
>> > --
>> > such as blocking/non-blocking unidirectional streams and bidirectional
>> > “sockets” -- and implementing them using QUIC Wire Protocol)
>> > 3.       Application (something that uses QUIC Library APIs)
>> >
>> > I think there is some argument to be made that Martin's original
>> > proposal did not take into account how we would achieve (2) for
>> > bi-directional streams.  (I don't think it strictly said "thou shalt
>> > not do (2)" either, but that is up to interpretation.)
>> >
>> >
>> >
>> > Several people have argued that we do not want every application to
>> > have to re-implement bi-directional streams (3) for every application,
>> > and this is not how g-quic (our largest deployment) works right now.
>> > These arguments make sense to me, but YMMV.
>> >
>> >
>> >
>> > Just because the particular *mechanism* that is being proposed has
>> > some issues, however, doesn't scream out to me, at least, that we
>> > should abandon this particular *design goal*.  The goal being a
>> > transport protocol that can elegantly fit with a uni/bi stream model.
>> > Now, if we conclude that there can never be an elegant model that
>> > achieves this goal, then so be it.  But I also feel like we haven't
>> > reached that point in the discussion yet.  (At the very least, this
>> > discussion has been fruitful to me in terms of mapping the design space
>> > and elucidating requirements).
>> >
>> >
>> >
>> > One of the reasons I still think this design goal is under
>> > consideration is that Ian and Igor/Mike have been talking about
>> > alternate solutions which have a similar flavor.  During the recent
>> > "quiet"ness on the thread, personally, I've been waiting to hear more
>> > from them.
>> >
>> >
>> >
>> > On Wed, Jun 28, 2017 at 9:41 AM, Mikkel Fahnøe Jørgensen
>> > <[email-masked]> wrote:
>> >
>> > In reply to Ranjeeth
>> >
>> >
>> >
>> > It is not only a matter of simplicity for the sake of simplicity:
>> >
>> >
>> >
>> > - A complex transport layer might end up being poorly implemented
>> > leading to reduced interoperability and ultimately adoption. This
>> > complexity is not only in implementation but also in understanding the
>> > exact semantics of stream lifetime. Even if the spec is sufficiently
>> > clear, it will still be open to misinterpretations.
>> >
>> >
>> >
>> > - Bi-directional state may have to be maintained longer and with more
>> > overhead than with uni-directional streams, especially under loss,
>> > potentially leading to poor performance and poor resource utilisation
>> > because the transport layer has insufficient information.
>> >
>> >
>> >
>> > - The extra complexity at the application layer may be overstated - it
>> > is significantly simpler to manage a map that associates to two
>> > streams than it is to maintain bi-directional state at the transport
>> > layer. It is even possible to implicitly link streams with same
>> > identifiers, e.g. in a RPC scenario. That said, I do see a potential
>> > benefit of a wrapper that implements the common bi-directional case.
>> >
>> >
>> >
>> > - Complexity at the application layer may be duplicated, but
>> > implementation errors are also isolated to that application.
>> > Specifically for HTTP I would assume that QUIC transport and QUIC HTTP
>> > implementers would be large the same for a long time to come, so I
>> > would not expect the tradeoff here to be particularly concerning.
>> >
>> >
>> >
>> > - Unix pipes are traditionally constructed as a pair of
>> > uni-directional file descriptors and that is a reasonably proven
>> > model. C’s standard library stdin, stdout and stderr is an example of
>> > an asymmetric model with implicit linkage between uni-directional file
>> > descriptors.
>> >
>> >
>> >
>> > - There are lots of use cases for non-HTTP like connectivity - Kafka
>> > high volume message queuing for example. The industry trend appears to
>> > move towards asynchronous processing and messaging. It depends on
>> > whether you look at QUIC as a TCP + TLS replacement, or as a HTTPS /
>> > REST RPC replacement.
>> >
>> >
>> >
>> > - Uni-directional streams may currently be unproven in the wild, but a
>> > proposal is needed before an implementation can be made and testet. I
>> > agree that it is easy to design into wrong assumptions without real
>> > world testing.
>> >
>> >
>> >
>> > - There will hopefully not be a large number of successors to QUIC -
>> > perhaps some purpose specific variants, e.g. for embedded use.
>> > Widespread adaptation and compatibility is very necessary so it makes
>> > sense to have QUIC being sufficiently simple and expressive to achieve
>> > this goal. A polymorf QUIC will not achieve that goal. On the other
>> > hand, a solid QUIC foundation can be used for a large number of
>> > application protocols.
>> >
>> >
>> >
>> > - Finally, it may turn out that uni-directional streams just is a bad
>> > idea - I doubt it, but I do believe real world tests are needed.
>> >
>> >
>> >
>> > Kind Regards,
>> >
>> > Mikkel Fahnøe Jørgensen
>> >
>> >
>> >
>> > On 28 June 2017 at 14.42.36, Dmitri Tikhonov
>> > ([email-masked])
>> > wrote:
>> >
>> > On Tue, Jun 27, 2017 at 02:31:38PM -0700, Ranjeeth Kumar Dasineni wrote:
>> >> 2. We are overplaying the simplicity of design. Even if we deem
>> >> deployment experience not a concern, if every application layer
>> >> protocol that needs support for bidirectional streams has to
>> >> implement some correlators and such above, that's a net negative in
>> >> terms of complexity.
>> >
>> > This is an important point: we want QUIC adoption to be made easy.
>> > A program that speaks HTTP today should be able to use an existing
>> > QUIC library without having to emulate bidirectional streams in order
>> > to fit it into HTTP usage pattern. Forcing every one of these programs
>> > to do this is certainly a hurdle.
>> >
>> > - Dmitri.
>> >
>> >
>>
>

-- 
Kazuho Oku

---

From: "Lubashev, Igor" <[email-masked]>
Date: Thu, 6 Jul 2017 03:50:05 +0000

> If a client needs a way to reset the server's response before observing the first frame, it cannot use FIN as an indicator of the end of the request.

There is a DISINTEREST frame proposal (PR #171: https://github.com/quicwg/base-drafts/pull/171).  That's something that I believe is valuable independent of this Unidirectional streams PR.

- Igor

-----Original Message-----
From: Kazuho Oku [mailto:[email-masked]] 
Sent: Wednesday, July 05, 2017 1:41 AM
To: Ian Swett <[email-masked]>
Cc: Lubashev, Igor <[email-masked]>; Mike Bishop <[email-masked]>; Dmitri Tikhonov <[email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <[email-masked]>; Jo Kulik <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>
Subject: Re: Unidirectional streams PR

2017-06-29 10:23 GMT+09:00 Ian Swett <[email-masked]>:
> I updated my PR(#656) today, sorry for the delay.  I attempted to 
> address the issues identified with text.  Some may prefer more tweaks 
> to the state diagram, which are also possible, so I'm open to suggestions.

After all, I think that the approach proposed here is the most well-balanced one.

As much as I think that having first-class support for unidirectional streams is preferable, I also think that we should keep the overall architecture (i.e. sum of the complexity in the transport layer and the application layer) as simple as possible.

IMO having support for bidirectional streams (along with support for unidirectional streams) aligns with such intention.

Most of the application protocols that will be deployed over the QUIC transport will be in request-response style, at least to some extent.
Hence it is preferable for the transport layer to provide a framework that fits such style.

Bidirectional streams provide the necessary features. FIN provides a way to indicate the end of the message. RST provides a way to cancel the exchange of the message in both directions.

IMO the biggest issue with the unidirectional-only approach is that you cannot have the two features together. If a client needs a way to reset the server's response before observing the first frame, it cannot use FIN as an indicator of the end of the request.

It is true that the issue can be evaded by doing one's own framing in the application layer. #643 does that by introducing a CANCEL_REQUEST frame.

But I do not think that we should require every application protocol built above QUIC to do its own framing. In addition to that, need to keep two uni-directional streams open would consume more resources than being able to close one direction of a bi-directional stream at an earlier moment.

> variations on Mike's direction and a variation of the "Do Nothing" 
> option which involved the application signaling to the transport that 
> streams of a certain sort(ie: server to client for server push) were 
> unidirectional, as GQUIC does today.  Overall, I think the approach 
> I've outlined does a good job of iterating on existing deployment 
> experience and adding explicit signaling for unidirectional streams 
> instead of implicit signaling at the application layer.  There are 
> pros and cons to both explicit and implicit signaling, but I think 
> explicit signaling is more complete and less prone to application error.
>
> Thanks, Ian
>
>
>> I agree with this. We should avoid adding complexity into transport 
>> for rare use cases, since it goes against KISS principle.
>>
>> On the other hand, adding support for a by far the most common use 
>> case makes a lot of sense.  This helps apps avoid screwing up 
>> implementing that common case and lets us optimize that common case 
>> in the lower layer. BiDi streams are such common cases. Uni streams 
>> are likely to be the second-most-common cases (hence you offered this PR to optimize them).
>>
>>
>> The Associated Streams proposal offers extra semantic flexibility at 
>> a cost of some semantic complexity (someone would need to verify that 
>> the associated stream numbers make sense -- api? apps?) and a few extra bytes.
>>
>> There is probably a simpler approach here, take a bit (as Ian did) 
>> and say that if that bit is set, then the stream is in response to 
>> another and the stream ID of the stream to which this is responding 
>> follows immediately after the stream ID of the stream itself.  You 
>> could then include that only at the start of the stream, or in multiple frames (or as we decide).
>>
>> discussing, is that unless what the transport provides is a perfect 
>> fit for application semantics, you end up building those semantics 
>> into the application anyway.  HTTP certainly can't survive without 
>> its own association semantics for pushes.  That suggests to me that 
>> having bidirectional semantics in the transport creates more 
>> duplication than otherwise.  Hence my proposal.
>>
>> > deliberately builds on top of MT’s PR – it’s adding a primitive 
>> > which can be used to construct various abstractions atop 
>> > unidirectional streams, but the lifecycle is still fundamentally unidirectional.
>> >
>> >
>> >
>> > Unidirectional (no response expected) Initial bidirectional (one 
>> > response expected) Initial multi-response (one or more responses 
>> > expected; needs a better name) Response
>> >
>> > If the type is Response, there's an Associated Stream ID field, 
>> > length given by two more bits following the same pattern as the SS 
>> > bits in the STREAM frame ID.
>> >
>> > Personal Opinion
>> >
>> > On the plus side, these stream types seem to cover the abstractions 
>> > I can envision for most applications. You can unilaterally send 
>> > I don't care for the fact that I still need the stream type header 
>> > in HTTP after putting this in the transport. That will be 
>> > ameliorated if we go back to one stream per request, since all 
>> > unidirectional streams will be push streams. (As a side-note, I 
>> > considered using the multiple-response option in the HTTP mapping, 
>> > but then I need a stream header again to indicate which is the 
>> > response and which the pushes.)
>> >
>> > I particularly don't like that you now have to look at the frame 
>> > type header to find out whether a field exists which tells you the 
>> > length of something else in the header. I'd like to simplify that. 
>> > I went with this model over a CREATE_STREAM frame because of 
>> > @mikkelfj's use-case of very small messages
>> > separate frame type would be somewhat larger, but could be cleaner 
>> > in that respect.
>> >
>> >
>> >
>> >
>> >
>> > bi-directional communication flows, supported at either the library 
>> > or application layer.
>> >
>> >
>> >
>> > style api. It needs to be easy to move an existing application from 
>> > TCP to QUIC.
>> > This move may be attractive in many situations as QUIC gives 
>> > improved security and may allow greater throughput due to the more 
>> > modern (and
>> > For migrating standard socket api applications I don’t think it 
>> > would be appropriate to leave the work to the application to do 
>> > correlation, at least the library should be providing this service 
>> > using the wire protocol as appropriate. Clearly we want a client 
>> > written with one library to be able to communicate successfully 
>> > with a server written using a different library.
>> > This needs some form of standardization of the signalling. This 
>> > could either be a building block overlay on top of QUIC, or 
>> > implemented at the wire protocol level.
>> >
>> >
>> >
>> > common patterns (with potential to be provided at the library and 
>> > or wire protocol level).
>> >
>> > I/O pattern  : Example
>> >
>> > 0/1  : an output only flow, perhaps a topic message bus service 
>> > with no confirmation/feedback
>> >
>> > Obviously, an application would always have the option to combine 
>> > any single direction flows with application level correlators to 
>> > construct more complex flows if desired.
>> >
>> >
>> >
>> > At the moment my gut says the 1/1 use-case is common enough that 
>> > the wire protocol should provide a standard mechanism to support it 
>> > as a standard overlay would probably end up being treated as part 
>> > of the wire format anyway.
>> >
>> >
>> >
>> > (1/n) combinations could be supported (with an application defined 
>> > way to identify the use of each of the output streams). A step 
>> > further may be that there is a transport parameter that defines 
>> > whether the server is allowed to create additional streams, or if 
>> > stream creation is purely client driven (like TCP). I don’t know if 
>> > either of these would simplify how to handle stream accounting, and 
>> > in particular only creating a flow when all parties have sufficient allowances left.
>> >
>> >
>> >
>> > Thomas
>> >
>> >
>> >
>> > such as blocking/non-blocking unidirectional streams and 
>> > bidirectional “sockets” -- and implementing them using QUIC Wire Protocol)
>> > bi-directional streams.  (I don't think it strictly said "thou 
>> > shalt not do (2)" either, but that is up to interpretation.)
>> >
>> >
>> >
>> > have to re-implement bi-directional streams (3) for every 
>> > application, and this is not how g-quic (our largest deployment) works right now.
>> > discussion has been fruitful to me in terms of mapping the design 
>> > space and elucidating requirements).
>> >
>> >
>> >
>> > "quiet"ness on the thread, personally, I've been waiting to hear 
>> > more from them.
>> >
>> >
>> >
>> > complexity is not only in implementation but also in understanding 
>> > the exact semantics of stream lifetime. Even if the spec is 
>> > sufficiently clear, it will still be open to misinterpretations.
>> >
>> >
>> >
>> > - Bi-directional state may have to be maintained longer and with 
>> > more overhead than with uni-directional streams, especially under 
>> > loss, potentially leading to poor performance and poor resource 
>> > utilisation because the transport layer has insufficient information.
>> >
>> >
>> >
>> > - The extra complexity at the application layer may be overstated - 
>> > it is significantly simpler to manage a map that associates to two 
>> > streams than it is to maintain bi-directional state at the 
>> > transport layer. It is even possible to implicitly link streams 
>> > with same identifiers, e.g. in a RPC scenario. That said, I do see 
>> > a potential benefit of a wrapper that implements the common bi-directional case.
>> >
>> >
>> >
>> > Specifically for HTTP I would assume that QUIC transport and QUIC 
>> > HTTP implementers would be large the same for a long time to come, 
>> > so I would not expect the tradeoff here to be particularly concerning.
>> >
>> >
>> >
>> > model. C’s standard library stdin, stdout and stderr is an example 
>> > of an asymmetric model with implicit linkage between 
>> > uni-directional file descriptors.
>> >
>> >
>> >
>> > - There are lots of use cases for non-HTTP like connectivity - 
>> > Kafka high volume message queuing for example. The industry trend 
>> > appears to move towards asynchronous processing and messaging. It 
>> > depends on whether you look at QUIC as a TCP + TLS replacement, or 
>> > as a HTTPS / REST RPC replacement.
>> >
>> >
>> >
>> > - Uni-directional streams may currently be unproven in the wild, 
>> > but a proposal is needed before an implementation can be made and 
>> > testet. I agree that it is easy to design into wrong assumptions 
>> > without real world testing.
>> >
>> >
>> >
>> > - There will hopefully not be a large number of successors to QUIC 
>> > - perhaps some purpose specific variants, e.g. for embedded use.
>> > Widespread adaptation and compatibility is very necessary so it 
>> > makes sense to have QUIC being sufficiently simple and expressive 
>> > to achieve this goal. A polymorf QUIC will not achieve that goal. 
>> > On the other hand, a solid QUIC foundation can be used for a large 
>> > number of application protocols.
>> >
>> >
>> >
>> > - Finally, it may turn out that uni-directional streams just is a 
>> > bad idea - I doubt it, but I do believe real world tests are needed.
>> >
>> >
>> >
>> > Kind Regards,
>> >
>> > Mikkel Fahnøe Jørgensen
>> >
>> >
>> >
>> >> implement some correlators and such above, that's a net negative 
>> >> in terms of complexity.
>> >
>> > QUIC library without having to emulate bidirectional streams in 
>> > order to fit it into HTTP usage pattern. Forcing every one of these 
>> > programs to do this is certainly a hurdle.
>> >
>> > - Dmitri.
>> >
>> >
>>
>

--
Kazuho Oku

---

From: Subodh Iyengar <[email-masked]>
Date: Thu, 6 Jul 2017 05:16:56 +0000

> Without such a signal, a generic proxy that is not aware of your application internals is not able to know when to send the FIN

I see, this use case does make somewhat sense, however I'm curious if someone has a concrete use case for a generic QUIC proxy which is not aware of the application protocol at all. In TCP this was more common for middleboxes to do because there was no end to end encryption of the transport, however QUIC is encrypted. Even with cases when we tunneling protocols at our end, we usually have some knowledge of the app that we are running because we need to route them differently to different backends. I was mostly thinking that proxies would at least have some knowledge about HTTP/2 or app protocols.

Maybe it is possible to make #656 more palatable, at least to me. I had 2 things against it initially, which were:

  1.  We might need to add another state to the machine to the sender state, i.e. normally when sending a FIN, it is ok to keep receiving stream frames after, however in a uni directional stream it is illegal to get a stream frame.
  2.  In its current form it also has an implicit requirement to not open lower streams. I think opening lower streams have some desirable properties which I've commented on in https://github.com/quicwg/base-drafts/issues/662.

An implementation could probably implement 1. with not too much trouble by doing a special case in the stream state machine when it receives a stream frame in the half closed local state.

For 2. thinking about it a bit more I think the underlying question here is who enforces directionality, i.e. did I receive data I should not have. If it is the transport, then we need to answer that question with issue #662, however if it can be the application or a higher layer API between the application and transport that enforces directionality then we don't need to know whether a lower stream is bi-directional or not. I'm leaning towards an API between the transport and application enforcing directionality. The reason for that is that an application that needs to use unidirectionality needs to use a different API anyway. Given that we could leave lower streams to be opened as it is now, but leaving the exact semantics of enforcing bi-directionality or uni-directionality to a higher layer. That should suffice for even a generic proxy right?

I still prefer do nothing with app directed close though.

> But I do not think that we should require every application protocol built above QUIC to do its own framing.

Second Kazuho on this one.

Subodh

________________________________
From: QUIC <[email-masked]> on behalf of Lubashev, Igor <[email-masked]>
Sent: Wednesday, July 5, 2017 8:50:05 PM
To: Kazuho Oku; Ian Swett
Cc: Mike Bishop; Dmitri Tikhonov; Swindells, Thomas (Nokia - GB/Cambridge, UK); Jo Kulik; Mikkel Fahnøe Jørgensen; QUIC WG; Martin Thomson
Subject: RE: Unidirectional streams PR

- Igor

-----Original Message-----
From: Kazuho Oku [mailto:[email-masked]]
Sent: Wednesday, July 05, 2017 1:41 AM
To: Ian Swett <[email-masked]>
Cc: Lubashev, Igor <[email-masked]>; Mike Bishop <[email-masked]>; Dmitri Tikhonov <[email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <[email-masked]>; Jo Kulik <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>
Subject: Re: Unidirectional streams PR

--
Kazuho Oku

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 6 Jul 2017 18:48:51 -0700

On Wed, Jul 5, 2017 at 10:16 PM, Subodh Iyengar <[email-masked]> wrote:

> > Without such a signal, a generic proxy that is not aware of your
> application internals is not able to know when to send the FIN
>
> I see, this use case does make somewhat sense, however I'm curious
> if someone has a concrete use case for a generic QUIC proxy which is not
> aware of the application protocol at all. In TCP this was more common for
> middleboxes to do because there was no end to end encryption of the
> transport, however QUIC is encrypted. Even with cases when we tunneling
> protocols at our end, we usually have some knowledge of the app that we are
> running because we need to route them differently to different backends. I
> was mostly thinking that proxies would at least have some knowledge about
> HTTP/2 or app protocols.
>

This is my thinking too. I'm not sure I understand what sort of proxy would
want to know about stream closures... but if such a proxy exists, it
justifies the explicit signal.

Maybe it is possible to make #656 more palatable, at least to me. I had 2
> things against it initially, which were:
>
>
>    1. We might need to add another state to the machine to the sender
>    state, i.e. normally when sending a FIN, it is ok to keep receiving stream
>    frames after, however in a uni directional stream it is illegal to get a
>    stream frame.
>    2. In its current form it also has an implicit requirement to not open
>    lower streams. I think opening lower streams have some desirable properties
>    which I've commented on in https://github.com/quicwg/
>    base-drafts/issues/662.
>
>
> An implementation could probably implement 1. with not too much trouble by
> doing a special case in the stream state machine when it receives a stream
> frame in the half closed local state.
>

I think this is covered by the current half-closed state. When an outgoing
uni stream is created, it should have a final received offset of 0. If any
STREAM frames are received that have data at an offset > the final received
offset (which is 0 in this case), then the connection is torn down with
QUIC_STREAM_DATA_AFTER_TERMINATION error (see Section 11.3
<https://tools.ietf.org/html/draft-ietf-quic-transport-04#section-11.3>).

> For 2. thinking about it a bit more I think the underlying question here
> is who enforces directionality, i.e. did I receive data I should not have.
> If it is the transport, then we need to answer that question with issue
> #662, however if it can be the application or a higher layer API between
> the application and transport that enforces directionality then we don't
> need to know whether a lower stream is bi-directional or not. I'm leaning
> towards an API between the transport and application enforcing
> directionality. The reason for that is that an application that needs to
> use unidirectionality needs to use a different API anyway. Given that we
> could leave lower streams to be opened as it is now, but leaving the exact
> semantics of enforcing bi-directionality or uni-directionality to a higher
> layer. That should suffice for even a generic proxy right?
>

I thought about this, and I'm leaning towards having the transport enforce
this.

I like your suggestion of moving this logic up to the application. Implicit
open is then just "open", and then the stream moves into half-closed state
as it receives an application signal or a STREAM frame with the bit set.
The tradeoff to consider here is that an implementation will not be able to
instantiate an "optimized" uni stream on implicit creation (or will have to
swap a bidirectional stream with a uni one). There's value in allowing
implementations to implement optimized uni/bi stream objects, which would
mean that the transport ought to be able to create one of these two types
of objects, therefore allowing (requiring?) enforcement of directionality
in the transport.

Can you resolve #662 with a separate state? Meaning that implicitly open
streams are now considered "reserved" (or some such). This allows you to
have a separate state until you know for sure which type of stream you're
constructing, which can happen via an application signal or from a received
STREAM frame.

Having an explicit bit in the frame does allow for some more flexibility,
which may be useful.

> built above QUIC to do its own framing.
>

+1. This is the biggest argument for bidirectionality -- most applications
need that, and expecting them to all create their own correlators and state
machine to manage bidirectionality goes against what I'd consider useful
transport API design. Extremely common design patterns, such as
bidirectionality, should be part of the transport.

- jana

Subodh
>
> ------------------------------
> *From:* QUIC <[email-masked]> on behalf of Lubashev, Igor <
> [email-masked]>
> *Sent:* Wednesday, July 5, 2017 8:50:05 PM
> *To:* Kazuho Oku; Ian Swett
> *Cc:* Mike Bishop; Dmitri Tikhonov; Swindells, Thomas (Nokia -
> GB/Cambridge, UK); Jo Kulik; Mikkel Fahnøe Jørgensen; QUIC WG; Martin
> Thomson
>
> *Subject:* RE: Unidirectional streams PR
>
> > If a client needs a way to reset the server's response before observing
> the first frame, it cannot use FIN as an indicator of the end of the
> request.
>
> There is a DISINTEREST frame proposal (PR #171: https://github.com/quicwg/
> base-drafts/pull/171).  That's something that I believe is valuable
> independent of this Unidirectional streams PR.
>
> - Igor
>
>
> From: Kazuho Oku [mailto:[email-masked] <[email-masked]>]
> Cc: Lubashev, Igor <[email-masked]>; Mike Bishop <
> [email-masked]>; Dmitri Tikhonov <
> [email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge,
> UK) <[email-masked]>; Jo Kulik <[email-masked]>; Mikkel
> Fahnøe Jørgensen <[email-masked]>; QUIC WG <[email-masked]>; Martin
> Thomson <[email-masked]>
> architecture (i.e. sum of the complexity in the transport layer and the
> application layer) as simple as possible.
>
> Bidirectional streams provide the necessary features. FIN provides a way
> to indicate the end of the message. RST provides a way to cancel the
> exchange of the message in both directions.
>
> IMO the biggest issue with the unidirectional-only approach is that you
> cannot have the two features together. If a client needs a way to reset the
> server's response before observing the first frame, it cannot use FIN as an
> indicator of the end of the request.
>
> It is true that the issue can be evaded by doing one's own framing in the
> application layer. #643 does that by introducing a CANCEL_REQUEST frame.
>
> But I do not think that we should require every application protocol built
> above QUIC to do its own framing. In addition to that, need to keep two
> uni-directional streams open would consume more resources than being able
> to close one direction of a bi-directional stream at an earlier moment.
>
> >> From: Martin Thomson [mailto:[email-masked]
> <[email-masked]>]
> >> > From: QUIC [mailto:[email-masked] <[email-masked]>] On
> Behalf Of Swindells,
> >> > From: QUIC [mailto:[email-masked] <[email-masked]>] On
> Behalf Of Jo Kulik
> >> > Sent: 28 June 2017 15:09

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 7 Jul 2017 03:07:23 +0000

> I'm not sure I understand what sort of proxy would want to know about stream closures...

Any proxy that keeps per-stream state would want to know about stream closures (just like the endpoint transport implementation would).

Say you have a fast and reliable network to which devices with very limited amount of memory are attached (sensors on a Mars station connected to the station network). But this network is connected to the Internet via a slow and unreliable link (satellite?). To communicate with far-away endpoints over slow and unreliable links, you need a lot of memory, or your throughput suffers. The solution could be a proxy on the border of this network. Such proxy would mediate QUIC connections between the limited-memory devices and Internet endpoints. The proxy would ACK all packets itself, while maintaining big enough buffers to retransmit stream data sent to the Internet and reassemble streams from the Internet. (One can think of similar situations closer to Earth, of course.)

-----Original Message-----
From: Jana Iyengar [[email-masked]]
Received: Thursday, 06 Jul 2017, 9:48PM
To: Subodh Iyengar [[email-masked]]
CC: Lubashev, Igor [[email-masked]]; Kazuho Oku [[email-masked]]; Ian Swett [[email-masked]]; Mike Bishop [[email-masked]]; Dmitri Tikhonov [[email-masked]]; Swindells, Thomas (Nokia - GB/Cambridge, UK) [[email-masked]]; Jo Kulik [[email-masked]]; Mikkel Fahnøe Jørgensen [[email-masked]]; QUIC WG [[email-masked]]; Martin Thomson [[email-masked]]
Subject: Re: Unidirectional streams PR

This is my thinking too. I'm not sure I understand what sort of proxy would want to know about stream closures... but if such a proxy exists, it justifies the explicit signal.

  1.  We might need to add another state to the machine to the sender state, i.e. normally when sending a FIN, it is ok to keep receiving stream frames after, however in a uni directional stream it is illegal to get a stream frame.
  2.  In its current form it also has an implicit requirement to not open lower streams. I think opening lower streams have some desirable properties which I've commented on in https://github.com/quicwg/base-drafts/issues/662<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_662&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=nUIKlIBqPqjVljTh1PhBhHP75fAW7-4WhWrKD26rfz4&s=SZ7WecZx9-JS9RjTS0c2nhWXRlgScSi1XqtFOSDn9W8&e=>.

I think this is covered by the current half-closed state. When an outgoing uni stream is created, it should have a final received offset of 0. If any STREAM frames are received that have data at an offset > the final received offset (which is 0 in this case), then the connection is torn down with QUIC_STREAM_DATA_AFTER_TERMINATION error (see Section 11.3<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dquic-2Dtransport-2D04-23section-2D11.3&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=nUIKlIBqPqjVljTh1PhBhHP75fAW7-4WhWrKD26rfz4&s=uCU3ExruisXB9KCiImOF2DuRt36XRdn2Tn6n3tqg2YM&e=>).

I thought about this, and I'm leaning towards having the transport enforce this.

I like your suggestion of moving this logic up to the application. Implicit open is then just "open", and then the stream moves into half-closed state as it receives an application signal or a STREAM frame with the bit set. The tradeoff to consider here is that an implementation will not be able to instantiate an "optimized" uni stream on implicit creation (or will have to swap a bidirectional stream with a uni one). There's value in allowing implementations to implement optimized uni/bi stream objects, which would mean that the transport ought to be able to create one of these two types of objects, therefore allowing (requiring?) enforcement of directionality in the transport.

Can you resolve #662 with a separate state? Meaning that implicitly open streams are now considered "reserved" (or some such). This allows you to have a separate state until you know for sure which type of stream you're constructing, which can happen via an application signal or from a received STREAM frame.

Having an explicit bit in the frame does allow for some more flexibility, which may be useful.

+1. This is the biggest argument for bidirectionality -- most applications need that, and expecting them to all create their own correlators and state machine to manage bidirectionality goes against what I'd consider useful transport API design. Extremely common design patterns, such as bidirectionality, should be part of the transport.

- jana

Subodh

From: QUIC <[email-masked]<mailto:[email-masked]>> on behalf of Lubashev, Igor <[email-masked]<mailto:[email-masked]>>

> If a client needs a way to reset the server's response before observing the first frame, it cannot use FIN as an indicator of the end of the request.

There is a DISINTEREST frame proposal (PR #171: https://github.com/quicwg/base-drafts/pull/171<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_pull_171&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=nUIKlIBqPqjVljTh1PhBhHP75fAW7-4WhWrKD26rfz4&s=vUU1jpQB1L_xAsDNdBWa6XhQ8_fzM4EF2uP5uq57dEo&e=>).  That's something that I believe is valuable independent of this Unidirectional streams PR.

- Igor

-----Original Message-----
From: Kazuho Oku [mailto:[email-masked]]
Sent: Wednesday, July 05, 2017 1:41 AM
To: Ian Swett <[email-masked]<mailto:[email-masked]>>
Cc: Lubashev, Igor <[email-masked]<mailto:[email-masked]>>; Mike Bishop <[email-masked]<mailto:[email-masked]>>; Dmitri Tikhonov <[email-masked]<mailto:[email-masked]>>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <[email-masked]<mailto:[email-masked]>>; Jo Kulik <[email-masked]<mailto:[email-masked]>>; Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>; QUIC WG <[email-masked]<mailto:[email-masked]>>; Martin Thomson <[email-masked]<mailto:[email-masked]>>
Subject: Re: Unidirectional streams PR

> On Wed, Jun 28, 2017 at 8:14 PM, Lubashev, Igor <[email-masked]<mailto:[email-masked]>> wrote:
>>
>> To: Mike Bishop <[email-masked]<mailto:[email-masked]>>
>> <[email-masked]<mailto:[email-masked]>>; QUIC WG <[email-masked]<mailto:[email-masked]>>; Mikkel Fahnøe
>> Jørgensen <[email-masked]<mailto:[email-masked]>>; Dmitri Tikhonov
>> <[email-masked]<mailto:[email-masked]>>; Jo Kulik <[email-masked]<mailto:[email-masked]>>
>> On 28 June 2017 at 15:26, Mike Bishop <[email-masked]<mailto:[email-masked]>>
>> wrote:
>> > https://github.com/quicwg/base-drafts/pull/672<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_pull_672&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=nUIKlIBqPqjVljTh1PhBhHP75fAW7-4WhWrKD26rfz4&s=BpuekjI8WR8_GGDuIgRp0wW2dylR52OtLbuxONJGujY&e=>.  This very
>> > To: Jo Kulik <[email-masked]<mailto:[email-masked]>>; Mikkel Fahnøe Jørgensen
>> > <[email-masked]<mailto:[email-masked]>>
>> > Cc: QUIC WG <[email-masked]<mailto:[email-masked]>>; Dmitri Tikhonov
>> > <[email-masked]<mailto:[email-masked]>>
>> > To: Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>
>> > Cc: QUIC WG <[email-masked]<mailto:[email-masked]>>; Dmitri Tikhonov
>> > <[email-masked]<mailto:[email-masked]>>
>> > <[email-masked]<mailto:[email-masked]>> wrote:
>> >
>> > In reply to Ranjeeth
>> >
>> >
>> >
>> > ([email-masked]<mailto:[email-masked]>)
>> > wrote:
>> >

--
Kazuho Oku

---

From: Martin Duke <[email-masked]>
Date: Fri, 7 Jul 2017 09:48:23 -0700

> was mostly thinking that proxies would at least have some knowledge about

Some load balancers (including ours) terminate the transport connection.
Being trusted middleboxes, they have the certificates and this sort of
thing works in principle.

That said, in such a case it's entirely feasible to implement HTTP/2 on the
middlebox as well.

However, I think we should look forward to the day where QUIC is
implemented in kernel space and HTTP/2 (and everything else) remains an
application. I imagine counting on all applications to do this kind of
garbage collection could cause problems.

---

From: Jana Iyengar <[email-masked]>
Date: Fri, 7 Jul 2017 09:58:21 -0700

On Fri, Jul 7, 2017 at 9:48 AM, Martin Duke <[email-masked]> wrote:

> That said, in such a case it's entirely feasible to implement HTTP/2 on
> the middlebox as well.
>

If these middleboxes terminate the transport connection for QUIC, would
they need to understand stream semantics? Or would they simply terminate
the QUIC connection, but relay the STREAM frames without any changes?
I'll note that all transports after TCP have two general parts to them --
the packet transport part, and the application semantics part. Middleboxes
are usually interested in the "lower half" of the transport - the packet
transport part.

That said, I'm not opposed to having an explicit bit. I'm also wondering if
there's a real use case, or if we're provisioning for the future here.

- jana

---

From: Martin Duke <[email-masked]>
Date: Fri, 7 Jul 2017 13:44:38 -0700

As I understand it, QUIC's contract with the upper layer includes streams,
so any such middlebox would maintain state for each stream. In general,
these middleboxes will support awareness of most L7s, and users will turn
it on.

The exceptions are probably kind of corner-case-ish. Perhaps there's a new
application directly on top of QUIC that the middlebox doesn't understand
natively. In that case, with no FIN the middlebox would be forced to hold
stream state until the end of the connection.  I guess one question is if
passing STREAM frames whole (at most changing the stream ID) is stateless
for the middlebox.

Also, I'm not clear where we ended up on silent close. If we're indeed
expecting endpoints to see that all the streams are closed to begin the
shutdown, then we *have to see that all the streams are closed.*

On Fri, Jul 7, 2017 at 9:58 AM, Jana Iyengar <[email-masked]> wrote:

> That said, I'm not opposed to having an explicit bit. I'm also wondering
> if there's a real use case, or if we're provisioning for the future here.
>
> - jana
>

---

From: Jana Iyengar <[email-masked]>
Date: Sat, 8 Jul 2017 09:51:20 -0700

> shutdown, then we *have to see that all the streams are closed.*

This is a really good point. If an application-agnostic middlebox maintains
state for a QUIC connection, it will likely need to know that all streams
are gone, especially if we do silent close. This argues for having the
explicit bit as Ian's PR proposes.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Mon, 10 Jul 2017 16:11:19 +0000

*   I guess one question is if passing STREAM frames whole (at most changing the stream ID) is stateless for the middlebox.

Just passing frames may be mostly stateless (in far as streams are concerned) – it is just transient RAM to buffer and account for the streams before the frames are delivered.  In fact, the proxy may not need to strictly pass the STREAM frames but can re-frame the data it has buffered.  One thing the proxy would not be able to do is stream state validation and would pass on invalid frames to endpoints.

A related question is how would a proxy (or a middlebox) identify QUIC traffic as QUIC and not some random UDP traffic?  We’ve had a magic byte in the header but lost it in the header update.  We still have a port number, but that is likely application-specific, unless we add something to ClientInitial to identify an application.

From: Martin Duke [mailto:[email-masked]]
Sent: Friday, July 07, 2017 4:45 PM
To: Jana Iyengar <[email-masked]>
Cc: Subodh Iyengar <[email-masked]>; Mike Bishop <[email-masked]>; Dmitri Tikhonov <[email-masked]>; Ian Swett <[email-masked]>; Lubashev, Igor <[email-masked]>; Jo Kulik <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <[email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <[email-masked]>; Kazuho Oku <[email-masked]>
Subject: Re: Unidirectional streams PR

As I understand it, QUIC's contract with the upper layer includes streams, so any such middlebox would maintain state for each stream. In general, these middleboxes will support awareness of most L7s, and users will turn it on.

The exceptions are probably kind of corner-case-ish. Perhaps there's a new application directly on top of QUIC that the middlebox doesn't understand natively. In that case, with no FIN the middlebox would be forced to hold stream state until the end of the connection.  I guess one question is if passing STREAM frames whole (at most changing the stream ID) is stateless for the middlebox.

Also, I'm not clear where we ended up on silent close. If we're indeed expecting endpoints to see that all the streams are closed to begin the shutdown, then we have to see that all the streams are closed.

On Fri, Jul 7, 2017 at 9:58 AM, Jana Iyengar <[email-masked]<mailto:[email-masked]>> wrote:
On Fri, Jul 7, 2017 at 9:48 AM, Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:

Some load balancers (including ours) terminate the transport connection. Being trusted middleboxes, they have the certificates and this sort of thing works in principle.

That said, in such a case it's entirely feasible to implement HTTP/2 on the middlebox as well.

If these middleboxes terminate the transport connection for QUIC, would they need to understand stream semantics? Or would they simply terminate the QUIC connection, but relay the STREAM frames without any changes?
I'll note that all transports after TCP have two general parts to them -- the packet transport part, and the application semantics part. Middleboxes are usually interested in the "lower half" of the transport - the packet transport part.

That said, I'm not opposed to having an explicit bit. I'm also wondering if there's a real use case, or if we're provisioning for the future here.

- jana

However, I think we should look forward to the day where QUIC is implemented in kernel space and HTTP/2 (and everything else) remains an application. I imagine counting on all applications to do this kind of garbage collection could cause problems.

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Mon, 10 Jul 2017 09:21:44 -0700

Just passing frames may be mostly stateless (in far as streams are
concerned) – it is just transient RAM to buffer and account for the streams
before the frames are delivered.  In fact, the proxy may not need to
strictly pass the STREAM frames but can re-frame the data it has buffered.
One thing the proxy would not be able to do is stream state validation and
would pass on invalid frames to endpoints.

An endpoint might use ACK as a commit signal indicating that the receiver
actually received the data. If a proxy repackages frames and does its own
ACK handing this assumption will break if not handled very carefully. If
middle boxes mess with this, it would be good to at least require them to
document this. Obviously they can’t mess unless they have a valid
certificate. So middle boxes should also be prepared for clients doing
detailed certificate validation beyond you average SSL cert. E.g. knowing
exactly which host or sub-cluster is connected. With hacking abundant these
days, this is not theoretical. But it also isn’t the std. HTTP use case.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Mon, 10 Jul 2017 16:43:22 +0000

*   An endpoint might use ACK as a commit signal indicating that the receiver actually received the data. If a proxy repackages frames and does its own ACK handing this assumption will break if not handled very carefully.

This treatment of “transport ACK” as an “application commit” is independent of STREAM frame repackaging.  The value of a proxy could be in reducing buffer requirements on endpoints, so the proxy would ACK.

In any case, such use of a “transport ACK” as an “application commit” would be a particularly tight coupling of application semantics and QUIC library (which would be required to expose an ability to delay ACKs). Since QUIC ACKs packets and not streams, it may be very difficult to even implement this without breaking QUIC.  One could be well-advised to add an explicit “commit” signal to application protocols.

From: Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
Sent: Monday, July 10, 2017 12:22 PM
To: Lubashev, Igor <[email-masked]>; Jana Iyengar <[email-masked]>; Martin Duke <[email-masked]>
Cc: QUIC WG <[email-masked]>; Jo Kulik <[email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <[email-masked]>; Mike Bishop <[email-masked]>; Martin Thomson <[email-masked]>; Ian Swett <[email-masked]>; Dmitri Tikhonov <[email-masked]>; Subodh Iyengar <[email-masked]>; Kazuho Oku <[email-masked]>
Subject: RE: Unidirectional streams PR

An endpoint might use ACK as a commit signal indicating that the receiver actually received the data. If a proxy repackages frames and does its own ACK handing this assumption will break if not handled very carefully. If middle boxes mess with this, it would be good to at least require them to document this. Obviously they can’t mess unless they have a valid certificate. So middle boxes should also be prepared for clients doing detailed certificate validation beyond you average SSL cert. E.g. knowing exactly which host or sub-cluster is connected. With hacking abundant these days, this is not theoretical. But it also isn’t the std. HTTP use case.

---

From: "Charles 'Buck' Krasic" <[email-masked]>
Date: Mon, 10 Jul 2017 10:11:56 -0700

On Mon, Jul 10, 2017 at 9:11 AM, Lubashev, Igor <[email-masked]> wrote:

>
>    - I guess one question is if passing STREAM frames whole (at most
>    changing the stream ID) is stateless for the middlebox.
>
>
>

This may not always be true?  A proxy that has certs might want to improve
perfromance, e.g. it might be aware of HTTP priorities and decide to
forward newer higher priority stream frames before older ones, in which
case it might even re-packetize.

A related question is how would a proxy (or a middlebox) identify QUIC
> traffic as QUIC and not some random UDP traffic?  We’ve had a magic byte in
> the header but lost it in the header update.  We still have a port number,
> but that is likely application-specific, unless we add something to
> ClientInitial to identify an application.
>
>
>
>
>
> *From:* Martin Duke [mailto:[email-masked]]
> *Sent:* Friday, July 07, 2017 4:45 PM
> *To:* Jana Iyengar <[email-masked]>
> *Cc:* Subodh Iyengar <[email-masked]>; Mike Bishop <
> [email-masked]>; Ian Swett <[email-masked]>; Lubashev,
> Igor <[email-masked]>; Jo Kulik <[email-masked]>; Mikkel Fahnøe
> Jørgensen <[email-masked]>; QUIC WG <[email-masked]>; Martin Thomson <
> [email-masked]>; Swindells, Thomas (Nokia - GB/Cambridge, UK) <
> [email-masked]>; Kazuho Oku <[email-masked]>
> *Subject:* Re: Unidirectional streams PR
>
>
>

-- 
Charles 'Buck' Krasic | Software Engineer | [email-masked] | +1 (408)
412-1141

---

From: "Charles 'Buck' Krasic" <[email-masked]>
Date: Mon, 10 Jul 2017 10:17:53 -0700

On Mon, Jul 10, 2017 at 9:43 AM, Lubashev, Igor <[email-masked]> wrote:

>
>    - An endpoint might use ACK as a commit signal indicating that the
>    receiver actually received the data. If a proxy repackages frames and does
>    its own ACK handing this assumption will break if not handled very
>    carefully.
>
>
>
> This treatment of “transport ACK” as an “application commit” is
> independent of STREAM frame repackaging.  The value of a proxy could be in
> reducing buffer requirements on endpoints, so the proxy would ACK.
>
>
>
> In any case, such use of a “transport ACK” as an “application commit”
> would be a particularly tight coupling of application semantics and QUIC
> library (which would be required to expose an ability to delay ACKs). Since
> QUIC ACKs packets and not streams, it may be very difficult to even
> implement this without breaking QUIC.  One could be well-advised to add an
> explicit “commit” signal to application protocols.
>

I don't agree that surfacing transport acks to the application requires an
ability to delay ACKs.   gQUIC has an AckNotifier class that does the
former but not the latter.     I think it depends on the application and
what sort of commit semantic it requires.    The line between using
transport ack or an application explicit ack might be exactly whether there
is any reason that using transport ack is too early for the app.

>
>
>
>
> *From:* Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
> *Sent:* Monday, July 10, 2017 12:22 PM
> *To:* Lubashev, Igor <[email-masked]>; Jana Iyengar <[email-masked]>;
> Martin Duke <[email-masked]>
> *Cc:* QUIC WG <[email-masked]>; Jo Kulik <[email-masked]>; Swindells,
> Thomas (Nokia - GB/Cambridge, UK) <[email-masked]>; Mike
> Bishop <[email-masked]>; Martin Thomson <
> [email-masked]>; Ian Swett <[email-masked]>; Dmitri
> Tikhonov <[email-masked]>; Subodh Iyengar <[email-masked]>;
> Kazuho Oku <[email-masked]>