Subject: Re: #385: HTTP2 Upgrade / Negotiation

From: Erik Nygren <[email-masked]>
Date: Tue, 6 Nov 2012 00:05:57 -0500

Some thoughts after having read through this thread on the flight over...
(Note that some of these are skewed from the perspective of having
spent too much time staring at one particular CDN / Acceleration Gateway
over the years, but I think some are generally applicable.)

* Any multi-port negotiation and caching (ie, for an
  Alternate-Protocol) may make sense to consider in conjunction with
  "Alt-Server" (see draft-nygren-httpbis-connection-redirect which I need
to post
  an updated version of soon) as they have quite a bit of similarity.
  One consideration in this area is that the Alternate-Protocol hint could
additionally convey the
  urgency of making the switch (e.g., as an option along with the hint
  indicating whether the switch should be synchronous or
  asynchronous).  In many cases it might be reasonable for the client
  to continue using the HTTP/1.1 connection in-parallel with
  establishing and migrating to the HTTP/2.0 connection.

* One reason Alternate-Protocol is potentially attractive as an
  option: when using it to switch to http2-over-tls (with cert
  validation), we could require that it only be used by clients which
  MUST support TLS-SNI.  This addresses at least one issue with TLS.
  (One BIG problem with manditory TLS-with-cert-validation in my view
  is that there isn't enough IPv4 space left to sanely manage the
  limitation of a Host-per-IP-address resulting from the slow rate of
  TLS-SNI adoption.  Providing a way to upgrade to TLS for individual
  connections and only for clients that support TLS-SNI can help
  here.)

* SRV records aren't too bad from a RTT perspective for sites already
  using CNAMEs as part of their setup (such as is common with some
  CDNs).  This is only one use-case as a counter-point to some of the
  other use-cases discussed here.  For example, if lookups for A and
  SRV records are made in parallel for this set of hostnames, the same
  number of round trips is involved:

     www.example.com CNAME www.example.com.example-cdn.net
     www.example.com.example-cdn.net A 10.2.3.4
     www.example.com.example-cdn.net A 10.2.3.5
     www.example.com.example-cdn.net AAAA  2001::abcd

     _http2._tcp.www.example.com SRV 0 10 80 _
http2.www.example.com.example-cdn.net
     _http2.www.example.com.example-cdn.net A 10.2.3.4
     _http2.www.example.com.example-cdn.net A 10.2.3.5
     _http2.www.example.com.example-cdn.net AAAA  2001::abcd

  Note that it would not be surprising if some
  integrations sometimes resulted in names further down the chain like
  _http2.www.example.com.example-cdn.net sometimes returning NXDOMAINs.

  It is annoying that the SRV record ends up needing to be under the
  domain of control of initial authority rather than of the actual
  server operators.  One approach here might be for the domain
  authority to CNAME the SRV record name over to a server operator
  (eg, CDN) which could address this concern (but won't help for names
  that can't be CNAMEs):

     _http2._tcp.www.example.com CNAME _http2._
tcp.www.example.com.example-cdn.net
     _http2._tcp.www.example.com.example-cdn.net SRV 0 10 80 _http2._
tcp.www.example.com.example-cdn.net
     _http2._tcp.www.example.com.example-cdn.net A 10.2.3.4
     _http2._tcp.www.example.com.example-cdn.net A 10.2.3.5
     _http2._tcp.www.example.com.example-cdn.net AAAA  2001::abcd

  As a side-note, having SRV records work sanely for HTTP would
  help in the unfortunate cases of names like "example.com"
  where a CNAME isn't allowed or supported by DNS.

* A downside of using a finer-grained prototocol version in SRV
  records (eg, "_http2_draft1") is that they are hard to get
  third-parties to deploy frequently (eg, for customer sites of a
  CDN).  As such, it may make sense to leave this as just "_http2" and
  make sure that the initial HTTP/2 draft wire versions can signal
  protocol versions and trigger a downgrade if needed.

* One complication for client authors to consider here is the matrix
  of HTTP/2 -> HTTP/1.1 fall-back behavior mixed with IPv6 -> IPv4 fall-back
  behavior.  There are perhaps some lessons to be learned from the
  IPv6 "happy eyeballs" work here as well (although the behaviors
  implemented there tend to involve much earlier fall-back from a failure
  to establish the TCP connection fast enough, while the protocol-level
  fallback is presumably higher in the stack).

* +1 to: Having a new scheme (ie, http2://) as an option seems
  potentially useful, but it shouldn't be manditory, and shouldn't be
  a primary way to operate.

One proposal might be:

1) Use HTTP/2 if the http2 scheme is specified.

2) Perform SRV record lookups in-parallel with A/AAAA lookups.

3) For HTTPS, use TPS+NPN even if the SRV record lookup failed.

4) For non-TLS, accept Alternate-Protocol hint which triggers a new
connection
    establishment  (perhaps asynchronously, as that also helps if an
alternate
    port is unavailable?)

5) An open question is whether we also want a synchronous "Upgrade"
     option as well on the same port.  (The transparent/interception proxy
issue
     seems to be the biggest risk here...)

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 15 Nov 2012 11:34:56 +1100

On 24 October 2012 14:44, Mark Nottingham <[email-masked]> wrote:

>
> > One possible way to mitigate this would be to combine the Upgrade header
> approach with a new Expect token. For instance...
> >
> >   POST /default.htm HTTP/1.1
> >   Host: server.example.com
> >   Connection: Upgrade
> >   Upgrade: HTTP/2.0
> >   Expect: 101-upgrade
> >
> > The addition of the Expect header does two things...
> >
> > 1. It makes the upgrade a requirement rather than optional...
> > 2. The client would then wait as reasonable time for the 101 Switching
> Protocols before it proceeds to send the http/2.0 encoded data...
> >
> > When the client does start sending the data (after receiving the 101
> response), it would do so by first sending a SYN_STREAM.. To http/1.1
> servers, this would appear to be nothing more than regular POST payload
> data. For http/2 traffic, this requires an initial round trip but it
> addresses the upgrade on payload-bearing request issue... and is exactly
> the kind of thing the Expect header was intended for. The scheme is
> obviously far from perfect, however.
>
> Hm. I'd be interested to hear from implementers on this. The only
> advantage that I see is that you'd be able to start multiplexing other
> requests on the same connection sooner. However, you could also just use
> SRV, or a different connection if it's not available.

Sorry for slow response - been out of the loop for a few weeks.

As an implementer, this kind of upgrade is fine for new server
implementations, but I don't think it helps much.

If the client wants to assume success of the upgrade and start sending
http/2 frames before the 101, then that could easily be handled by a new
server regardless of METHOD.    Having the POST there does not really
change the situation, and I'd actually prefer it if the method was left to
be the real method, so that the server could send a valid response after
the 101 as a http/2 frame (eg if the upgrade is in a GET /index.html
request then the first frames back could be a push of the index.html
content or a 304 etc.).

Using a POST to make subsequent HTTP/2 frames sent by the client look like
a body to a HTTP/1 server/infrastructure is fraught with issues and buys us
very little. Most site interactions commence with a single request and it
is only once content for that request is received that the client will
start generating demand for more requests in parallel.  So that demand
coincides with knowing if the upgrade was successful or not, so the
protocol is known.

So let's totally avoid sending http/2 frames to http/1 servers and getting
into the whole how-might-it-be-interpreted debate that stalled the hybi-wg
for over a year.

So long as the server is able to push a response to the upgraded request
immediately after the 101 (along with a push of any other expected
content), then I think upgrade along the lines of
http://tools.ietf.org/html/draft-montenegro-httpbis-http2-negotiation-00can
be a good low latency way to establish a HTTP/2 connection.

There may be some other use-cases that do need to open many http/2 mux
streams in parallel immediately, but they are not from normal browsers, so
I think we can address them with the optimisation ideas for making direct
HTTP/2 connections (DNS, new port etc.).

cheers

-- 
Greg Wilkins <[email-masked]>
http://www.webtide.com
Developer advice and support from the Jetty & CometD experts.