Subject: HTTP 1.1 --> 2.0 Upgrade

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 11:16:51 +0300

Hi

There's been a recent discussion about signaling support for HTTP 2.0 in the DNS. I think that is a good way to go about it, but it doesn't cover all cases.

I believe there is value in an upgrade mechanism, for example for home routers and other devices that use HTTP for configuration long before (if ever) they have DNS entries, let alone SRV records.  While such a mechanism may also be useful for HTTPS, something like NPN can be used as part of the TLS handshake.

So here's my proposal:

 1.  When first connecting to a website, the client begins with HTTP 1.0/1.1.
 2.  A server that supports HTTP/2.0 responds with a new header: "HTTP-VERSIONS: 1.1,2.0". This header is included in the server's response to the client's request.
 3.  The client uses a new pseudo-method "UPGRADE-TO_VER 2.0 HTTP/1.1\r\n\r\n". Everything following is HTTP/2.0. There can be headers there, but I don't see much need for any.
 4.  The client also caches that this server supports HTTP/2.0. Next time, it won't need the upgrade mechanism.
 5.  The server replies with a "HTTP/1.1 200 OK\r\n\r\n" and everything that follows is HTTP/2.0. Again there may be headers.

A related but orthogonal issue is finding out if the path between the client and the server supports HTTP/2.0. Fortunately, middleware does not tend to appear mid-path. It's usually either server-side (reverse proxies, load balancers) or it's a client-side firewall / proxy.  Server side should not concern us, because there's no point in configuring a server to support (and advertise through this header or a DNS record) that HTTP/2.0 is supported, when the load balancer blocks it. That's a clear misconfiguration. Client side middleware is either there or it's not. It's not likely to be present on-path to one generally available website while not being present for another.

One obvious way to find out, is to have some well-known site that supports HTTP/2.0 (browser vendors or others can run these sites). A client can perform a simple HTTP/2.0 GET from such a site whenever it starts, or whenever something in the network settings has changed (interface, IP address, next-hop router, routing table). The actual heuristics would have to be more involved, since we wouldn't want downtime on a single server to push a significant part of the web to HTTP/1.1.

If people find this idea interesting, I'd be happy to write it up in a draft.

Yoav

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Aug 2012 10:25:46 +0200

On 2012-08-20 10:16, Yoav Nir wrote:
> Hi
>
> There's been a recent discussion about signaling support for HTTP 2.0 in
> the DNS. I think that is a good way to go about it, but it doesn't cover
> all cases.
>
> I believe there is value in an upgrade mechanism, for example for home
> routers and other devices that use HTTP for configuration long before
> (if ever) they have DNS entries, let alone SRV records.  While such a
> mechanism may also be useful for HTTPS, something like NPN can be used
> as part of the TLS handshake.
>
> So here's my proposal:
>
>  2. A server that supports HTTP/2.0 responds with a new header:
>     "HTTP-VERSIONS: 1.1,2.0". This header is included in the server's
>     response to the client's request.

There's already an "Upgrade" header field for that.

>  3. The client uses a new pseudo-method "UPGRADE-TO_VER 2.0
>     HTTP/1.1\r\n\r\n". Everything following is HTTP/2.0. There can be
>     headers there, but I don't see much need for any.

I don't think we need a new mechanism here; we already have "Upgrade" as 
a request header field.

>  4. The client also caches that this server supports HTTP/2.0. Next
>     time, it won't need the upgrade mechanism.

How does the recipient then know that the message is HTTP/2.0?

>  ....

Best regards, Julian

---

From: Eliot Lear <[email-masked]>
Date: Mon, 20 Aug 2012 10:32:30 +0200

On 8/20/12 10:16 AM, Yoav Nir wrote:
> Hi
>
> There's been a recent discussion about signaling support for HTTP 2.0
> in the DNS. I think that is a good way to go about it, but it doesn't
> cover all cases.
>
> (if ever) they have DNS entries, let alone SRV records.

What about using bonjour / mdns / etc?

Eliot

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Aug 2012 10:49:53 +0200

> There's already an "Upgrade" header field for that.

+1.

All this work has been done over and over for WebSocket, I don't see
why we need to reinvent the wheel.

Upgrade was designed *exactly* for that purpose and fits it very well.
Moreover it is the only one (along with NPN) which considers the
intermediary chain, while side-band mechanisms (such as DNS) cannot
take that into account.

Best regards,
Willy

---

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 11:50:18 +0300

> There's already an "Upgrade" header field for that.

I was going to say that "Upgrade" is so rarely used, there may be buggy implementations out there that will balk at seeing this. But since websockets uses the upgrade mechanism, I guess that problem is solved by now.

> I don't think we need a new mechanism here; we already have "Upgrade" as 

Agree. 

> How does the recipient then know that the message is HTTP/2.0?

Remains to be seen what HTTP/2.0 will actually look like, but SPDY-00 is very easily distinguishable from HTTP/1.x. Connections start with control frames, and control frames have the high bit on the first octet set. HTTP/1.1 requests tend to have ASCII characters at the start. I think a heuristic approach to telling HTTP/1.x from HTTP/2.x is good enough for the servers.

Yoav

---

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 11:55:05 +0300

On Aug 20, 2012, at 11:32 AM, Eliot Lear wrote:

>> I believe there is value in an upgrade mechanism, for example for home routers and other devices that use HTTP for configuration long before (if ever) they have DNS entries, let alone SRV records.
> 

Like DNS SRV records, it's great when it works, but both are often filtered (or not implemented)

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Aug 2012 10:59:17 +0200

On Mon, Aug 20, 2012 at 11:50:18AM +0300, Yoav Nir wrote:
> Remains to be seen what HTTP/2.0 will actually look like, but SPDY-00 is very
> easily distinguishable from HTTP/1.x. Connections start with control frames,
> and control frames have the high bit on the first octet set. HTTP/1.1
> requests tend to have ASCII characters at the start. I think a heuristic
> approach to telling HTTP/1.x from HTTP/2.x is good enough for the servers.

In fact those arguments were made for WebSocket too, and this went into a
one-year long dabate with examples of how dumb intermediaries could be fooled
etc... In theory I agree with you on the point above, but in practice we want
to avoid annoying corner cases and more specifically we want to avoid wasting
another year like we did when designing the HTTP/1.1->WS handshake :-)

Cheers,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Aug 2012 11:02:33 +0200

> Like DNS SRV records, it's great when it works, but both are often filtered (or not implemented)

Or not applicable. Think about how many web developers connect to
$IP:$PORT during their development phase and have tens of available
ip:port combinations to test various software versions/variants
without any DNS being aware of them.

We must never forget that what made HTTP succeed is also the fact that
it's self-contained, does not require additional side-band protocols,
works end-to-end over a single connection and is 100% NAT-friendly by
not storing any IP-related information anywhere.

Willy

---

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 12:02:31 +0300

> why we need to reinvent the wheel.

Agree. I forgot that WebSocket uses Upgrade

> intermediary chain, while side-band mechanisms (such as DNS) cannot

Intermediaries generally leave unknown headers alone, so they will have to be upgraded to either identify (and remove) attempts to upgrade to 2.0, or else to support 2.0. It's possible that this work (by intermediaries) has already been done because of WebSockets, though.

NPN is different, because TLS proxies generally don't just pass along extensions they don't know about, so blocking the NPN extension effectively disables SPDY.

Yoav

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 20 Aug 2012 05:35:24 -0400

I think we need a lot more explanation of what this intermediary chain
is and which side of the wall it lies.

Configuration of server side proxies such as squid has zero impact on
this issue. Server installations that support HTTP/2.0 will be
appropriately configured.

Are client side proxies really more than a corner case issue today?
When we first developed them the whole of CERN was sitting on a T1 and
caching was a critical concern. Only a small percentage of the HTML on
the Web is even cachable.

Content Delivery Networks have caching but they work in a very
different way to HTTP caching.

I am pretty skeptical as to the value of firewall proxies when most
let port 443 through.

-- 
Website: http://hallambaker.com/

---

From: Roberto Peon <[email-masked]>
Date: Mon, 20 Aug 2012 03:23:32 -0700

For port 80:
What WebSockets has done on port 80 with upgrade seems basically fine. The
part that is unfortunate is that it requires about as much time as doing
the TLS handshake (an additional round-trip).
Having the data in DNS could short-circuit this, but shouldn't be a
dependency.

I still do like the idea of "alternate-protocol", which is extremely poorly
described in the SPDY spec. The idea of alternate-protocol was that the
server adds the header detailing the protocol, port, and negotiation
mechanism. The client, if it understands it, may attempt in parallel one of
the alternate protocols. This allows a switch of protocol and port, and a
possible upgrade (no guarantee, given that HTTP offers plaintext and is
100% completely insecure) to using TLS for HTTP schemes, assuming both
client and server agree to do it. This is separate from an upgrade,
however, in that the new protocol is spoken over a new connection which is
used only after it is established (with the previous connection remaining
in use until supplanted), and thus the transfer of resources is not blocked
by application-layer handshakes at any point in time.

For other-than-port-80:
I prefer something NPN-like, hopefully where the protocol being negotiated
is not exposed (that will be a matter for the TLS WG), is my favorite
w.r.t. :443, as one might guess. The reason, however, is that when someone
decides to do HTTP/3 or something else, I'd like to have left the ground
fertile and helped to prevent the ossification of the web.
In other words, design with the idea that we won't solve the entire suite
of problems on this go-round.

-=R

On Mon, Aug 20, 2012 at 2:35 AM, Phillip Hallam-Baker <[email-masked]>wrote:

> in

---

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 13:27:12 +0300

Agree.

> Are client side proxies really more than a corner case issue today?

Yes, they're everywhere - hotels, airports, workplaces, coffee shops.

> the Web is even cachable.

At CERN you didn't have lawyers telling you you had to get people to agree to terms and conditions. You probably didn't redirect them to a secure site where they could enter credit card numbers to get an hour of access either.

> I am pretty skeptical as to the value of firewall proxies when most

Value is subjective, but TLS proxies have been available for six years, and are now part of most of the "next generation firewall" products. They inspect deeply enough that they can enforce a policy of "accept facebook, but not farmville". And they're getting installed in corporate networks all over the place. Obviously they need to understand the HTTP protocol, so we have to be careful of deploying something that they will block.

As long as there's a well-defined algorithms to figure out whether a particular stream is 1.1 or 2.0, those intermediaries can "grow" to deal with 2.0. But we also need client and server to be able to work together when an old intermediary is present that drops what it considers to be malformed HTTP.

Yoav

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Aug 2012 12:36:11 +0200

Hi Roberto,

> the TLS handshake (an additional round-trip).

As we suggested in our network-friendly draft, its possible to send multiple
GET requests at once during the handshake by passing the list of URIs in a
dedicated header. It basically allows us not to waste the round trip.

> Having the data in DNS could short-circuit this, but shouldn't be a

The DNS requires a round trip anyway and it currently where time is wasted
in mobile environments.

> by application-layer handshakes at any point in time.

Maybe this could still be done over the established connection using an
Upgrade ? However, I'd hope we avoid announcing IP and ports in exchanges,
as it will cause the NAT mess we managed to avoid for years with HTTP and
which made other protocols disappear.

Regards,
Willy

---

From: Eliot Lear <[email-masked]>
Date: Mon, 20 Aug 2012 12:56:27 +0200

Yoav, Willy,

>> Like DNS SRV records, it's great when it works, but both are often filtered (or not implemented)

Good for the home.  And nothing stops this. By the way, I am still not
convinced, myself, that SRV records are the correct approach.  Or at
least not what we call SRV records today.  Again, the http URI schema
defines a port #.  To me, that's the end of SRV as we know it.  But, one
could imagine a record that would address this concern.  How about this:

www.example.com    IN    BLARF    (80, HTTP2.0,other capabilities...),
(83,HTTP1.1,other capabilities), ...
                                   IN   AAAA    2001:...

And in the default case fall back to 1.1, which is what Phillip pretty
much suggested.  This gets you to 2.0 before you connect.  Then for
those cases which are not in the DNS, as willy suggests, you do it in
path (NPN, upgrade, or whatever).  But here's the question: would such a
scheme really provide substantial benefit?  That is (a) does it really
save an RTT and (b) does that RTT matter to people?

> not storing any IP-related information anywhere.

Well indeed.  I work at a routing vendor, for crying out loud.  Our
routers probably don't have DNS entries when they're being configured.

Eliot

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 20 Aug 2012 09:18:51 -0400

I really don't see content filtering proxies as a problem

The reason being that anyone doing content filtering has to constantly
keep ahead of the game and track the current status of the Web.
Facebook and Farmvile are only the fads of the season, there will be
another and another. Remember MySpace, TheWorld.com, Yahoo, GeoCities?
FB may survive but it certainly won't be the last big thing on the
Net.

So are these firewalls products or services? An actively maintained
service is really not a problem at all, they will keep up to date. Its
fire and forget products that cause grief.

Restrictive home routers on the other hand could be here for decades.
[Well they could if they have started making them out of non
substandard parts. Mine used to last about 7 months until I switched
to Apple. I have a box full of Lucent, 3Com, Linksys, Netgear and
Belkin routers that all turned out to be crap.]

---

From: Patrick McManus <[email-masked]>
Date: Mon, 20 Aug 2012 09:45:12 -0400

In vancouver we agreed not to slaughter too many bytes over mandatory
TLS; but we also talked about the need to describe how HTTP/2 binds to
TLS. I'm going to narrowly scope this email to the issues involved with
those situations and how the upgrade should, imo, work without getting
into every detail:

Case 1: https:// scheme - go to the port in the url (default 443) and
use NPN to select http/2 or http/1 (default is http/1 in the case NPN is
not implemented on the server)

Case 2: http:// scheme with default port - optimistically consult DNS
SRV looking for either http2-tls or http2-plaintext depending on what
the client wants. If you get a SRV result back "quickly" go to the host
and port provided and start speaking http2 over tls or plaintext as
appropriate. No extra indirections, no use of http/1. Reasonable server
implementations are forseeable.

Case 3: http:// scheme with an explicit port or lack of SRV information.
This is where we need to bootstrap from HTTP/1. AFAICT people want two
different patterns:

Case 3a: Direct the traffic to another location like Alternate-Protocol
does... basically bundling srv information with an http/1 response..
"here is the answer to your http/1 request and btw over on port 443 we
speak http2-tls and on port 8080 we speak http2-plaintext".. clients can
decide whether they want to jump over there right away or overlap some
more http/1 transactions with the handshake on the other port.. its
probably not important for http/2 to define exactly what happens other
than the lifespan of that alternate-protocol announcement.

Case 3b: transform the current connection into an http/2 connection
(possibly even adding in tls the same way CONNECT does) with http/1
upgrade roughly like websockets does. 

Does that framing seem reasonable?

Does anyone want to make an argument for having both? I'd prefer not to
at this stage but the question seems worth asking.

In favor of 3a - 1] the results are cached and future connections are
pure http/2 without bootstrapping them each time. 2] It also can move
http/2 to a non-port-80 location which alleviates concerns about
transparent proxies being confused.

In favor of 3b - 1] it doesn't require another connection as the current
one is transformed in place. 2] it sidesteps issues about caching the
discovery information of alternate-protocol and its scope and maybe even
security considerations with it. 3] websockets uses port 80 and I'm not
overwhelemed with bug reports about unreachable services due to the http
proxy problem, though the amount of use of websockets is tiny compared
to http and many websockets applications have comet/etc fallbacks just
to support browsers without a ws implementation - so the truth here
might not yet be known.

Mostly I favor 3a because the caching reduces the need for latency
inducing bootstraps while maximizing the amount of http/2 that is used.
I don't constantly like redoing that bootstrap. It also dovetails nicely
with the srv approach as it basically tunnels the srv into http/1.

-Patrick

---

From: Yoav Nir <[email-masked]>
Date: Mon, 20 Aug 2012 17:46:31 +0300

> keep ahead of the game and track the current status of the Web.

It's somewhere between "has to" and "should". 

> FB may survive but it certainly won't be the last big thing on the

There are more fine-grained stages here. At first it was some game that PFYs were enjoying. Nothing to be concerned about. Later it was seen as a time waster where people would "like" and comment on their friends statuses. It would be trivial, even with a previous-generation firewall (or a router with an ACL) to block the FB addresses. The later stage is when administrators don't want just a "block" or "allow" option, but more fine-grained control - allow people to comment on friends' statuses, but not play a bunch of time-waster games. 

That is when the firewall vendors come out with the special content filtering features. That's months or years after the thing exists on the Net.

Then there's the issue of when administrators deploy it. Firewalls are generally not treated like Chrome. The customers are not running the latest version. They might very well be running a two- or three-year old version. 

So by the time that the intermediary is deployed even in well-managed networks, the thing may have been on the net for several years.

If Chrome and the Google websites deploy quickly, they're bound to bump into intermediaries that are not ahead of the game. That is why the NPN works so well. Any intermediary that does TLS proxying, but does not know about SPDY will drop the NPN extension, disabling SPDY. If SPDY was signaled through a DNS record, those connections would be dropped as invalid HTTP.

> fire and forget products that cause grief.

Both kinds of products exist, as well as combinations. Firewalls are sometimes kept past their update date, and are sometimes not updated at all. Just as you can't rely on the browser not being IE6, you can't rely on the intermediary being up to date.

> Belkin routers that all turned out to be crap.]

My Netgear is 7 years old, and would still be going strong, except my cable modem died last month, and the cable company replaced it with one that includes a wireless router. Luckily it never tried to filter anything.

Yoav

> Scanned by Check Point Total Security Gateway.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 20 Aug 2012 19:39:28 +0200

Hi Patrick,

> not implemented on the server)

Ok.

> implementations are forseeable.

As in happy eyeballs?  Sure.

> Does that framing seem reasonable?

Reasonable?  Yes.  Best course of action?  I don't know.  I know I'm
beginning to sound like a broken record, but I don't know if SRV is the
right record for the job.  For one thing, if you use another record, you
can use that same optimistic approach for an explicit port in the URI. 
Of course, then there is the added implementation headache of a new
record.  We've gotten around that elseewhere with TXT records, but I
don't recommend it here.

> with the srv approach as it basically tunnels the srv into http/1.

I'd be a bit nervous about cache management, and specifically how/when
to invalidate.  Two obvious cases are when the IP address changes, and
when for some reason the other side starts barfing on SPDY.

Eliot

---

From: Ted Hardie <[email-masked]>
Date: Mon, 20 Aug 2012 12:00:08 -0700

> record.  We've gotten around that elseewhere with TXT records, but I

Do you mean to suggest something like
http://tools.ietf.org/id/draft-faltstrom-uri-06.txt ?

regards,

Ted

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 20 Aug 2012 15:25:49 -0400

I think the URI proposal is an excellent way to allow a Web Service to
be identified by means of just a DNS name. So instead of specifying
http://wev.google.com/_well-known/wev, someone can just type
'google.com' into their wev client and it works.

It is not a good match for resolving http URLs but it is a good way to
resolve a raw dns name and a service identifier to a web service
endpoint.

That said, how does a Web service using the URI scheme tell clients
that the service is available via http/1.1 or http/2.0 so that it can
select the appropriate host and port first time?

It would be rather useful to have http2: and http2s: defined as URL
schemes for 'this really has to run over http2, don't try anything
else, just do it', ditto for http1 and http1s and have http:, https:
be used for 'try what works'.

http2 is not a scheme people should use as a regular thing, it should
only be used when the distinction between http1 and http2 really
matters. For example, when the Web site is only available on http2 and
http1 is not supported or for debugging Web sites or to enable
negotiation of the protocol version whether via SRV or some other
scheme.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 20 Aug 2012 13:47:56 -0700

> record.  We've gotten around that elseewhere with TXT records, but I

For my part, I don't really care which record is chosen to bear the
information, but do believe that DNS is a good mechanism for providing an
appropriate hint.

> > to support browsers without a ws implementation - so the truth here

The argument for also doing upgrade is that it would be awesome if we could
get upgrade to work reliably on port 80 so that replacements to HTTP/2 can
be done reliably in the clear on port 80.
Additionally, upgrade could be an alternate-protocol hint,
e.g. alternate-protocol: 80:upgrade-http/2

> > with the srv approach as it basically tunnels the srv into http/1.

Ditto.

> when for some reason the other side starts barfing on SPDY.

The same problem (figuring out when to fall back to using HTTP/1.1) exists
in the same form with upgrade over port 80 and intermediaries. This problem
must be solved regardless of the negotiation hint/solution, even if it
affects small percentages of users, because the nature of the error would
be persistent. There has to be a decent heuristic mandated which allows a
user to eventually get their data.
-=R

---

From: Roberto Peon <[email-masked]>
Date: Mon, 20 Aug 2012 13:55:46 -0700

> dedicated header. It basically allows us not to waste the round trip.

I agree that this can work, assuming all the URLs fit in the headers (many
servers/proxies have a maximum limit on header size to reduce attack
surface) and no other headers are required.
Having seen some incredible URLs and headers (e.g. cookies), I'm wary of
assuming this is reasonable.
For this to work, it also mandates some annoying interaction between the
HTTP/1.1 code and whatever follows it.

> The DNS requires a round trip anyway and it currently where time is wasted

This is an unavoidable round-trip, though. If you don't know where to go,
one must figure that out.

> which made other protocols disappear.

I'm not understanding how this would cause NAT messes? I'd guess that most
external sites would just continue to use :80 and :443. Some internal sites
wouldn't, and the network there is controlled well enough that this should
cause no problem. In any case, sites are incented to be sure that users can
connect to them, and will quickly decide to simply keep using :80 and :443.
When one has an inspecting intermediary, it probably won't like to see the
upgrade, but probably would be perfectly fine with a different connection
(e.g. TLS on :443). There is simply less complexity there, imho.

-=R

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 20 Aug 2012 23:27:14 +0200

> surface) and no other headers are required.

Agreed but Apache's 8kB per header tends to be around the common usage
nowadays. And if we have to push 8kB worth of request URIs, we'll often
hit the initcwnd anyway.

> assuming this is reasonable.

Agreed but quite frankly, what we're trying to achieve is a way to make
the web fast for everyone, provided there is no deliberate server-side
sabotage :-)

> HTTP/1.1 code and whatever follows it.

Yes, that's what I find a bit annoying eventhough it's not terribly
complex as long as we support GET requests over URIs with the same
headers as the first one.

> one must figure that out.

Not necessarily (eg: explicit proxies). Eventhough the proxy knows it can
talk 2.0 with the server, how does the client know it can talk 2.0 to the
proxy ?

Also, you can still have /etc/hosts pre-filled with some of your operator's
important services (login portal or whatever helps you troubleshoot your
connection settings). One might even imagine that common DNS entries could
regularly be pushed in multicast to end users to save a few round trips. Or
these entries might have been advertised in HTTP response headers as you once
explained to me :-)

In fact there are plenty of ways to access services over HTTP and I tend to
find it akwards to increase dependency on DNS.

> external sites would just continue to use :80 and :443.

There should be a single reliable way of getting a host IP and a port. Right
now it works well. If you change your /etc/hosts to access your development
site, you reach it without any issue and everything works well. If you want
to install a reverse-proxy-cache in front of it, you can have this cache
use the hard-coded IP address and have it work fine.

At the moment you start exchanging IP addresses in the protocol, you're getting
into trouble. Sometimes the IP is valid, sometimes it doesn't mean anything and
must be rewritten by *someone* along the chain, but most often, this *someone*
doesn't care or is not the same as the one which translates the address. I guess
everyone here has already been involved in helping any admin to rewrite Location
headers that were improperly set by a customer application that had no clue
about the valid host name or IP from the outside. This is typically the
situation we should try to avoid.

In my opinion, once an IP address and a port are known, whatever the means,
whether they're entered by hand in a URL bar of a browser, configured in
/etc/hosts, are specified in a config file as the address of a web service
or as a next hop for a reverse-proxy, we shouldn't care how that address was
learned, we should be able to use HTTP/2 over it if desired.

> Some internal sites

Possibly, but this doesn't require relying on a side-band registry to know
the address and ports. After all, you have enumerated the well-known ports
above. It could be mandated that if an http connection to :80 fails, a browser
should automatically retry on 443 for instance, without having to deal with
registries such as DNS which are OK on the internet and never maintainable
internally, nor with IP addresses advertised in headers.

Regards,
Willy

---

From: Roberto Peon <[email-masked]>
Date: Mon, 20 Aug 2012 15:43:16 -0700

> the web fast for everyone, provided there is no deliberate server-side

I hope we make it fast(er) even in spite of some small dosages of stupid,
but yea, you can't fix stupid. :)

> talk 2.0 with the server, how does the client know it can talk 2.0 to the

Good question. I'd assume proxy.pac or equivalent configuration would have
to spell it out.
In any case, the cost of getting it wrong with most explicit proxy setups
is fairly small because the proxy is generally close.
We need to worry about avoiding additional RTTs when RTTs are big, not so
much when they're small.

> these entries might have been advertised in HTTP response headers as you

Agreed, in the cases where you've replaced DNS, DNS is irrelevant. :)

> find it akwards to increase dependency on DNS.

The idea as I understood it wasn't to add any dependency-- merely a way of
advertising what protocol can be spoken next during what is (for all but
those who ignore DNS). In these cases, the client would attempt the new
protocol first, likely get an error quickly, and downgrade immediately.
With a local proxy, this should not at all be costly as, well, it has a
small RTT to the user.

> situation we should try to avoid.

Everything which doesn't handle this already is significantly broken--
sometimes links have IP addresses in 'em. This is expected and valid for
the web...

> learned, we should be able to use HTTP/2 over it if desired.

Wouldn't the proxy be doing the DNS resolutions here to figure this out?
The client would likely have to do UPGRADE or alternate-protocol or NPN to
the proxy to figure out what it'd be using, but that'd be normal...

> internally, nor with IP addresses advertised in headers.

Agreed. I don't think anyone is yet suggesting *mandating* DNS schemes...

-=R

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 21 Aug 2012 16:23:57 +1200

On 21.08.2012 10:43, Roberto Peon wrote:
>> > > The DNS requires a round trip anyway and it currently where time 
>> is
>> wasted
>> > > in mobile environments.
>> > >
>> >

.. .for values of "small" which can reach more than 20 seconds:
http://www.potaroo.net/ispcol/2012-05/notquite.html

Since a failed connection attempt is a full TCP handshake connection. 
Doing one failure for HTTP/2 and a second attempt for HTTP/1 would be 
worst-case around 45 seconds to *start* the request process.

>> these entries might have been advertised in HTTP response headers as 
>> you
>> once
>> explained to me :-)
>>
>

+1.

> With a local proxy, this should not at all be costly as, well, it has 

Consider the pathway scenario, which is more common:
   A->B->C->D

DNS can tell A what D's capabilities are. But A is not connecting 
there. Depending on those capability advertisements when available would 
be a costly mistake.

>> > > as it will cause the NAT mess we managed to avoid for years with 
>> HTTP
>> and
>> > I'm not understanding how this would cause NAT messes? I'd guess 
>> that
>> most
>> There should be a single reliable way of getting a host IP and a 
>> port.
>> Right

Swap "host" with "recipient" and you have the requirements lined up 
correctly. Final end-server in the chain does not matter in the 
architecture like HTTP where proxy has  existence, the host being 
connected to is all that matters. Any overlap between the two is just a 
convenience.

>>
>> At the moment you start exchanging IP addresses in the protocol, 
>> you're
>> getting
>> doesn't care or is not the same as the one which translates the 
>> address. I
>> guess
>> everyone here has already been involved in helping any admin to 
>> rewrite
>> Location
>> or as a next hop for a reverse-proxy, we shouldn't care how that 
>> address
>> was

*sometimes*. There is a lack of MUT NOT clause in the specs to prevent 
clients doing for example; same-origin validation.

> the proxy to figure out what it'd be using, but that'd be normal...

Exactly why, IMHO, alternatives to those are a waste of time 
considering unless they can provide the same guarantee for handling 
transit over middleware (both explicit and interceptors) safely.

SRV is a non-starter due to end-host servers not being aware of the 
details to emit for intermediaries.

I think we should be taking Upgrade: and defining ways to optimize it. 
Willy had some proposals for merging multiple requests and defining 
1.1->2.0 mapping in the headers of request #1. Figuring out how much of 
that is useful and where it could be improved would be a good way to 
progress the bootstrapping.

>>
>> > Some internal sites
>> > wouldn't, and the network there is controlled well enough that 
>> this
>> should
>> > cause no problem. In any case, sites are incented to be sure that 
>> users
>> can
>> > connect to them, and will quickly decide to simply keep using :80 
>> and
>> :443.
>> > When one has an inspecting intermediary, it probably won't like to 
>> see
>> the
>> above. It could be mandated that if an http connection to :80 fails, 
>> a
>> browser

I think we should be doing a twofold definition:
* port 80 Upgrade: from 1.x to 2.0
    - long term plans to obsolete HTTP/1.x

* port 80 Upgrade: from HTTP to HTTPS/2
    - long term plans to obsolete port 443 usage and make the http:// 
scheme TLS-enabled/secure whenever possible.

Amos

---

From: James M Snell <[email-masked]>
Date: Mon, 20 Aug 2012 21:59:51 -0700

So, I've been catching up on this thread and there's just one thing that I
just want to clarify: the upgrade mechanism that's being discussed is not
going to be a mandatory component of HTTP/2.0, correct? That is, I as an
implementor am not going to be required to do this upgrade dance with every
new connection, regardless of how it actually happens... right? I'm just
saying because there are a vast array of use cases where such a dance is
going to be wholly unnecessary... I would even argue that, in MANY cases,
having something as simple as http2://www.example.com that operates on a
different default port is going to be more than sufficient if the
environment is fairly controlled and predictable on both ends. Throughout
this discussion, there seems to be an implicit assumption that upgrade
discoverability is an absolute requirement.

- James

On Mon, Aug 20, 2012 at 9:23 PM, Amos Jeffries <[email-masked]> wrote:

> http://www.potaroo.net/ispcol/**2012-05/notquite.html<http://www.potaroo.net/ispcol/2012-05/notquite.html>
>
> DNS can tell A what D's capabilities are. But A is not connecting there.
> Depending on those capability advertisements when available would be a
> costly mistake.
>
>
>
>
>>
> architecture like HTTP where proxy has  existence, the host being connected
> to is all that matters. Any overlap between the two is just a convenience.
>
>
>
>
>>> doesn't care or is not the same as the one which translates the address.
>>> I
>>> guess
> Exactly why, IMHO, alternatives to those are a waste of time considering
> unless they can provide the same guarantee for handling transit over
> middleware (both explicit and interceptors) safely.
>
>    - long term plans to obsolete port 443 usage and make the http://scheme TLS-enabled/secure whenever possible.
>
>
> Amos
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 21 Aug 2012 07:04:23 +0200

Hi Roberto,

> We need to worry about avoiding additional RTTs when RTTs are big, not so

Not in mobile environments. The highest waste of time between your smartphone
and the net are the first DNS requests for the various subdomains required to
download a page. That's why operators are rewriting/inlining contents with IP
addresses as prefixes to fetch additional objects.

When mobile operators finally decide to advertise explicit proxies, the proxy
will be far from the mobile (say 300ms) but close to the net. There you want
your smartphone to always use v2 to talk to these proxies and let those proxies
act as they want on the net.

> With a local proxy, this should not at all be costly as, well, it has a

If you remember, this is exactly like all the conversations we had about
WS 2 years ago to save round trips in case of failures. The beauty of the
HTTP Upgrade mechanism is that it can fail softly with an automatic fallback
to HTTP/1. So as long as port 80 is in use, it is possible to always advertise
the "Upgrade: HTTP/2.0" header and hope for upgrades, regardless of any other
information. Similarly for https, you can announce HTTP/2.0 in NPN and hope
for the next hop to accept it. For alternate-protocol, I don't know (I need
to read this specific part of your spec first :-)).

There should be no valid reason not to make tentative upgrades for any site,
except if it's blacklisted. But then, only the user-agent will be able to
blacklist it, we won't expect the site's owner to advertise itself as
blacklisted.

> sometimes links have IP addresses in 'em. This is expected and valid for

I'm not speaking about what is delivered in contents, because contents are
not interesting for intermediaries, but about the protocol. If you expect
a proxy or reverse-proxy to make certain use of a header value and this
header transports an address or port information, I already bet there will
be a large number of deployment issues (possibly even security issues).

> Wouldn't the proxy be doing the DNS resolutions here to figure this out?

Typically in the examples above, there would be no DNS resolution. I don't
count the number of times I've seen IP:port in app servers config files, or
even in haproxy config file. I'm used to see Apache in reverse-proxy where
you have your /etc/hosts map the server name to the next hop too. There is
no DNS involved there either.

Another example, think about your latest set-top box you unbox and plug to
your network. The manual says "connect to http://192.168.1.100/ with your
browser". If a new auth scheme is only available to 2.0, you'd rather have
your browser automatically try 2.0 there without any DNS request.

> Agreed. I don't think anyone is yet suggesting *mandating* DNS schemes...

No, however if we need to use it to advertise the protocol version, it means
we failed to design a proper and efficient upgrade mechanism instead. If the
mechanism is valid, there is no need for help from side-band protocols. That
is my point.

Regards,
Willy

---

From: Roberto Peon <[email-masked]>
Date: Mon, 20 Aug 2012 23:08:02 -0700

> addresses as prefixes to fetch additional objects.

I hope this isn't the case, as renaming origins in the links that
content-providers send will break security models and break pages or allow
for some interesting exploits.

That aside, as I understand it, much of the delay on the DNS request is
actually attributable to the radio start-up time (channel acquisition,
signaling, etc). This should be viewed as a fixed latency cost for the
first RTT. Otherwise, sure, DNS resolutions can take some time, around
100-200ms.

> act as they want on the net.
This would be a matter of proxy configuration in the user-agent, possibly
including the protocol version to attempt to speak.
I don't see how this is relevant? A browser so configured, if it did do DNS
resolutions, would/should discard that information as it isn't making that
connection.
If it did make a connection not going through the proxy, then the DNS
information is valuable.
The DNS information is valuable to the proxy in this case, which is what I
was trying to say earlier...

> to read this specific part of your spec first :-)).
As I remember those discussions, the design-intent was to cause a
non-compliant proxy to close the connection. This is related, but different
from requiring browsers to have decent heuristics for deciding that they
got the protocol version wrong.

One of the problems I have with upgrade as the sole mechanism is that it
will keep HTTP/1.1 around indefinitely as a required part of HTTP/2's
handshake. That doesn't seem very good to me. A user-agent should be able
to learn about the protocol the site is likely speaking, and use that
first. If it messes up (which will happen, though rarely), it should be
able to recover withing the attention-span of the user and remember...

> blacklist it, we won't expect the site's owner to advertise itself as
I think I agree here. I'd interpret "tentative upgrades" as negotiation for
the next protocol (via upgrade header), but it is nitpicking.

> be a large number of deployment issues (possibly even security issues).
Note that in all cases, it is the server, which is either public, in which
case there is no issue, or it is private, and should not be sending this
header, or the headers should be filtered out if it traverses the firewall
and refers to an internal IP. Generally, publicly reachable servers need to
know their own name, whether that be domain-name, or IP, or cert, or all of
the above.

The only party for whom any DNS information or alternate-protocol
information should be interesting is that which is connection to the
address given. If it isn't, then it should ignore it. Alternate-protocol as
it exists today assumes failure for the alternate-protocol until the
channel is actually setup, at which point the browser can proceed to use
that channel and possible remember this for future communications.

> I'm inclined to believe this could be semi-common for internal sites, but
it seems implausible that, as a percentage, there are that many DNS
disobeyers out there.

> your browser automatically try 2.0 there without any DNS request.

Well, I'd rather have the browser makers figure out the most appropriate
heuristic for deciding which to attempt to speak first, and be assured that
there is a fast-fail fallback to ensure that when it chooses wrong, it
downgrades within my attention span. IPv6 got this wrong, but since we've
hopefully learned from that deployment mistake...

> mechanism is valid, there is no need for help from side-band protocols.

What percentage of the world is in a situation where they don't have to do
DNS resolutions? This seems like optimization for the few instead of for
the many.
In cases where a user-agent (whether that be a proxy or a browser) has to
do a DNS resolution, having information that it speaks HTTP/2 or whatever
seems only beneficial provided such information is used only when the
user-agent is *actually* making a connection to that IP.

-=R

---

From: Eliot Lear <[email-masked]>
Date: Tue, 21 Aug 2012 09:21:14 +0200

James,

IMHO upgrade discovery is an absolute requirement for anything using the
http: scheme.  Otherwise how do you get to 2.0?  Also, nobody has
proposed an alternate scheme, and I suspect such a proposal would be
contentious, to say the least.

Eliot

On 8/21/12 6:59 AM, James M Snell wrote:
> So, I've been catching up on this thread and there's just one thing
> that I just want to clarify: the upgrade mechanism that's being
> discussed is not going to be a mandatory component of HTTP/2.0,
> correct? That is, I as an implementor am not going to be required to
> do this upgrade dance with every new connection, regardless of how it
> actually happens... right? I'm just saying because there are a vast
> array of use cases where such a dance is going to be wholly
> unnecessary... I would even argue that, in MANY cases, having
> something as simple as http2://www.example.com
> <http://www.example.com> that operates on a different default port is
> going to be more than sufficient if the environment is fairly
> controlled and predictable on both ends. Throughout this discussion,
> there seems to be an implicit assumption that upgrade discoverability
> is an absolute requirement.
>
> - James
>
> On Mon, Aug 20, 2012 at 9:23 PM, Amos Jeffries <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>         On Mon, Aug 20, 2012 at 2:27 PM, Willy Tarreau <[email-masked]
>         <mailto:[email-masked]>> wrote:
>
>             > > The DNS requires a round trip anyway and it currently
>             where time is
>             wasted
>             > > in mobile environments.
>             > >
>             >
>     Since a failed connection attempt is a full TCP handshake
>     connection. Doing one failure for HTTP/2 and a second attempt for
>     HTTP/1 would be worst-case around 45 seconds to *start* the
>     request process.
>
>
>
>
>             these entries might have been advertised in HTTP response
>             headers as you
>             once
>             explained to me :-)
>
>
>     there. Depending on those capability advertisements when available
>     would be a costly mistake.
>
>
>
>
>
>             > > Upgrade ? However, I'd hope we avoid announcing IP and
>             ports in
>             exchanges,
>             > > as it will cause the NAT mess we managed to avoid for
>             years with HTTP
>             and
>             > I'm not understanding how this would cause NAT messes?
>             I'd guess that
>             most
>             There should be a single reliable way of getting a host IP
>             and a port.
>             Right
>     Swap "host" with "recipient" and you have the requirements lined
>     up correctly. Final end-server in the chain does not matter in the
>     connected to is all that matters. Any overlap between the two is
>     just a convenience.
>
>
>
>
>             At the moment you start exchanging IP addresses in the
>             protocol, you're
>             getting
>             into trouble. Sometimes the IP is valid, sometimes it
>             doesn't mean
>             anything and
>             must be rewritten by *someone* along the chain, but most
>             often, this
>             *someone*
>             doesn't care or is not the same as the one which
>             translates the address. I
>             guess
>             everyone here has already been involved in helping any
>             admin to rewrite
>             Location
>             or as a next hop for a reverse-proxy, we shouldn't care
>             how that address
>             was
>     *sometimes*. There is a lack of MUT NOT clause in the specs to
>     prevent clients doing for example; same-origin validation.
>
>
>     considering unless they can provide the same guarantee for
>     handling transit over middleware (both explicit and interceptors)
>     safely.
>
>     SRV is a non-starter due to end-host servers not being aware of
>     the details to emit for intermediaries.
>
>     I think we should be taking Upgrade: and defining ways to optimize
>     it. Willy had some proposals for merging multiple requests and
>     defining 1.1->2.0 mapping in the headers of request #1. Figuring
>     out how much of that is useful and where it could be improved
>     would be a good way to progress the bootstrapping.
>
>
>
>
>             > Some internal sites
>             > wouldn't, and the network there is controlled well
>             enough that this
>             should
>             > cause no problem. In any case, sites are incented to be
>             sure that users
>             can
>             > connect to them, and will quickly decide to simply keep
>             using :80 and
>             :443.
>             > When one has an inspecting intermediary, it probably
>             won't like to see
>             the
>             above. It could be mandated that if an http connection to
>             :80 fails, a
>             browser
>        - long term plans to obsolete port 443 usage and make the
>     http:// scheme TLS-enabled/secure whenever possible.
>
>
>     Amos
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 21 Aug 2012 10:38:41 +0200

On Mon, Aug 20, 2012 at 11:08:02PM -0700, Roberto Peon wrote:
> > That's why operators are rewriting/inlining contents with IP

Unfortunately yes, as ugly and disgusting as it is, I have seen it with my own
eyes :-( It even has the side effect of having the browser not send cookies to
fetch images, which can be desirable sometimes but might break other times (eg
when trying to load an avatar maybe).

> first RTT.

The first one is huge but next ones are still important due to shaping and
massive queuing on the RAN.

> Otherwise, sure, DNS resolutions can take some time, around
> 100-200ms.

Exactly.

> including the protocol version to attempt to speak.

It can be done that way indeed.

> was trying to say earlier...

I see your point, but again, I don't see this as much valuable if the
upgrade can be safely done without relying on the DNS at all.

> got the protocol version wrong.

I see it similarly, because the need is how to ensure that either the upgrade
succeeds or we fallback to the alternative method.

> able to recover withing the attention-span of the user and remember...

Quite frankly I don't see this ever changing on port 80. We'll more likely
see more and more sites advertise "https://" URIs and browsers will probably
try "https://" by default when an URI is entered. Then the http scheme might
remain port-80 compliant for all sites which do not need https and it will
not be much of a trouble.

> the next protocol (via upgrade header), but it is nitpicking.

Yes if you want :-)

> and refers to an internal IP.

Not only. "Public" servers are often reached from the inside on another IP
address by admins, by monitoring systems or even by other users coming from
other networks. That's quite common. In fact the "public" server often is
just a public view of an internal server with a layer of reverse-proxy in
front of it.

> Generally, publicly reachable servers need to

Yes I agree.

> that channel and possible remember this for future communications.

OK.

> it seems implausible that, as a percentage, there are that many DNS

In general you don't want to make DNS requests for a known next-hop, it is
expensive for no added value. This is why you see so many hard-coded addresses
or /etc/hosts on server-side reverse-proxies and load-balancers. One could say
that such components could be configured differently to have the HTTP version
along with the address, but then again it means we have two referentials for
the version, one part in those elements' config files and another part in the
DNS for outsiders.

> downgrades within my attention span.

Totally agreed.

> IPv6 got this wrong,

Precisely and IPv6 relied on the DNS to make the transition, which really
did not help at the beginning, to say the least!

> but since we've hopefully learned from that deployment mistake...

Hehe :-)

> DNS resolutions? This seems like optimization for the few instead of for

In my opinion it's just on server-side, and internal networks. It's not a
big deal, but it creates a number of dirty situations which are generally
solved the worst way. If we can avoid this it's preferable in my opinion.

> user-agent is *actually* making a connection to that IP.

Another point is that DNS makes it difficult to manage multiple services by
multiple entities on the same name. But then again it's mainly an internal
issue only.

Regards,
Willy

---

From: Patrick McManus <[email-masked]>
Date: Tue, 21 Aug 2012 08:30:51 -0400

Hi James,

On Mon, 2012-08-20 at 21:59 -0700, James M Snell wrote:
> That is, I as an implementor am not going to be required to do this
> upgrade dance with every new connection, regardless of how it actually
> happens... right?

If we define something on port 80 that uses Http Upgrade, then likely
you will - like websockets does. That's an argument against http
upgrade. I would like to think some mechanism that lets you explicitly
know that http/2 is in use on another port would be preferable and let
you skip that dance once discovery is complete (i.e. dns srv and/or
alternate-protocol). I don't think sending http/2 on port 80 without
upgrade would ever be viable due to proxies etc..

>  http2://www.example.com that operates on a different default port is 

a new scheme is out of scope of the proposed charter as far as I
understand it as the charter expects HTTP/2 to retain the existing URI
semantics of HTTP/1.1.

---

From: Nicolas Mailhot <[email-masked]>
Date: Tue, 21 Aug 2012 15:25:03 +0000 (UTC)

Yoav Nir <ynir@...> writes:

> A related but orthogonal issue is finding out if the path between the client
> and the server supports HTTP/2.0. Fortunately, middleware does not tend to
> appear mid-path.

That depends what you call "mid-path". Middleware appears at network boundaries.
VPNs now enable easy stacking of multiple private networks, and the boundary the
middleware is deployed on can end up quite a long path away from the client.

> Client side middleware is either there or it's not. It's not likely to be
> present on-path to one generally available website while not being present
> for another.

If things were that simple no one would use PAC files. Their whole point is that
middleware is not a simple binary setting.

---

From: James M Snell <[email-masked]>
Date: Tue, 21 Aug 2012 09:42:27 -0700

While I completely understand (and agree) with the various arguments in
favor of reusing port 80 for general HTTP traffic over existing Internet
infrastructure, there are countless perfectly valid cases where such
discovery simply is not even remotely necessary. My concern is just in
making sure that I won't have to do a bootstrap dance on every connection
that I make. Perhaps a http2 URL scheme would be appropriate as an
orthogonal component to the work on the basic 2.0 protocol; and certainly
whatever the bootstrapping/upgrade mechanism ends up being, it should be
entirely optional on the part of the client... especially in cases where I
know in advance that it's 2.0 all the way down the line.

- James

On Tue, Aug 21, 2012 at 12:21 AM, Eliot Lear <[email-masked]> wrote:

> http: scheme.  Otherwise how do you get to 2.0?  Also, nobody has proposed
> an alternate scheme, and I suspect such a proposal would be contentious, to
> say the least.
>
> Eliot
>
>
> On Mon, Aug 20, 2012 at 9:23 PM, Amos Jeffries <[email-masked]>wrote:
>