Subject: The HTTP vs SPDY debate

From: Willy Tarreau <[email-masked]>
Date: Thu, 29 Mar 2012 14:22:25 +0200

Hello,

after seeing all the disagreements that were expressed on the list these
days (including from me) about what feature from SPDY we'd like to have
mandatory or not in HTTP, I'm thinking that part of the issue comes from
the fact that there are a number of different usages of HTTP right now,
all of them fairly legitimate.

First I think that everyone here agrees that something needs to be done
to improve end user experience especially in the mobile networks. And
this is reflected by all proposals, including the http-ng draft from
14 years ago!

Second, the privacy issues are a mess because we try to address a social
problem by technical means. It's impossible to decide on a protocol if
we all give an example of what we'd like to protect and what we'd prefer
not to protect because it is useless and possibly counter-productive.

And precisely, some of the disagreement comes from the fact that we're
trying to see these impacts on the infrastructure we know today, which
would obviously be a total breakage. As PHK said it, a number of sites
will not want to afford crypto for privacy. I too know some sites which
would significantly increase their operating costs by doing so. But
what we're designing is not for now but for tomorrow.

What I think is that anyway we need a smooth upgrade path from current
HTTP/1.1 infrastructure and what will constitute the web tomorrow without
making any bigbang.

SPDY specifically addresses issues observed between the browser and the
server-side infrastructure. Some of its mandatory features are probably
not desirable past the server-side frontend *right now* (basically
whatever addresses latency and privacy concerns). Still, it would be
too bad not to make the server side infrastructure benefit from a good
lifting by progressively migrating from 1.1 to 2.0.

What does this mean ? Simply that we have to consider HTTP/2.0 as a
subset of SPDY or that SPDY should be an add-on to HTTP. And that
makes a lot of sense. First, SPDY already is an optimized messaging
alternative to HTTP. It carries HTTP/1.1, it can as well carry HTTP/2.0
since we're supposed to maintain compatible semantics.

We could then get to a point where :
  - an http:// scheme indicates a connection to HTTP/1.x or 2.x server
  - an https:// scheme indicates a connection to HTTP/1.x or 2.x server
    via an SSL/TLS layer
  - a spdy:// scheme indicates a connection to HTTP/1.x or 2.x server
    via a SPDY layer

By having HTTP/2.0 upgradable from 1.1, this split is natural :

        +----------------------------+
        |       Application          |
        +----+-----------------------+
        | WS |     HTTP/2.0          |
        +----+--------------+        |
        |      HTTP/1.1     |        |
        |         +-----+---+--------+
        |         | TLS | SPDY       |
        +---------+-----+------------+   server-side
            ^        ^        ^  
            |        |        |  
            |        |        |  
            |        |        |  
        +---------+-----+------------+  user-agent
        |         | TLS | SPDY       |
        |         +-----+-------+----+
        |  HTTP/1.1, 2.0        |    |
        +-------------------+---+    |
        |                   |   WS   |
        |  Applications     +--------+
        |                            |
        +----------------------------+

The upgrade path would then be much easier :

  1) have browsers, intermediaries and servers progressively
     adopt HTTP/2.0 and support a seamless upgrade

  2) have browsers, some intermediaries and some servers
     progressively adopt SPDY for the front-line

  3) have a lot of web sites offer URLs as spdy:// instead of http://,
     and implement mandatory redirects from http:// to spdy:// like a
     few sites are currently doing (eg: twitter)

  4) have browsers at some point use the SPDY as the default scheme
     for any domain name typed on the URL bar.

  5) have browsers at one point disable by default transparent support
     for the old http:// scheme (eg: put a warning or have to tweak
     some settings for this). This will probably 10-20 years from now.

Before we get to point 5, we'd have a number of sites running on the
new protocol, with an efficient HTTP/2.0 deployed at many places
including the backoffice, and with SPDY used by web browsers for
improved performance/privacy. That will not prevent specific agents
from still only using a simpler HTTP/2.0 for some uses.

So I think that what we should do is to distinguish between what is
really desirable to have in HTTP and what is contentious. Everything
which increases costs or causes trouble for *some* use cases should
not be mandatory in HTTP but would be in the SPDY layer (as it is
today BTW).

I think that the current SPDY+HTTP mix has shown that the two protocols
are complementary and can be efficient together. Still we can significantly
improve HTTP to make both benefit from this, starting with the backoffice
infrastructure where most of the requests lie.

Willy

---

From: Karl Dubost <[email-masked]>
Date: Thu, 29 Mar 2012 08:44:19 -0400

>  - a spdy:// scheme indicates a connection to HTTP/1.x or 2.x server

Usually with 

    <img src="//example.org/foo.png" alt=""/>

you can create content which is easily interact with http and https 
without headaches (except for IE I think). It creates a nicer transition 
path from http to https.

I wonder if this would work with SPDY too.

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations, Opera Software

---

From: Julian Reschke <[email-masked]>
Date: Thu, 29 Mar 2012 14:58:06 +0200

On 2012-03-29 14:44, Karl Dubost wrote:
 > ...

My understanding was that this works cross-browser (as it should).

Do you have a test case?

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 29 Mar 2012 15:00:39 +0200

> I wonder if this would work with SPDY too.

I was not aware of this trick, this is very useful !
I think that at one point we'll get pages which only display contents
using the same scheme for the whole page, but I may be wrong.

Willy

---

From: Mike Belshe <[email-masked]>
Date: Thu, 29 Mar 2012 15:26:44 +0200

I thought the goal was to figure out HTTP/2.0; I hope that the goals of
SPDY are in-line with the goals of HTTP/2.0, and that ultimately SPDY just
goes away.

Mike

---

From: Salam J <[email-masked]>
Date: Thu, 29 Mar 2012 23:53:39 +0430

On Thu, Mar 29, 2012 at 5:14 PM, Karl Dubost <[email-masked]> wrote:
> It is supported in IE with some *headaches* when used in HTTPS; see
http://paulirish.com/2010/the-protocol-relative-url/

---

From: Henry Story <[email-masked]>
Date: Fri, 30 Mar 2012 00:14:22 +0200

Hi Will,

your proposal sounds good to me. 

I should say I am a complete novice in SPDY and HTTP2.0 still. But I will be very
interested to try to get the WebID authentication 'protocol' [1] working 
with SPDY. (I put protocol in quotes, because it is such a simple protocol,
that really I even hesitate to call it a protocol: it's just using TLS the
way it was meant to be used, if you look at the history of X500 and directories.
We just use URIs instead of ldap names)

WebID enables authentication for distributed social networks. It works with HTTP1.0
and 10 year old browsers. It should work out of the box for SPDY too. 

I am in Paris so if people would like to chat about this a bit more to see
how this can tie into the authentication stack, I'd be very happy to exchange
ideas.

 Henry

[1] video and links on http://webid.info/
    spec at http://www.w3.org/2005/Incubator/webid/spec/
    community http://www.w3.org/community/webid/

> By having HTTP/2.0 upgradable from 1.1, this split is natural :

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 30 Mar 2012 09:07:57 +0200

Hi Mike,

> SPDY are in-line with the goals of HTTP/2.0, and that ultimately SPDY just

Of course, but since there are obviously controversial points which some
absolutely want and others really don't want, I think that having SPDY be
the bleeding edge "next" HTTP version on which research is performed etc
and having the "stable" HTTP version makes a lot of sense. After all,
this is more or less what is happening today : you present some results
with running code and deployments, you're still open to make some changes,
and both you and Roberto also told me you have a fair number of exciting
new features in mind for the future. So why not consider keeping HTTP and
SPDY work side by side with HTTP getting backports from SPDY every few
years ? Even if SPDY became HTTP2 tomorrow, at the end of the year you
would not be satisfied because you'd have many new features that you would
like to have again and you'd reopen the SPDY project !

There will always be users and websites who like to live on the bleeding
edge and running your latest spec because they benefit from it. Standards
benefit from experimental results and must not frequently change. Having
both live together would make each one benefit from the other. I think
you will agree with me that in the future, changes to the standard will
become more and more frequent in order to better address end users needs,
and at the same time we need to maintain some stability for the server
side, with a compatibility layer between the two. There's probably an
emerging market for latest-spdy-to-stable-http gateways to allow people
to live on the bleeding edge.

Last point is that by making such a relation something publicly known,
you can more easily make implementers aware of future requirements, and
ask their vendors for compliance with the forthcoming new standard. It's
just like in 1999 we had many vendors writing "Y2K compliant" on anything
ranging from cars to spoons. But at least customers wanted to be sure.

Really I think it makes sense to have stable and bleeding edge.

We can discuss about this today if you like.

Cheers,
Willy

---

From: Mike Belshe <[email-masked]>
Date: Fri, 30 Mar 2012 15:15:12 +0200

You're welcome to ask the browser implementors, but a new protocol
namespace is really a non-starter.  We can't kick protocol versioning out
to the users for sorting out.

>From a protocol specification perspective, SPDY does not mandate SSL.

>From a browser perspective, I don't think the browsers are going to be keen
on having multiple, experimental HTTP/2.0 or SPDY specs implemented
concurrently.  It's just a coding mess.

So we probably have to pick just one implementation per piece of software.
 As for what people implement, they should try the variants they think make
sense, and then come back to the group with information about what they
learned.

Mike

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 30 Mar 2012 16:49:14 +0200

Hi Mike,

> concurrently.  It's just a coding mess.

This is exactly what they've been doing and both Chrome and Mozilla have
SPDY right now. I'm not talking about having *multiple* experimental specs,
just one stable standard and one experimental from which the next standard
will be built. This is exactly what is happening right now, and you know too
well that it will not stop here !

> sense, and then come back to the group with information about what they

Agreed.

Cheers,
Willy

---

From: Mike Belshe <[email-masked]>
Date: Sat, 31 Mar 2012 00:32:04 +0200

> well that it will not stop here !

I'm pretty familiar with what the browsers do, and they don't do this :-)
 So I must be misunderstanding what you mean.

If you're talking about how a feature goes from dev channel to beta channel
to release channel in chrome,  thats true.  But its not used the way I
thought you were asking it to be used - features don't stay locked at one
level - they either get rolled into the next version (which is ~3 weeks
apart!) or they get kicked out.  In other words, all features are always on
path to get shipped within ~10weeks.

Mike

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 08:58:20 +0200

Hi Mike,

On Sat, Mar 31, 2012 at 12:32:04AM +0200, Mike Belshe wrote:
> > I'm not talking about having *multiple* experimental specs,
> > will be built. This is exactly what is happening right now, and you know
> > too well that it will not stop here !
> >
> 

OK, I'll try to explain below (long version). Too bad we didn't have more
time to discuss this on thursday!

> path to get shipped within ~10weeks.

Interesting, but I'm not talking about having features in experimental
browser versions but experimental protocols in browsers. Right now, SPDY
is evolving. Maybe you'll say that it has reached a stable point, but
anyway given all the things we discussed about on Thursday, I'm pretty
sure that you won't stop there, and that new versions will appear and
will be adopted by browsers :-)

So at the moment we have HTTP as the legacy protocol and SPDY as a new
protocol, adopted by some websites and some browsers willing to accept
a few changes if/when the protocol becomes the new legacy protocol.

What this means to me is that it's more or less a matter of "regularly"
(I mean every few years maybe) rebasing HTTP on the latest stable SPDY
spec to ease integration of new features.

What I'm observing is that right now there are two maint points in SPDY :
  - improve HTTP and work around a number of its current shortcomings ;
  - improve/secure the transport layer

The first point is a strong one which all 4 proposals identified as a
"must be fixed". The second point is probably where SPDY is the most
advanced at the moment eventhough as you suggested, it clearly targets
end users and does not necessarily bring any specific value to webservices
or such usages.

At the moment, SPDY is more or less HTTP/1.1 + MUX + an improved transport
layer. And I have a number of reasons to think that the transport layer
will be the place which will get the most changes in the future due to
its current limitations in mobile environments, and because of the way
mobile applications are evolving (less web, more json).

So what I'm saying is that we really need two things for HTTP :

   - fix some of the 1.1 design issues that we all hate (metadata weight,
     parsing complexity, in-order delivery, etc...). We need to address
     these issues at once for a long time (probably as long as 1.1 has
     lived) so that 2.0 can become the legacy protocol for everything as
     1.1 currently is ;

   - define a way to improve transport for a number of evolving usages
     and to be able to easily plug new transport mechanisms that are
     being experimented (which might include registering new schemes
     or field values to exchange information between layers).

The first point is in line with the WG's charter. The second point is
around all the research you guys have been (and are) doing. At the moment,
we know that people are using HTTP over a few different transport layers
and that it works pretty well, meaning that HTTP currently is transport
agnostic (SSL/TLS is often offloaded by front gateways, UNIX-stream is
regularly used for inter-process communication in the same host, and UDP
is used as a transport layer in UPNP).

By keeping the HTTP core independant of the transport layer, we can
regularly bring new transport combinations on the table, and if some of
them require minor improvements to the protocol, these changes can lead
to a new revision of the protocol. Want HTTP over TLS over TCP ? trivial.
Want HTTP over DTLS over UDP ? Some adaptations might be needed but at
least we'd have the HTTP core ready for this. Want HTTP over two different
channels for people using satellite links so that meta-data passes over
the wire and entities pass over the air ? Create a new transport layer.
Want HTTP to pass over a new to-be-defined streaming secure transport
protocol ? Just change the lower layer again.

In fact, even the flow control is transport-specific. Seeing how many
flow control algorithms there are for TCP depending on whether you use
a slow link, a long fat pipe, a big local pipe etc... it would make
sense to have the transport layer take care of this and take the latency
and underlying buffers into account.

The transport layer is where I expect a lot of activity in the coming
years, and having a stable and efficient protocol base for this would
make experimentations and deployments very easy by the simple addition
of transport layer gateways. Then maybe we can standarize a new transport
layer every year or two and expect them to be quickly adopted because it's
cheap and provides real benefit. For instance, let's say that a company
develops a WiFi-to-WiMax gateway to provide connectivity to small villages.
Users will probably use TLS+TCP to the local gateway and the gateway might
at one point prefer to use a new layer more suited to WiMax than what we
have. Then it's important to make them able to standardize what they're
using and that it's transparent to the end user.

But I also know that some improvements will be transport-independant and
will be worth backporting into the spec (server push maybe, or new auth
schemes). So maybe 1 year after 2.0 will be out, a number of shiny new
features will be ready and new transport protocols will require minor
adaptations and it will be time to issue 2.1 to benefit from all of this
in the standard protocol.

Regards,
Willy