Subject: Make HTTP 2.0 message/transport format agnostic

From: Kevin Cathcart <[email-masked]>
Date: Fri, 30 Mar 2012 00:46:37 +0000

I'm not really sure why everybody is getting so worked up about SPDY.
The correct thing to do is pretty obvious
to me. Document the core HTTP protocol in a message format agnostic
way. This means not specifying
the format of the messages or the transport.

Instead the core HTTP spec would be defined in terms of two types of
abstract messages. One is a request
message, which includes the following components: a version number, a
resource URI, a method, headers,
and an optional body. The other message is a response, which includes
a version, a status code, a status
phrase, headers, and an optional body. The core spec would then define
the meaning of the methods and
headers, the rules regarding proxies and caching, and the
authentication rules, etc. Basically everything
that would apply to all message/transport formats.

Separate specifications would define message/transport formats.

Headers that are specific to the textual Internet Message
message/transport format currently used in
HTTP/1.1 would not be included in the core specification but would be
part of a separate specification
that would define the textual format for HTTP 2.0 (unless of course we
decide not to support such a
format for HTTP 2.0).

SPDY could pretty trivially be such a message/transport format,
requiring only slight tweaks to its specification
like specifying that it presents its ":host" pseduo-header to as a
Host header in the abstract message.

Similarly we could define a S+M based message/transport format, or
perhaps even a hybrid of the the two, whatever
is deemed best.

Kevin Cathcart

---

From: Roberto Peon <[email-masked]>
Date: Fri, 30 Mar 2012 19:35:10 +0200

Kevin--

You've described part of what SPDY already does and something which is
certainly a requirement for anything that meets HTTP's base semantics.
Unfortunately, you need to carry more than just the information you
provided if you wish to use the rest of HTTP's semantics such as auth (even
the crappy auth we currently have), cookies, caching, compression, etc.

And then there are the features that improve speed, like multiplexing, and
compression... The Chrome and FF code is open, as is the code for many
server implementations (including a number of 'toy' servers that are
perhaps easier to hack on).

Much of our experience with SPDY was hard fought and hard learned by
attempting, failing, and learning from the failure.

-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 30 Mar 2012 22:55:12 +0000

In message <[email-masked]>
, Kevin Cathcart writes:

>way.
>[...]

Seconded.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: tom <[email-masked]>
Date: Sat, 31 Mar 2012 11:08:48 +0800

Separate HTTP message format and under-hood transport as two specs should
earn expansibility and portability.

Actually, Web app runs above HTTP and don't care what's transport to use.
And, the different transport can provide the specific benefits.
For example, UDP is easy to setup P2P communication, TCP is for
client/server communication and SPDY for multiplexing connection.

Best regards
  Tom

On Sat, Mar 31, 2012 at 6:55 AM, Poul-Henning Kamp <[email-masked]>wrote:

> In message <
> [email-masked]>
> , Kevin Cathcart writes:
>

---

From: Roberto Peon <[email-masked]>
Date: Sat, 31 Mar 2012 06:23:27 +0200

So, we are looking to define three layers: the HTTP compatible layer on top
of a "stream" management layer on top of a transport?

Is that what you're wanting?

---

From: Mike Belshe <[email-masked]>
Date: Sat, 31 Mar 2012 06:23:32 +0200

> earn expansibility and portability.

The SPDY Spec is here:
http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00

>From the spec:
    Section 2: SPDY Framing
Layer<http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00#section-2>
    Section 3: HTTP Layering over
SPDY<http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00#section-3>

The reason we didn't separate them so far is because we didn't have any
alternate messaging layers in mind.  Further, it wasn't a goal to make the
framing layer generic.  That could be done.  I'm personally not a supporter
of designing for things you don't know, which is why I vote to keep them
together.  If someone has a specific purpose in mind for the framing layer,
we could talk about this.

But hopefully the current separation allows you to accomplish what you want
until we have some other specific purpose in mind.

Mike

---

From: tom <[email-masked]>
Date: Sat, 31 Mar 2012 14:00:22 +0800

Personally, Keep HTTP less dependence on transport is not bad idea at
least. A middle framing/session layer can glue HTTP message format and
transport layer well, just like what SPDY to do. But, SPDY don't abstract
what transport it rely on. if SPDY defines more portable under-hood
transport to rely on, it is better.

Best regards
  Tom

On Sat, Mar 31, 2012 at 12:23 PM, Mike Belshe <[email-masked]> wrote:

>     Section 2: SPDY Framing Layer<http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00#section-2>
>     Section 3: HTTP Layering over SPDY<http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00#section-3>
>
> But hopefully the current separation allows you to accomplish what you
> want until we have some other specific purpose in mind.
>
> Mike
>
>
>
>>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 31 Mar 2012 06:52:04 +0000

In message <[email-masked]>
, Mike Belshe writes:

>of designing for things you don't know, [...]

I'll be happy to do the design for high performance delivery,
something which SPDY clearly isn't attempting to tackle.

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 31 Mar 2012 11:53:08 +0200

Kevin (et al),

> that would apply to all message/transport formats.

This is effectively what HTTPbis p2 - p7 already do.

> Separate specifications would define message/transport formats.

We're chartered to come up with one of these and call it "HTTP/2.0".

> perhaps even a hybrid of the the two, whatever

Some people seem to be arguing for multiple serialisations of HTTP from the start. Since the value of a standard is largely in interop and market forces, I'd strongly suggest that we not assume this until we have proven and agreed to it being necessary. 

I.e., just because SPDY (or S+M, or any other proposal) isn't good as-is right now does not automatically mean that we need two (or more) serialisations. We need to discuss our requirements and the proposals that emerge, so we can choose an appropriate path forward forward. If we end up in a corner where we can't serve all of our requirements from one, *then* we can open this box.

Regards,

--
Mark Nottingham
http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 12:06:59 +0200

Hi Mark,

just one question to clarify one point below :

On Sat, Mar 31, 2012 at 11:53:08AM +0200, Mark Nottingham wrote:
> Some people seem to be arguing for multiple serialisations of HTTP from the
> start. Since the value of a standard is largely in interop and market forces,
> I'd strongly suggest that we not assume this until we have proven and agreed
> to it being necessary. 
> 
> I.e., just because SPDY (or S+M, or any other proposal) isn't good as-is
> right now does not automatically mean that we need two (or more)
> serialisations. We need to discuss our requirements and the proposals that
> emerge, so we can choose an appropriate path forward forward. If we end up in
> a corner where we can't serve all of our requirements from one, *then* we can
> open this box.

When you say "serialization", you seem to imply the on-wire format, while
for me (and possibly for others) serialization is what the stream looks
like. Right now HTTP/1.1 is serialized over multiple streaming protocols
(TCPv4/v6, SSL/TLS over these ones, unix sockets), with the {clear,SSL/TLS}
over TCP* combinations being more common than anything else and the standard
ones. Could you please clarify this point so that there is no ambiguity ?

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 31 Mar 2012 12:09:01 +0200

> ones. Could you please clarify this point so that there is no ambiguity ?

I'm just contorting language to avoid using the term "binding." "On-wire format" is more verbose, but clear; I'll try to use it from here.

Cheers,

--
Mark Nottingham
http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 12:16:36 +0200

On Sat, Mar 31, 2012 at 12:09:01PM +0200, Mark Nottingham wrote:
> I'm just contorting language to avoid using the term "binding." "On-wire
> format" is more verbose, but clear; I'll try to use it from here.

OK thanks for this precision.
(in fact in case of TCP, it's more packetization than serialization BTW).

Cheers,
Willy

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sat, 31 Mar 2012 10:36:31 +0000

------ Original Message ------
From: "Mark Nottingham" <[email-masked]>
>Some people seem to be arguing for multiple serialisations of HTTP from the start. Since the value of a standard is largely in interop and market forces, I'd strongly suggest that we not assume this…
  
I agree.
  
We should be able to learn from experience here... ASN.1 encoding 
rules... have any become dominant?
  
Adrien

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 31 Mar 2012 10:56:29 +0000

In message <em0bd24bf1-0df5-4eb2-aba4-7ca1e81c7ade@boist>, "Adrien W. de Croy" 

>rules... have any become dominant?

Have any ever broken the 1Gbit/s barrier ?

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sat, 31 Mar 2012 11:30:43 +0000

------ Original Message ------
From: "Poul-Henning Kamp" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Mark Nottingham" <[email-masked]>;"Kevin Cathcart" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 31/03/2012 11:56:29 p.m.
Subject: Re: Make HTTP 2.0 message/transport format agnostic
>Have any ever broken the 1Gbit/s barrier ?
no idea.  What I meant was dominant amongst themselves... e.g. one 
preferred encoding (like DER) rather than the other 6 or 7.
  
Adrien

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 31 Mar 2012 12:14:29 +0000

In message <em8582f3b4-bdc2-4f0b-ad2f-0dccfd9729fb@boist>, "Adrien W. de Croy" 

>preferred encoding (like DER) rather than the other 6 or 7.

I don't care how dominant, if it doesn't do at least 40Gbit/sec with
less than 10% of a contemporary machine, it's not relevant.

---

From: Kevin Cathcart <[email-masked]>
Date: Sat, 31 Mar 2012 12:07:42 +0000

> This is effectively what HTTPbis p2 - p7 already do.

Good. Lets just make sure that p2-p7 truly have no hacks
or cruft remaining from the current synchronous textual
Internet Message over TCP message-format/transport.

> We're chartered to come up with one of these and call it "HTTP/2.0".

If the idea is that with HTTP/2.0 we are simply writing a replacement
for part 1, and perhaps one or more additional parts that may specify
new features that rely on framing/message format of the new part 1,
then that is great!

> Some people seem to be arguing for multiple serialisations
>of HTTP from the start. Since the value of a standard is largely
>in interop and market forces, I'd strongly suggest that we not
>assume this until we have proven and agreed to it being necessary.

Except that by my understanding there will inherently be two
serializations, the old HTTP/1.1 serialization, and the new
HTTP/2.0 one.

I think you mean that we should not consider multiple new
implementations. That is fine to me if my understanding
is correct, and that even once HTTP/2.0 is out it
remains possible for new methods/headers to be defined
in a way that both HTTP/1.1 and HTTP/2.0
(along with any other non-standard serialization that
is built on p2-p7) are supported.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sat, 31 Mar 2012 23:36:45 +0000

------ Original Message ------
>less than 10% of a contemporary machine, it's not relevant.
  
that's not the point.
  
I'm trying to see if there is any knowledge out there about whether 
there is any benefit to having multiple encodings or not, whether 
experience has shown that it was a pointless exercise or not.
  
So throughput is meaningless.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sat, 31 Mar 2012 23:53:16 +0000

p.s.
  
I wasn't for a second suggesting we should actually _use_ ASN.1 or any 
of its encodings.
  
It's just the only other system I know of which uses an abstract 
notation, and has numerous transfer encodings, so I wondered if any 
people knew whether that turned out to just be a waste of time because 
everyone just ended up using one of them.

------ Original Message ------
From: "Adrien W. de Croy" <[email-masked]>
To: "Poul-Henning Kamp" <[email-masked]>
Cc: "Mark Nottingham" <[email-masked]>;"Kevin Cathcart" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 1/04/2012 11:36:45 a.m.
Subject: Re: Make HTTP 2.0 message/transport format agnostic

---

From: Tim Bray <[email-masked]>
Date: Sat, 31 Mar 2012 17:44:06 -0700

Funny you should mention that.  ASN.1 has historically been a
miserable failure; the vast majority of net traffic today is a media
format (jpg/mp4/whatever) or text encoded in HTML/JSON/XML, in most
cases wrapped in an HTTP envelope.  None of these (media formats,
markup language, HTTP) have an underlying “abstract model”;
interoperability is achieved at the level of message syntax and
interchange patterns.

Which is to say, syntax-based-interoperability wins, and attempts to
have an abstract model with a secondary syntax is a trail of tears.
HTTP2.0 is a little unusual in that the operational semantics and
payload-carrying capabilities are predefined, but at the end of the
day it’s a syntax/message-interchange design problem; attempts to do
it at a higher level of abstraction are a waste of time -T

On Sat, Mar 31, 2012 at 4:53 PM, Adrien W. de Croy <[email-masked]> wrote:
>  p.s.
>  I wasn't for a second suggesting we should actually _use_ ASN.1 or any of
> its encodings.
>  It's just the only other system I know of which uses an abstract notation,
> and has numerous transfer encodings, so I wondered if any people knew
> whether that turned out to just be a waste of time because everyone just
> ended up using one of them.
>
>
>> ------ Original Message ------ From: "Poul-Henning Kamp"
>> <[email-masked]>
>>>
>>> In message <em8582f3b4-bdc2-4f0b-ad2f-0dccfd9729fb@boist>, "Adrien W. de
>>> Croy" writes:
>>>
>>>>>>
>>> I don't care how dominant, if it doesn't do at least 40Gbit/sec with less
>>> than 10% of a contemporary machine, it's not relevant.
>>
>>
>> that's not the point.
>> I'm trying to see if there is any knowledge out there about whether there
>> is any benefit to having multiple encodings or not, whether experience has
>> shown that it was a pointless exercise or not.
>>> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [email-masked] | TCP/IP
>>> since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to
>>> malice what can adequately be explained by incompetence.
>>>
>>
>>
>
>