Subject: HTTP/2.0 Magic

From: Martin Thomson <[email-masked]>
Date: Fri, 1 Feb 2013 10:24:30 +0900

The conclusion that we reached in the interim was that no matter how
HTTP/2.0 was started, there would be some magic that started the
session.

The requirements for that magic is that it is designed to cause a
reasonable proportion of HTTP/1.1 implementations to fail, preferably
to close the connection.

This magic also provides a high degree of confidence that the protocol
you are talking is actually HTTP/2.0 and not something else.

As far as I am aware, the actual sequence does not matter much, though
having the first bit set ensures that this isn't valid HTTP/1.1.

I generated a random number.  In this case, a 32-bit value.  Happily,
the high bit is set:

  e1c54784

As we discussed, this would be sent at the start of every session and
be followed immediately by a SETTINGS frame.  Both client and server
send this sequence.

The concern here is that some implementations will swallow this and
proceed anyway.  Those implementations wont fail as a result of seeing
this.  It may be the case that for those implementations no amount of
magic is sufficient as the tests that lead to websockets masking
revealed.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 31 Jan 2013 18:35:24 -0800

How about: d3 d0 c4 d9? (as an hommage).

Honestly, though, anything which spells out something easy to remember in
7-bit ascii when the high-bits are all xor'd would be nice :)

-=R

---

From: Eliot Lear <[email-masked]>
Date: Fri, 01 Feb 2013 12:04:49 +0900

Separately we will also need a version identifier.  This field can ALSO
server as a version identifier.  We just rev/change the magic on a new
version.

On 2/1/13 11:35 AM, Roberto Peon wrote:
> Honestly, though, anything which spells out something easy to remember
> in 7-bit ascii when the high-bits are all xor'd would be nice :)
>
> -=R
>
>
> On Thu, Jan 31, 2013 at 5:24 PM, Martin Thomson
> <[email-masked] <mailto:[email-masked]>> wrote:
>

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 01 Feb 2013 03:09:34 +0000

------ Original Message ------
From: "Eliot Lear" <[email-masked]>
To: "Roberto Peon" <[email-masked]>
Cc: "Martin Thomson" <[email-masked]>; "HTTP Working Group" 
<[email-masked]>
Sent: 1/02/2013 4:04:49 p.m.
Subject: Re: HTTP/2.0 Magic
>server as a version identifier.  We just rev/change the magic on a new 
I wouldn't do that.

network sniffers will get coded to the magic number.  They'd need to be 
recoded to even recognise the new version with a different magic number 
as being still HTTP

Adrien

>><[email-masked]> wrote:

---

From: Eliot Lear <[email-masked]>
Date: Fri, 01 Feb 2013 12:11:09 +0900

The alternative is to append something for version info.

On 2/1/13 12:09 PM, Adrien W. de Croy wrote:
> From: "Eliot Lear" <[email-masked] <mailto:[email-masked]>>
> To: "Roberto Peon" <[email-masked] <mailto:[email-masked]>>
> Cc: "Martin Thomson" <[email-masked]
> <mailto:[email-masked]>>; "HTTP Working Group"
> <[email-masked] <mailto:[email-masked]>>
>> Separately we will also need a version identifier.  This field can
>> ALSO server as a version identifier.  We just rev/change the magic on
>> a new version.
> I wouldn't do that.
>  
> network sniffers will get coded to the magic number.  They'd need to
> be recoded to even recognise the new version with a different magic
> number as being still HTTP
>  
> Adrien
>  
>>
>>
>>> Honestly, though, anything which spells out something easy to
>>> remember in 7-bit ascii when the high-bits are all xor'd would be
>>> nice :)
>>>
>>> -=R
>>>
>>>

---

From: Dr Robert Mattson <[email-masked]>
Date: Fri, 1 Feb 2013 14:47:19 +1100

This will destroy transparent backward-compatability with older HTTP versions.
Any reduction in page retrieval time could be dwarfed by such awkward interaction.
HTTP-MPLEX proposed the use of TXT records in the DNS to use a /proc/cpu:flags like mechanism to accompany the uri/ip conversion request. Highly scalable and only delays the request by the time it takes to send and receive slightly larger udp packets.

Sent on a mobile device, typos expected.

On 01/02/2013, at 2:11 PM, Eliot Lear <[email-masked]> wrote:

>> Cc: "Martin Thomson" <[email-masked]>; "HTTP Working Group" <[email-masked]>
>>> Separately we will also need a version identifier.  This field can ALSO server as a version identifier.  We just rev/change the magic on a new version.
>> I wouldn't do that.
>>  
>> network sniffers will get coded to the magic number.  They'd need to be recoded to even recognise the new version with a different magic number as being still HTTP
>>  
>> Adrien
>>  
>>> 
>>> 
>>>> Honestly, though, anything which spells out something easy to remember in 7-bit ascii when the high-bits are all xor'd would be nice :)
>>>> 
>>>> -=R
>>>> 
>>>> 
>>>> On Thu, Jan 31, 2013 at 5:24 PM,                 Martin Thomson <[email-masked]> wrote:

---

From: Chris Elving <[email-masked]>
Date: Thu, 31 Jan 2013 21:37:04 -0800

> How about: d3 d0 c4 d9? (as an hommage).

If the objective is to make existing HTTP/1.1 clients barf, I imagine the
NUL byte will be more effective than anything else. The non-whitespace
control characters are probably a close second.

---

From: Frédéric Kayser <[email-masked]>
Date: Fri, 1 Feb 2013 07:39:13 +0100

What about "91 52 a5 0_" with _ available for versioning?

9152a50_ comes from "HTTP" in ASCII: 48 54 54 50 (0100 1000    0101 0100    0101 0100    0101 0000) with the highest bit –underlined here– removed it translates into the following 28 bits (1001 0001    0101 0010    1 010 0101    0000 xxxx) the last 4 bits (added to form a 32-bits value) could be used for versioning.

SPDY
0x53504459 + 0x80808080 = 0xd3d0c4d9

Regards
-- 
Frédéric Kayser

---

From: Martin Thomson <[email-masked]>
Date: Fri, 1 Feb 2013 16:09:12 +0900

One more thing before I forget:

I know that it is tempting to come up with a vaguely funny or "clever"
magic.  (Someone suggested 0xdeadbeef, which does meet the criteria)
However, what would be really good is if there were input in the form:

"4 bytes is not sufficient, client/server X ignores N garbage bytes
before deciding that a connection is thrown away";
or
"X ignores lots of bytes, but sending Connection:close in the upgrade
might work";
or
"um, that's valid UTF-8, some implementations parse that".

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 31 Jan 2013 23:17:26 -0800

> having the first bit set ensures that this isn't valid HTTP/1.1.

Umm, sending high bit characters is far more likely to be ignored
or trigger parsing bugs than simple ascii.

The best way to a quick and safe end (actually tested in
practice 10 years ago) is to send something that looks like a method
and ends with CRLF.  Hence, waka sends a 64bit value

   wakaVV<CRLF>

where the first V is the version being sent and the second V is the
highest version accepted.  Both are a variant of base64 that starts
at 0, so the beginning of each message is

  77 61 6b 61 30 30 0d 0a

> magic is sufficient as the tests that lead to websockets masking

Don't worry about it.  The group is working on HTTP/2, not some
private hack, and it simply doesn't matter how many bugs you
trigger in broken intermediaries.  They will fix themselves.

....Roy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 1 Feb 2013 08:35:05 +0100

Hi Martin,

> magic is sufficient as the tests that lead to websockets masking

I'm really scared that we're doing again the same mistakes as websocket
initially did and that we go through the exact same process of stacking
idiocies on top of each other until everyone is more or less happy with
the result. We should be very careful here because it started like this.

HTTP is a well defined protocol and it supports an upgrade mechanism, I
think it should be better to reuse this standardized and documented
mechanism than guessing what will happen with some random magic and then
have many people complain about broken devices which transparently pass
the signature, then have some security folks show to the world the high
risk it represents and then have everyone suggest how to cripple the
design so that caches are not at risk.

BTW if we're really going down that road, Roy is right that the prefix
should look like a method so that at least most devices take this for
a minimalistic HTTP/0.9 request and don't try to parse more than what
they get.

Willy

---

From: James M Snell <[email-masked]>
Date: Thu, 31 Jan 2013 23:34:25 -0800

On Thu, Jan 31, 2013 at 11:17 PM, Roy T. Fielding <[email-masked]> wrote:

> [snip]
+1 ... sending a simple "HTTP20<CRLF>" seems it would be sufficiently
effective. The 20 is the major and minor version. The preamble for http/2.1
would be "HTTP21<CRLF>"

 - James

---

From: David Morris <[email-masked]>
Date: Thu, 31 Jan 2013 23:46:27 -0800 (PST)

> effective. The 20 is the major and minor version. The preamble for http/2.1

It seems to me that including CRLF is more likely to result in the string
being ignored then if it is an unterminated sequence which would get
catenated with the payload by a parser which didn't know about it.

Dave Morris