Subject: The use of binary data in any part of HTTP 2.0 is not good

From: Pablo <[email-masked]>
Date: Sun, 20 Jan 2013 19:20:25 -0300

Hello,

   I have readed this document
http://dev.chromium.org/spdy/spdy-protocol/spdy-protocol-draft1 today [1].

I just wanted to say that I think that the use of any binary data (framing,
header compression, etc.) in any place of the "header" part of HTTP
protocol is not good; so, please only use plaintext for HTTP 2.0 because,
otherwise, that will make very difficult to "see" the headers's protocol :)

Thats all,
Thanks for reading this few lines, sorry for my basic English, and I hope
that you can re-think all this of using binary data in any part of HTTP X.X
(ej: session layer).

[1] I started knowing about HTTP 2.0 here:
http://webscannotes.com/2012/10/09/http-2-0-officially-in-the-works/

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 20 Jan 2013 23:30:22 +0100

Hi Pablo,

> that you can re-think all this of using binary data in any part of HTTP X.X

As much as I love to read HTTP protocol in network traces or in programs,
I must say that we (humans) are very rare HTTP readers. I suspect that only
something like 1 request on 1 billion is read by a human. This is not a great
enough ratio for keeping an ambiguous, complex, and sometimes even insecure
protocol to parse.

I too tried as much as I could to see what would be achievable with a text
based protocol, but I finally admitted it was a dead end. At the moment the
challenges consist in feeding requests as fast as possible over high latency
connections and processing them as fast as possible on load balancers and
caches in order to maintain a scalable internet. Humans are very incapable
devices there.

Regards,
Willy

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 14:36:56 -0800

There are many advantages to using binary data. If you would like a
textual representation of a protocol, I advise using a utility to
generate one for you.

On Sun, Jan 20, 2013 at 2:20 PM, Pablo <[email-masked]> wrote:
> header compression, etc.) in any place of the "header" part of HTTP protocol
> is not good; so, please only use plaintext for HTTP 2.0 because, otherwise,
> that will make very difficult to "see" the headers's protocol :)
>
> Thats all,

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 10:04:08 +1100

In one of our recent meetings, one of the grey-bearded IETF old-timers (I forget which, sorry) said that a textual-protocol was a nice-to-have, but that it shouldn't be a determining factor in design. 

I.e., if you can get everything you need out of a protocol, *and* make it textual, do so, but if it detracts from the value you get from it, don't let that constrain you.

FWIW, I think that's a good rule of thumb. However, this means that the community is going to need *excellent* tooling for analysing, debugging, etc. HTTP traffic; and I don't just mean a Wireshark plugin!

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 21 Jan 2013 12:14:26 +1300

> textual representation of a protocol, I advise using a utility to

Perhapse one of the "developer tools" utilities, which *already* 
altering the actual ASCII bytes transmitted into other ASCII bytes with 
more whitespace, wrapping, and information markup for human display?

Amos

---

From: David Morris <[email-masked]>
Date: Sun, 20 Jan 2013 15:15:41 -0800 (PST)

> caches in order to maintain a scalable internet. Humans are very incapable

It won't be rocket science to create a plugin for Wireshark/etherial and
other network tools which can format the binary data for those cases where
humans need to do that for debugging.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 15:21:28 -0800

Indeed. It might be something similar to
https://code.google.com/p/spdyshark/. Or spdycat
(https://github.com/tatsuhiro-t/spdylay).

---

From: Tim Bray <[email-masked]>
Date: Sun, 20 Jan 2013 15:25:06 -0800

Would it be possible to be data-driven?  Textual formats are
well-known to be easier to debug; but clearly, if there’s a
substantial performance benefit to going all-binary, so be it. So what
is the advantage, quantitatively? -T

---

From: James M Snell <[email-masked]>
Date: Sun, 20 Jan 2013 15:36:55 -0800

This is still being worked out really, qualitative numbers will be coming.
So far, however, we're talking about around a 50% reduction in header
overhead on the wire without compression. Obviously, however, we're no
where near being done yet.
On Jan 20, 2013 3:26 PM, "Tim Bray" <[email-masked]> wrote:

> > In one of our recent meetings, one of the grey-bearded IETF old-timers
> (I forget which, sorry) said that a textual-protocol was a nice-to-have,
> but that it shouldn't be a determining factor in design.
> >
> > I.e., if you can get everything you need out of a protocol, *and* make
> it textual, do so, but if it detracts from the value you get from it, don't
> let that constrain you.
> >
> > FWIW, I think that's a good rule of thumb. However, this means that the
> community is going to need *excellent* tooling for analysing, debugging,
> etc. HTTP traffic; and I don't just mean a Wireshark plugin!
> >
> > Cheers,
> >
> >
> protocol

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sun, 20 Jan 2013 23:38:22 +0000

the thing that will make debugging harder won't be binary vs text, but 
the inter-dependence of messages.  Especially when it comes to looking 
through debug logs for issues.

On-the-wire, you already need to piece together a TCP stream to see 
what's going on, so having http messages effectively split over multiple 
frames (e.g. delta encoding, or compression) only becomes a problem when 
you don't capture enough to decode.

I think it might be worth-while specifying a requirement for a "debug" 
option for senders of binary messages which turns off all other 
optimisations, such as caching unchanged headers etc (so they are sent 
every time).  Just an idea.

------ Original Message ------
From: "Mark Nottingham" <[email-masked]>
To: "William Chan (陈智昌)" <[email-masked]>
Cc: "Pablo" <[email-masked]>; "HTTP Working Group" 
<[email-masked]>
Sent: 21/01/2013 12:04:08 p.m.
Subject: Re: The use of binary data in any part of HTTP 2.0 is not good
>(I forget which, sorry) said that a textual-protocol was a 
>nice-to-have, but that it shouldn't be a determining factor in design.
>
>it textual, do so, but if it detracts from the value you get from it, 
>don't let that constrain you.
>
>community is going to need *excellent* tooling for analysing, 
>debugging, etc. HTTP traffic; and I don't just mean a Wireshark plugin!
>
>Cheers,
>
>

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 21 Jan 2013 12:42:15 +1300

> is the advantage, quantitatively? -T

Simpler parsing, less CPU loading, fewer interoperbility problem 
possibilities, and ~60% reduction in headers bandwidth if I'm reading 
Marks earlier posts with metrics correctly.

Amos

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 10:47:25 +1100

Well, there are two aspects; one is the encoding for the headers, the other is the framing for the protocol itself.

Additionally, binary can bring benefits in size, leading to performance benefits. However, it can also bring scalability benefits, because it's a lot easier to parse, requiring less memory / copying / look-ahead...

Measuring size benefits is fairly straightforward (as we're already doing <http://http2.github.com/http_samples/>); measuring scalability benefits is more implementation- and deployment-specific, and so more difficult. We can try, however.

FWIW, my "simple" approach *is* textual*, at least for the header encoding part of the problem. Textual framing is also possible, of course, but it's going to come at a fair overhead (in terms of size and in terms of processing overhead), I suspect.

If anyone is interested in pursuing these approaches, an Internet-Draft making a concrete proposal is the best way to get our attention. And of course, if people want to see measurements, they can pitch in (as things seldom measure themselves).

Cheers,

* Or, at least, it was until a few days ago, when I switched to a 7-bit encoding of ASCII; that can be disabled (at a cost of 12.5% efficiency, naturally).

On 21/01/2013, at 10:36 AM, James M Snell <[email-masked]> wrote:

> This is still being worked out really, qualitative numbers will be coming. So far, however, we're talking about around a 50% reduction in header overhead on the wire without compression. Obviously, however, we're no where near being done yet.
>

---

From: Roberto Peon <[email-masked]>
Date: Sun, 20 Jan 2013 15:47:22 -0800

Text formats are, surprisingly, not easier to debug in my experience.

Admittedly, they only require a text editor to examine, but, for a binary
protocol, that is easy to remedy with a short utility. The ambiguity of how
to parse that text format, however makes this examination more time
consuming and error prone.

In terms of total complexity, binary framed protocols are often far
simpler, and thus more robust. In terms of total lines of code, a binary
framer plus utility to present the data in human readable format is still
less complex and smaller than most text framed protocols. In terms of CPU
consumption, binary framed protocols are often faster.

If you'd like to make a fast http parser, I'm happy to give you a binary
framed version which will be significantly faster, smaller in lines of
code, and more memory efficient.

-=R

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 10:48:30 +1100

On 21/01/2013, at 10:38 AM, "Adrien W. de Croy" <[email-masked]> wrote:

> 
> the thing that will make debugging harder won't be binary vs text, but the inter-dependence of messages.  Especially when it comes to looking through debug logs for issues.
> 
> On-the-wire, you already need to piece together a TCP stream to see what's going on, so having http messages effectively split over multiple frames (e.g. delta encoding, or compression) only becomes a problem when you don't capture enough to decode.
> 
> I think it might be worth-while specifying a requirement for a "debug" option for senders of binary messages which turns off all other optimisations, such as caching unchanged headers etc (so they are sent every time).  Just an idea.

That's been brought up a few times, and the reaction has been pretty positive.

Cheers,

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 15:48:42 -0800

Tim's question can already be partially answered. See the SPDY
whitepaper (http://www.chromium.org/spdy/spdy-whitepaper). I quote:
"Header compression resulted in an ~88% reduction in the size of
request headers and an ~85% reduction in the size of response headers.
On the lower-bandwidth DSL link, in which the upload link is only 375
Kbps, request header compression in particular, led to significant
page load time improvements for certain sites (i.e. those that issued
large number of resource requests). We found a reduction of 45 - 1142
ms in page load time simply due to header compression." I say that
this counts as the answer, because from a human's perspective, the
difference between header compression and bohe is marginal. It's
binary. Yes, maybe some humans will internalize a binary encoding of
headers and be able to grok hexdumps, but to the vast majority of
people, it's basically the same.

Yes, there are other concerns too, such as CPU/memory issues. Also,
that's a qualitative difference in terms of simplicity of parsing by
machines. Patrick wrote up a good explanation here:
http://news.ycombinator.com/item?id=4933567.

On Sun, Jan 20, 2013 at 3:36 PM, James M Snell <[email-masked]> wrote:
> overhead on the wire without compression. Obviously, however, we're no where
> near being done yet.
>
>> > (I forget which, sorry) said that a textual-protocol was a nice-to-have, but
>> > that it shouldn't be a determining factor in design.
>> >

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 15:55:19 -0800

Maybe it falls upon me to be the voice of concern here :) Depending on
what a "debug" option entails, I'm worried about it being used to
disable a performance feature. As an example of how options can be
dangerous, we've seen intermediaries that strip out Accept-Encoding
headers in order to force responses to be uncompressed (probably so
they can inspect the payloads more easily/cheaply), which is an issue
from a web performance perspective.

Back to the use case, if you're in a position to use the debug option,
is it likely that you would not also be in a position to capture
enough to decode? I'd like to understand the use case so I can
properly weigh the benefit of such an option, in contrast to the cost
that I highlighted above.

---

From: Nico Williams <[email-masked]>
Date: Sun, 20 Jan 2013 18:01:41 -0600

On Sun, Jan 20, 2013 at 5:48 PM, William Chan (陈智昌)
<[email-masked]> wrote:
> [...] Yes, maybe some humans will internalize a binary encoding of

Nah, we have plenty of packet capture parsers (Netmon, Wireshark,
tcpdump, snoop, ...).  Wireshark, in particular, is very easy to write
new plugins for, and it's portable.

I'm with Roberto too: it's not really true that textual protocols are
easier to debug, at least not now that we have extensible packet
capture inspection tools.  Further, textual protocols may inhibit the
creation of dissectors for them ("it's text already, what you do need
a dissector for?"), which makes them harder to inspect than binary
protocols.

We can probably apply a lot of minimal encodings of header values (and
header names) in a textual way, but the result would be nearly as
incomprehensible (without tools) to a human as a hex dump of a binary
protocol.  So merely trying to do as best as we can while retaining a
textual nature seems likely to lead to either few gains or a protocol
that's as [in]scrutable as a binary version.

Finally, as others have pointed out, human-readable textual protocols
are likely to allow lots of variations that complicate parsers, thus
being a cause of bugs.

Nico
--

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 11:05:50 +1100

It was discussed in terms of disabling it at the sender, not through negotiation, so I don't think the case you're talking about is going to be a problem.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Mon, 21 Jan 2013 00:05:28 +0000

Hi

------ Original Message ------
From: "William Chan (陈智昌)" <[email-masked]>
>from a web performance perspective.
guilty as charged.  we don't do that any more.

>that I highlighted above.

often one is dealing with a customer who you can reasonably expect to be 
capable enough to click a checkbox and send you a file, but maybe not to 
install a packet capturing utility, set up filters etc etc.  Maybe even 
not permitted to install e.g. wireshark.

I agree in most cases, ability to select debug and try a problem case 
would come along with an ability to capture enough packets.

So I guess it's a small use-case in that instance.

I can imagine other areas where reducing all messages to a single frame 
could be useful, like debugging issues with load balancers or http 
routers which may be switching frames incorrectly.

In short, we can't know all the issues we may come up against.

Adrien

>>>  the thing that will make debugging harder won't be binary vs text, 
>>>but the inter-dependence of messages. Especially when it comes to 
>>>looking through debug logs for issues.
>>>
>>>what's going on, so having http messages effectively split over 
>>>multiple frames (e.g. delta encoding, or compression) only becomes a 
>>>problem when you don't capture enough to decode.
>>>
>>>  I think it might be worth-while specifying a requirement for a 
>>>"debug" option for senders of binary messages which turns off all 
>>>other optimisations, such as caching unchanged headers etc (so they 
>>>are sent every time). Just an idea.
>>

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 16:07:08 -0800

I'd like to point out it seems highly likely you misread my email.
When I said the same, I was referring to bohe and header compression
from a human's perspective. You may want to read earlier on in the
email thread where I was one of the first to respond saying:

"There are many advantages to using binary data. If you would like a
textual representation of a protocol, I advise using a utility to
generate one for you."

and also recommended debugging utilities like spdyshark and spdycat as
examples of what we've done with SPDY.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 16:41:14 -0800

I need to consider it longer, since I still have some concerns -
namely, client-side intermediaries / 3rd-party software.

Note that it's common nowadays for 3rd party software to interact with
browser software at a very intimate level. Maybe do it in the form of
Winsock LSPs (see
https://code.google.com/p/chromium/issues/detail?id=27870 for an
example of how we've had to work around bugs here), (oft-times
malicious) extensions
(https://code.google.com/p/chromium/issues/detail?id=128748#c15), or
external applications directly mucking with browser specific data
(here are links to examples of Samsung Support Center directly mucking
with the browser cache:
https://bugzilla.mozilla.org/show_bug.cgi?id=825665 &
https://code.google.com/p/chromium/issues/detail?id=167578#c11).

Many times these intermediaries / 3rd party software are trying to do
"legitimate" things like sniffing headers so they can do filtering
based on that. If we add a sender-configured option to enable
disabling header encoding/compression at the client, it's very likely
that 3rd party software will take advantage of that option for popular
client software (popular browsers). Thus, even if such an option
existed in the spec, I would be nervous about exposing it within
Chromium in such a way that 3rd party software could force it on.

And if such a mechanism existed sender-side, I still don't see what
(beyond a secured transport like TLS) prevents an ISP or corporate
network administrator who has filtering software he/she wants to run
from adding a hop that enables this debug option to make it easier to
pass through to 3rd party authored filtering software running within
the ISP / corporate network.

My concerns are admittedly erring on the side of paranoia, but it is
definitely advised by our (Google Chrome) previous experience in this
area. Please see
https://developers.google.com/speed/articles/use-compression where we
provide data on the prevalence of the aforementioned Accept-Encoding
stripping issue. "There are a handful of ISPs, where the percentage of
uncompressed content is over 95%. One likely hypothesis is that either
an ISP or a corporate proxy removes or mangles the Accept-Encoding
header." Accept-Encoding is a sender-side controlled header, so unless
the proposed option is protected somehow via TLS or something, I would
imagine that it likewise would expose us to the issue we've seen with
Accept-Encoding stripping.

All options are ripe for abuse. We should be very careful to make sure
the option is truly necessary, rather than just potentially useful, in
order to counterbalance the downside of possible abuse.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 11:50:01 +1100

> Chromium in such a way that 3rd party software could force it on.

Roberto mentioned that the sender could just choose not to compress their output; how would you stop that?

> Accept-Encoding stripping.

Again, this isn't negotiation (as currently discussed); it's the sender choosing not to compress.

Even if it were a negotiation, I suspect you'd find that the dynamics that you saw in play with compressing payloads doesn't play out the same way as it does with headers. Intermediaries strip accept-encoding because decompressing and recompressing the response bodies coming through them presents a scalability challenge; if we do our job right with header compression, it shouldn't be nearly as much of a problem for them.

Besides which, if we design HTTP/2 to be unpalatable to middleboxes, they'll merely block it and force all traffic to HTTP/1, meaning that those users won't see *any* benefits from the new protocol.

> order to counterbalance the downside of possible abuse.

To counter that -- I'm somewhat wary of approaching protocol design as an exercise in controlling how the result is used; people *will* work around your intent. While we can do some social engineering in this process, it's very soft, and very limited, power.

Cheers,

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 17:42:14 -0800

> Roberto mentioned that the sender could just choose not to compress their output; how would you stop that?

TLS + providing no Chromium-configurable option to disable header
compression, at least from our (Chromium's) end :) If the origin
server disables header compression in their responses, we can't help
that, in the same way we can't help if they don't compress their
response bodies. We do what we can. It's possible that, since Google
Chrome includes auto-update capabilities, we can err on the side of
providing an option and revoking it if abused.

> Again, this isn't negotiation (as currently discussed); it's the sender choosing not to compress.

Sorry, I either misunderstood or was unclear. In my understanding,
while A-E is generally a negotiation, if you simply strip A-E, it's
effectively the same as "the sender choosing not to compress".

> Even if it were a negotiation, I suspect you'd find that the dynamics that you saw in play with compressing payloads doesn't play out the same way as it does with headers. Intermediaries strip acce…

This is speculation of course, but I suspect that many do it simply
for simplicity's sake. While I think intermediaries like
Varnish/HAProxy/Squid may have legitimate scalability concerns if/when
they disable stuff like compression, I think that many intermediaries,
like virus scanners and what not, simply disable things like
compression to make them easier to process. Maybe I'm more jaded about
the quality of engineering out there than you, but I suspect most
engineers simply consider what's easiest to get their job done, rather
than the overall implications of their actions. Otherwise, I'm pretty
surprised some ISPs strip A-E, since I'd imagine that the bandwidth
costs exceed the scalability costs for implementing their filtering
policies. Of course, I'm no expert here and would love to be educated.

> Besides which, if we design HTTP/2 to be unpalatable to middleboxes, they'll merely block it and force all traffic to HTTP/1, meaning that those users won't see *any* benefits from the new protocol.

This is why I asked about the use case, so we could evaluate how
necessary this option is. I'm hoping it is not unpalatable. Adrien's
response leaves me somewhat hopeful, although it's still uncertain.

> To counter that -- I'm somewhat wary of approaching protocol design as an exercise in controlling how the result is used; people *will* work around your intent. While we can do some social engineer…

Fair enough. I mostly wanted to chime in to provide the "other"
opinion here, since you stated that the previous reaction had been
pretty positive. Please take my comments as a contrary opinion for
people to consider before we draw any conclusions here.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2013 13:04:39 +1100

> effectively the same as "the sender choosing not to compress".

Nope; if you want to say that the client doesn't accept compression, you have to do so explicitly;

https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/latest/p2-semantics.html#header.accept-encoding

"If no Accept-Encoding field is in the request, any content-coding is considered acceptable by the user agent."

> compression to make them easier to process.

Lots of virus scanning happens in intermediaries :)

> people to consider before we draw any conclusions here.

Absolutely. 

Cheers,

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 20 Jan 2013 18:32:14 -0800

> "If no Accept-Encoding field is in the request, any content-coding is considered acceptable by the user agent."

Sorry, I was inexact. I should have said stripping of A-E values that
allow for compression. My understanding is that the knob that
intermediaries use here is exactly the knob you describe of "the
sender choosing not to compress", for the explicit purpose of
receiving uncompressed content, for reasons that include scalability.

Oops, wait, I think I did indeed misunderstand. I believe I was
conflating sender with client. You're talking about the request
headers in this case. Brain fart, sorry.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 21 Jan 2013 10:15:33 +0100

Let's separate the general case from this one.  In the general case,
interoperability can be helped with text you don't need a library to
interact the other endpoint.  You just need fingers and telnet.  To
underestimate that value is to I think misunderstand the history of some
of our success.  Early movement to ASN.1 for so-called ease of parsing
by computers and performance led to practically a SINGULAR
implementation of the encode/decode rules, which in turn led to a
horrendous security mess in 2002.

HTTP may require compression and performance improvement; we'll see, I
suppose.  Certainly it's vastly more complex than SMTP, requiring not
just a simple capabilities exchange and then a transaction, but quite a
bit more interpretation of results by both parties.  The struggle we
have before us is whether the ecosystem can or should move over, based
on such a low level change.  If it leads to a small number of encoders
and decoders, surely that would be bad.

Eliot

On 1/21/13 12:47 AM, Roberto Peon wrote:
> Admittedly, they only require a text editor to examine, but, for a
> binary protocol, that is easy to remedy with a short utility. The
> ambiguity of how to parse that text format, however makes this
> examination more time consuming and error prone.
>
> simpler, and thus more robust. In terms of total lines of code, a
> binary framer plus utility to present the data in human readable
> format is still less complex and smaller than most text framed
> protocols. In terms of CPU consumption, binary framed protocols are
> often faster.
>
> If you'd like to make a fast http parser, I'm happy to give you a
> binary framed version which will be significantly faster, smaller in
> lines of code, and more memory efficient.
>
> -=R
>
> On Jan 20, 2013 3:26 PM, "Tim Bray" <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     On Sun, Jan 20, 2013 at 3:04 PM, Mark Nottingham <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > In one of our recent meetings, one of the grey-bearded IETF
>     old-timers (I forget which, sorry) said that a textual-protocol
>     was a nice-to-have, but that it shouldn't be a determining factor
>     in design.
>     >
>     > I.e., if you can get everything you need out of a protocol,
>     *and* make it textual, do so, but if it detracts from the value
>     you get from it, don't let that constrain you.
>     >
>     > FWIW, I think that's a good rule of thumb. However, this means
>     that the community is going to need *excellent* tooling for
>     analysing, debugging, etc. HTTP traffic; and I don't just mean a
>     Wireshark plugin!
>     >
>     > Cheers,
>     >
>     >
>     > On 21/01/2013, at 9:36 AM, William Chan (陈智昌)
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     >
>     >> On Sun, Jan 20, 2013 at 2:20 PM, Pablo <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >>> Hello,
>     >>>

---

From: Patrick McManus <[email-masked]>
Date: Mon, 21 Jan 2013 10:04:07 -0500

On Sun, Jan 20, 2013 at 6:47 PM, Roberto Peon <[email-masked]> wrote:

> +1

Beyond debugging, text formats are actually ridiculously hard to get
consistently right - partially because they lend themselves too well to "be
liberal in what you receive policies". HTTP/1 has suffered from  CRLF
injection attacks, content-length bounds check failures, a wide variety of
line ending interop problems, etc..  all of which are derived from its text
roots. Sometimes these problems are unforseen spec issues, but sometimes
they are just derived from assumptions people have because the text format
feels more intuitive than it really is.

While text is convenient to eyeball, it is much harder to get unambiguously
correct especially in a open multiple implementation environment. 32 bits
of big endian is well defined and well bounded; a text string that
represents a quantity requires a lot more information to correctly
interpret.

http://blog.jgc.org/2012/12/speeding-up-http-with-minimal-protocol.html#c5703739431744738432

Frankly, I'd rather talk about byte order.. imo this is an application
level protocol that 98% of the time is going to be consumed by little
endian processors and could easily be defined that way. This of course runs
against tradition and isn't a huge deal computationally, but I'm not aware
of other arguments against giving the byte swap operation of our processors
a day off.

---

From: Patrick McManus <[email-masked]>
Date: Mon, 21 Jan 2013 10:10:55 -0500

On Sun, Jan 20, 2013 at 8:42 PM, William Chan (陈智昌)
<[email-masked]>wrote:

> On Sun, Jan 20, 2013 at 4:50 PM, Mark Nottingham <[email-masked]> wrote:y
> limited, power.
>

I agree with Will here - if we end up with a stateful mechanism (which we
seem to be headed towards; reasonably so) then let's have just that
mechanism.

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 22 Jan 2013 21:55:21 +1300

On 22/01/2013 4:04 a.m., Patrick McManus wrote:
>
>
>
> On Sun, Jan 20, 2013 at 6:47 PM, Roberto Peon <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> consistently right - partially because they lend themselves too well 
> to "be liberal in what you receive policies". HTTP/1 has suffered 
> from  CRLF injection attacks, content-length bounds check failures, a 
> wide variety of line ending interop problems, etc..  all of which are 
> derived from its text roots. Sometimes these problems are unforseen 
> spec issues, but sometimes they are just derived from assumptions 
> people have because the text format feels more intuitive than it 
> really is.
>
> While text is convenient to eyeball, it is much harder to get 
> unambiguously correct especially in a open multiple implementation 
> environment. 32 bits of big endian is well defined and well bounded; a 
> text string that represents a quantity requires a lot more information 
> to correctly interpret.
>
> endian processors and could easily be defined that way. This of course 
> runs against tradition and isn't a huge deal computationally, but I'm 
> not aware of other arguments against giving the byte swap operation of 
> our processors a day off.
>

Three possible reasons for keeping it big-endian:

1) the existing library functionality htons() and friends are 
big-endian. Don't underestimate the benefit using well-known 
functionality instead of having to locate uncommon little-endian 
converters will give to developers in the HTTP/2 rollout.

2) big-endian is more intuitive to read. I know we are arguing for tools 
to be used by the masses. But somebody, sometime is going to have to 
eyeball the raw binary to figure out a tricky interop problem. Lets not 
make that job harder than it has to be, because it will probably be one 
of us here doing it.

3) big-endian is more suited to streamed octet interpretation when we 
are defining data fields at less than 32-bit resolution.

That said, I don't have a preference.

Amos

---

From: Nico Williams <[email-masked]>
Date: Tue, 22 Jan 2013 03:04:04 -0600

On Tue, Jan 22, 2013 at 2:55 AM, Amos Jeffries <[email-masked]> wrote:
> 1) the existing library functionality htons() and friends are big-endian.
> Don't underestimate the benefit using well-known functionality instead of
> having to locate uncommon little-endian converters will give to developers
> in the HTTP/2 rollout.

I'm a fan of big-endian (for your reason #2), but this is not really a
good reason for it any longer.  By now we have lots of libraries for
swabbing that are bettern than hton* and ntoh*().

> That said, I don't have a preference.

Little-endianness is arguably more common, so we should go with that.
But then again, mobile devices seem to be causing an increase in
heterogeneity, so maybe not.  I don't really care.

We could also go with receiver-makes right.  In this case every
message (request, response) must include a BOM and then the receiver
makes right.  This strikes me as fair, if not, perhaps, compelling..

Nico
--

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Tue, 22 Jan 2013 09:33:38 +0000

------ Original Message ------
From: "Nico Williams" <[email-masked]>
>makes right. This strikes me as fair, if not, perhaps, compelling..
I'd rather have a milllion htons and ntohs than having to procede each 
one with an if statement as well.

so I think this is a bad idea

Adrien

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 22 Jan 2013 23:27:14 +1300

> so I think this is a bad idea

You could optimize by writing two parsers in complete duplicate and 
switching between them on sighting the BOM.
It only doubles the memory footprint instead of the CPU footprint. ;-)

Amos

---

From: Roberto Peon <[email-masked]>
Date: Tue, 22 Jan 2013 02:29:27 -0800

Ya, that still sounds bad :)
(Too bad self-modifying code is the hallmark of a virus these days...)
-=R