Subject: Re[2]: multiplexing -- don't do it

From: "Adrien W. de Croy" <[email-masked]>
Date: Sun, 01 Apr 2012 21:25:48 +0000

I agree as well, even though it will also cause me some pain.
  
We've been debugging binary / non-text / non-human-readable protocols 
for decades.  DNS and DHCP are 2 that spring immediately to mind.
  
Common network analysers shouldn't have much trouble decoding what has 
been proposed.
  
Adrien

------ Original Message ------
From: "Willy Tarreau" <[email-masked]>
To: "Mike Belshe" <[email-masked]>
Cc: "Peter L" <[email-masked]>;"[email-masked]" 
<[email-masked]>
Sent: 1/04/2012 4:39:43 p.m.
Subject: Re: multiplexing -- don't do it
>On Fri, Mar 30, 2012 at 03:22:12PM +0200, Mike Belshe wrote:
>
>>
>>What is "transparency on the wire"?  You mean an ascii protocol that you
>>can read?  I don't think this is a very interesting goal, as most people
>>don't look at the wire.
>>
>
>
>I agree with you here Mike, despite being used to look at network captures
>all the day and testing proxies with "printf|netcat" at both ends. But we
>must admit that if developers need tools, they will develop their tools.
>Having an HTTP option for netcat would work well, or even having an 1.1-to-2.0
>and 2.0-to-1.1 message converter on stdin/stdout would do the trick. So I
>prefer to lose the ability to easily debug and have something efficient than
>the opposite. And it costs me a lot to say this :-)
>
>Willy
>
>
>
>

---

From: Peter L <[email-masked]>
Date: Sun, 1 Apr 2012 17:33:54 -0400

It's not just a matter of reading a binary format. SPDY gzip needs a synch point. So there definitely will be times (when the capture does not go far enough back in time) when tools will not be able to decode the compressed headers.

This is undesirable in my opinion. Seems strange that others do not agree.

Thanks for putting up with reading my concerns. 

:-)

Peter

On Apr 1, 2012, at 5:25 PM, "Adrien W. de Croy" <[email-masked]> wrote:

> We've been debugging binary / non-text / non-human-readable protocols for decades.  DNS and DHCP are 2 that spring immediately to mind.
> Common network analysers shouldn't have much trouble decoding what has been proposed.
> Adrien
> 
> Cc: "Peter L" <[email-masked]>;"[email-masked]" <[email-masked]>

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Sun, 01 Apr 2012 21:39:15 +0000

Hi Peter
  
in my experience, the reason packet captures get created in the first 
place, is because someone has noticed a _recurring_ problem and is 
trying to debug it / track it down.
  
therefore it may take a bit more work to get a full capture that does 
go back far enough, but is usually possible, even if initial captures 
don't.
  
Our proposal for 2.0 doesn't apply gzip to the stream though, but does 
cache headers across a hop, so could be said to suffer from a similar 
problem, since depending how far back a capture goes, you may miss the 
transfer of some headers.
  
It could probably have a mode to disable this per-hop caching though.
  
Adrien

------ Original Message ------
From: "Peter L" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Willy Tarreau" <[email-masked]>;"Mike Belshe" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 2/04/2012 9:33:54 a.m.
Subject: Re: Re[2]: multiplexing -- don't do it

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 1 Apr 2012 23:50:24 +0200

I agree with Adrien here. Moreover, I think Mike already noted this
elsewhere, but Chromium has a about:net-internals frontend which includes
all the debugging information. It does not need to be opened before the
compression context was established, since it receives the decompressed
data from the network layer which obviously has the compression context.
Other software could do similar things. Therefore, I don't share Peter's
concern here.

On Sun, Apr 1, 2012 at 11:39 PM, Adrien W. de Croy <[email-masked]> wrote:

> place, is because someone has noticed a _recurring_ problem and is trying
> to debug it / track it down.
>  therefore it may take a bit more work to get a full capture that does go
> back far enough, but is usually possible, even if initial captures don't.
> Cc: "Willy Tarreau" <[email-masked]>;"Mike Belshe" <[email-masked]>;"ietf-http-
> [email-masked] <[email-masked]>" <[email-masked]>
>> It's not just a matter of reading a binary format. SPDY gzip needs a
>> synch point. So there definitely will be times (when the capture does not
>> go far enough back in time) when tools will not be able to decode the
>> compressed headers.
>>
>>> Cc: "Peter L" <[email-masked]>;"[email-masked]<[email-masked]>"
>>> <[email-masked]>

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 2 Apr 2012 07:23:53 +0200

Hi Peter,

> This is undesirable in my opinion. Seems strange that others do not agree.

I'm among the people who are used to plug a USB disk somewhere, fill it with
500 GB of captures then take it back for analysis. In such captures there are
generally a few losses which make some traces unexploitable. But you know,
you have the same issue when you lose a segment which contains a request or
response message, as well as when you lose one segment containing a chunk
size in gzip-encoded payload.

In practice, network troubleshooting *rarely* happens on a single session and
in general you want the full stream to be able to give a diagnostic. I agree
with you that when compressing it's harder to resync than it is with clear
text. Right now, just tcpdump -X |less, scroll and stop when you notice
something which says "HTTP". But quite frankly, I'd rather spend my time
adapting the analysis tools than continuing to optimize HTTP parsers at
the assembly level to save every single CPU cycle in order to support rare
useless cases (eg: missing space after the color or missing CR before the
LF).

> Thanks for putting up with reading my concerns. 

Your concerns are valid, but they eventually will be addressed as the
protocol standardizes.

Regards,
Willy