Subject: Re: Chopan - Compressed HTTP Over PANs (draft-frank-6lowpan-chopan-00)

From: Julian Reschke <[email-masked]>
Date: Tue, 14 Jul 2009 09:38:19 +0200

Julian Reschke wrote:
> I just looked at
> 
>     <http://tools.ietf.org/html/draft-frank-6lowpan-chopan-00>
> 
> and I'm a bit concerned by the fact that this essentially profiles HTTP.
> ...

I forgot to mention that I discovered this draft reading about the APPS 
area activities in Stockholm.

There is a BarBof about "Applications in resource-constrained networks" 
(<http://trac.tools.ietf.org/area/app/trac/wiki/BarBofs/IETF75/6LowApp>), 
and a problem statement is outlined in 
<http://tools.ietf.org/html/draft-bormann-6lowpan-6lowapp-problem-01>.

BR, Julian

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 14 Jul 2009 11:03:28 +0200 (CEST)

> and I'm a bit concerned by the fact that this essentially profiles HTTP.

I agree with your concerns. I would even argue that this spec takes it away so 
far so this is no longer HTTP, but a binary protocol inspired by HTTP...

-- 

  / daniel.haxx.se

---

From: Julian Reschke <[email-masked]>
Date: Tue, 14 Jul 2009 09:33:14 +0200

I just looked at

For instance, it makes it impossible to use HTTP extensions that use 
"new" method names or status codes. Profiling like this has been a 
concern in the past, such as when the Atom WG discussed whether it could 
rely on DELETE and PUT due to restricted implementations/mappings that 
would only allow GET and POST -- so the problem may not be apparent 
right now considering deployed HTTP applications, but it will once 
people want to do new things.

In particular (just from a quick reading):

- HTTP extension methods not supported
- some standard headers are restricted in what they can carry (ETag, 
Cache-Control)
- some headers are listed as "standard", but do not seem to be 
registered ("Awake-Time"?)
- new headers are allowed to be stripped (that closes an important 
extension point)

I can see why the compression is optimized for common HTTP features, but 
it really shouldn't make extensions hard or impossible.

Best regards, Julian

---

From: Adrian Chadd <[email-masked]>
Date: Tue, 14 Jul 2009 17:06:49 +0800

On Tue, Jul 14, 2009, Daniel Stenberg wrote:

> I agree with your concerns. I would even argue that this spec takes it away 
> so far so this is no longer HTTP, but a binary protocol inspired by HTTP...

Which means if people want to write HTTP over libraries which implement this,
they'll essentially be tunnelling HTTP over this. :)

Adrian

---

From: Jim Gettys <[email-masked]>
Date: Tue, 14 Jul 2009 06:48:12 -0400

Those of you with memory of my role in HTTP may find the following 
comments surprising, but bear with me.

Doing something like this has to pass test, in my mind:
   o that it be shown to be significantly more compact than HTTP + 
deflate style compression (probably with a pre-defined dictionary, and 
canonicalization of cases of strings to minimize the size of the 
dictionary).  Ad-hoc binary compression systems are often/commonly not 
less efficient than what gzip style compressors can do.  (to remind the 
audience, deflate is gzip without some preamble information; and IIRC, 
it can be used with a predefined dictionary, so you don't have to 
transmit said dictionary first when you know the material being compressed).
   o that it (by such actions) remain fully compatible with HTTP.

This is much easier than having yet another HTTP parser lying around 
which isn't quite HTTP, with ad-hoc, non-extensible coding of HTTP to a 
fixed vocabulary.

Now, if the needs/goal are for much better compression than what you get 
that way, you really need to go beyond HTTP as a framework, where 
information in common across a sequence of requests is transferred in 
the first request and used in later requests (a statefull protocol, at 
least for the length of the conversation).  But then it isn't HTTP 
anymore, but maybe "http style", and you don't have constraints.  Of 
course, you then have other questions people will ask you, when you are 
really different...
                             - Jim

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 14 Jul 2009 13:00:56 +0200

On Jul 14, 2009, at 12:48, Jim Gettys wrote:

> that it be shown to be significantly more compact than HTTP

It is worth pointing out that saving bytes is one goal; saving code  
complexity is another significant goal.  Tight binary coding with few  
options is much easier to parse than a lenient text-based protocol on  
an 8-bit processor with 48 KiB of flash.

That said, whether the result of such an effort still is HTTP is the  
interesting question.

Gruesse, Carsten

---

From: Jim Gettys <[email-masked]>
Date: Tue, 14 Jul 2009 07:19:05 -0400

Go read the draft more closely.

The premise is, low power, low bandwidth personal area networks.

Bytes translate into power, latency and bandwidth used.

So 3 strikes, and conventional text based HTTP is out, in my mind.  HTTP 
is *way* more verbose than almost any binary encoding (by a very large 
factor).

Lots of uses of HTTP are for things other than transporting large 
compressed images, in which the image transport dominates.  Don't 
extrapolate general web traffic to the intended applications.

So I don't think that arguing something else may be needed is really 
questionable.  I was arguing, however, that: 1) first explore completely 
  compatible and extensible encoding; 2) if that isn't compact enough, 
then relax compatibility constraints and go to a statefull protocol. 
(like we explored over a decade ago, but which have other issues).

I suspect 1) will be sufficient for the stated goals, because when you 
are tethered by a low bandwidth net, you always have enough cycles 
around.  It's when both performance and compactness are key that 2) 
really becomes reasonable.  HTTP parsers are way slower than simpler, 
better engineered protocols can be without the legacy than the old 
email/mime/http framework carries.
                            - Jim

J Ross Nicoll wrote:
> Jim Gettys wrote:
> I'd also want it shown that it's a saving even worth having. It seems a
> tiny saving in terms of bandwidth used, even on a low speed network. The
> specification talks about memory footprint as well ("Binary compression:
> HTTP headers are compressed into a binary format to save bandwidth and
> buffer space"), but I still find it hard to believe anything for which
> 100-200 bytes (if that) will make a difference, will be able to do
> anything sensible with the content.
>

---

From: J Ross Nicoll <[email-masked]>
Date: Tue, 14 Jul 2009 12:06:16 +0100

---

From: Jim Gettys <[email-masked]>
Date: Tue, 14 Jul 2009 07:26:28 -0400

Carsten Bormann wrote:
> options is much easier to parse than a lenient text-based protocol on an 
> 8-bit processor with 48 KiB of flash.
> 

Exactly.  Though it's not clear that the stated devices are having to 
parse the protocol, rather than just a client implementation generating 
a small subset of HTTP.

Of course, it also isn't clear how many people still use 8 bit 
processors with 48K of flash (which aren't probably running an IPv6 
stack, either).

And yes, HTTP parsing is radically more complex and takes a lot more 
code than roughly equivalent binary code frameworks.  But you don't have 
  to write them from scratch, either.  And for reasons I've never 
understood, some folks seem to think writing such binary protocols is 
hard.  They are just different (though many binary protocols have made 
insufficient/incorrect provisions for extensibility). X11, for example, 
made some (non-fatal) mistakes in that area.
                           - Jim

---

From: J Ross Nicoll <[email-masked]>
Date: Tue, 14 Jul 2009 10:37:33 +0000

Agreed, this looks like it would be far better suited as an HTTP-like
protocol, supporting key functionality, rather than trying (and failing)
to actually implement all of HTTP. Personally I'm unconvinced that any
sensor so limited in functionality that it needs to use a reduced HTTP
alternative for communication is going to be able to do anything
meaningful with the data moved around...

Also, this appears to be yet another protocol that uses UDP and then has
to work around the lack of TCP features (reliable delivery, in-order
delivery) it needs...

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 15 Jul 2009 16:40:46 +1000

This reminds me very much of the long-running discussions (and that's  
being polite) of Binary XML in the W3C.
   http://www.w3.org/XML/Binary/
   http://www.xml.com/pub/a/2003/08/13/deviant.html

Making an interchangeable binary representation of a flexible text- 
based format that beats gzip on size with full fidelity is a high bar.

"Inspired by HTTP" may indeed be a more reasonable goal.

Cheers,

On 14/07/2009, at 8:48 PM, Jim Gettys wrote:

> deflate style compression (probably with a pre-defined dictionary,  
> and canonicalization of cases of strings to minimize the size of the  
> dictionary).  Ad-hoc binary compression systems are often/commonly  
> not less efficient than what gzip style compressors can do.  (to  
> remind the audience, deflate is gzip without some preamble  
> information; and IIRC, it can be used with a predefined dictionary,  
> so you don't have to transmit said dictionary first when you know  
> the material being compressed).
> which isn't quite HTTP, with ad-hoc, non-extensible coding of HTTP  
> to a fixed vocabulary.
>
> Now, if the needs/goal are for much better compression than what you  
> get that way, you really need to go beyond HTTP as a framework,  
> where information in common across a sequence of requests is  
> transferred in the first request and used in later requests (a  
> statefull protocol, at least for the length of the conversation).   
> But then it isn't HTTP anymore, but maybe "http style", and you  
> don't have constraints.  Of course, you then have other questions  
> people will ask you, when you are really different...
>                            - Jim
>
>
> Julian Reschke wrote:
>> I just looked at
>> For instance, it makes it impossible to use HTTP extensions that  
>> use "new" method names or status codes. Profiling like this has  
>> been a concern in the past, such as when the Atom WG discussed  
>> whether it could rely on DELETE and PUT due to restricted  
>> implementations/mappings that would only allow GET and POST -- so  
>> the problem may not be apparent right now considering deployed HTTP  
>> applications, but it will once people want to do new things.
>> - some standard headers are restricted in what they can carry  
>> (ETag, Cache-Control)
>> I can see why the compression is optimized for common HTTP  
>> features, but it really shouldn't make extensions hard or impossible.
>> Best regards, Julian
>>
>
>

--
Mark Nottingham     http://www.mnot.net/