Subject: [6lowapp] HTTP and SIP

From: Shidan <[email-masked]>
Date: Sat, 10 Oct 2009 13:30:04 -0400

I'm going through the 6LoWAPP problem statement:

"request-response protocols like HTTP may require battery-operated, mostly
sleeping nodes to be listening for requests much more frequently than their
application processing requirements alone would need them to wake up."

"In addition, some of the applications may require optimizations in terms of
bandwidth usage; for example, the usual data formats (both headers and body)
are perceived to be too chatty for the 50-60 byte payloads possible in
LoWPANs.  Their interpretation and generation may require too much code for
the 8-bit and 16-bit processors dominating LoWPAN nodes."

None of these points seem evident to me. We are in co-operation with two
multinationals and faculty at Columbia University implementing,
unadulterated SIP for smart meter PLC communications and I'm
personally trying to steer the group to do a proof of concept of running our
sub 40K stack over 6 LoWPAN on Contiki.

Going back to the problem statement, definitely the second point is only
valid if considering XML .

For the first point, if we are assuming there is a gateway present that
isn't constrained by power requirements then the ways to tackle this issue
become wider.

My bias is not towards SIP, some sort of adulterated HTTP or anything else,
but I think anyone in this space should consider the fact that a SIP based
architecture will be the defacto standard for mobile networks, which is
already the most ubiquitos system out there. Thats why I'm interested in
both SIP for this space.

Its one thing standardizing how to implement application layers over 6LoWPAN
but standardizing around specific REST architecures, SIP architectures,
desription logics or anything else at this point without understanding
requirements is quite unreasonable.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 10 Oct 2009 20:13:55 +0200

On Oct 10, 2009, at 19:30, Shidan wrote:

> SIP for smart meter PLC communications

As I told everyone who is bringing some technology to the table, I'd  
suggest you write an Internet-Draft.
In this case, it should at least cover what problem SIP solves and  
what data protocols you expect to initiate/transport with it.

I would like to take the opportunity to remind everyone that the  
deadline for new Internet-Drafts for Hiroshima is October 19 (24:00  
UTC), please see http://www.ietf.org/meeting/cutoff-dates.html#76

Gruesse, Carsten

---

From: Adam Dunkels <[email-masked]>
Date: Sun, 11 Oct 2009 00:08:28 +0200

Shidan wrote:
> "request-response protocols like HTTP may require battery-operated, 
> mostly sleeping nodes to be listening for requests much more frequently 
> than their application processing requirements alone would need them to 
> wake up."
>  
> "In addition, some of the applications may require optimizations in 
> terms of bandwidth usage; for example, the usual data formats (both 
> headers and body) are perceived to be too chatty for the 50-60 byte 
> payloads possible in LoWPANs.  Their interpretation and generation may 
> require too much code for the 8-bit and 16-bit processors dominating 
> LoWPAN nodes."
>  
> None of these points seem evident to me. 

I very much agree with this. By so quickly dismissing the opportunities 
with existing protocols such as HTTP and SIP, this group may miss 
reaching widespread adoption of its protocols.

Both these arguments were frequently used in the past against IP for 
resource-constrained systems, yet today we take IP for granted for these 
systems despite the overheads of IP. We need to a pay price for 
interoperability and flexibility, but we think it is worth it.

FWIW, we've recently been working with REST web services 
(REST/HTTP/TCP/IP) over 802.15.4 with a low-power duty-cycling radio 
protocol (X-MAC) for Contiki. The implementation of the application 
layers (REST/HTTP) has roughly a 5kb code footprint. The performance is 
not bad: a full HTTP/TCP/IP REST transaction with sensor data in XML 
format is completed in roughly 0.5 seconds over a one-hop radio link and 
in less than 2 seconds over a four-hop radio network:

http://www.sics.se/~adam/yazar09efficient.pdf

Note that this is without any optimizations at the REST/HTTP/TCP/IP 
layers. Full HTTP headers are sent from the curl client, reducing the 
overall performance. Full XML is sent from the sensors. Thus there is 
plenty of opportunity to improve this without moving away from the 
HTTP/TCP/IP framework. For example, as we show in the above paper, there 
are significant savings in using conditional HTTP GETs when data have 
not changed between subsequent requests.

Like Shidan, I am not saying that we necessarily must use full-fledged 
REST/HTTP/TCP/IP or SIP for everything in our resource-constrained 
networks. But we mustn't so quickly rule out existing mechanisms just 
because of their perceived overhead. Interoperability and flexibility 
are very important properties, often more important than bit-level 
optimizations.

/adam
-- 
Adam Dunkels <[email-masked]>, +46707731614
http://twitter.com/adunk | http://www.sics.se/~adam/

---

From: "Don Sturek" <[email-masked]>
Date: Sat, 10 Oct 2009 17:56:43 -0700

Hi Adam.

.5 seconds for one hop and 2 seconds for 4 hops are really not acceptable
for many applications.

We should of course investigate using existing protocols.   However it does
not seem likely that full HTTP and XML deployed over devices like IEEE
802.15.4 with very small packet sizes will scale very well  (certainly, for
any application expecting to do home or building controls, those latencies
don't work).  Ideally, we should find something that allows for full
HTTP/etc. while enabling constrained devices to operate with a solution that
can be expanded/compressed.

Don

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Adam Dunkels
Sent: Saturday, October 10, 2009 3:08 PM
To: Shidan
Cc: [email-masked]
Subject: Re: [6lowapp] HTTP and SIP

/adam
-- 
Adam Dunkels <[email-masked]>, +46707731614
http://twitter.com/adunk | http://www.sics.se/~adam/
_______________________________________________
6lowapp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/6lowapp

---

From: Shidan <[email-masked]>
Date: Sat, 10 Oct 2009 21:40:41 -0400

Hi Don, I certainly don't disagree with you that we need better latency and
I'm pretty sure Adam doesn't either as he said this is without ANY
optimizations. Given that fact, his statement is quite impressive.
The key here is to actually back up any ideas and opinions with real results
and this is what we are in the process of doing.

Right now for our purposes, in the Smart Grid domain, we can run SIP, with
some minor changes beyond stack size optimizations, over networks with as
little bandwidth as 80kbps and highly constrained devices. We also
acknowledge for our use cases, latency is not really an issue.

I'm pretty convinced that with the RIGHT sort of optimizations, any latency
issues can be handled without having to dictating general architectural
standards like using REST, SNMP, SIP, etc. which can stifle innovation and
lead to building everything with a hammer.

Having said that Carsten's note is encouraging and I'm pretty convinced this
committee has in mind creating a base that will evolve
without stifling innovation and take real results and proof of concepts of
use cases as main deciding factors. We will definitely be submitting an ID
for Hiroshima.

--
Shidan Gouran

---

From: Jonathan Hui <[email-masked]>
Date: Sat, 10 Oct 2009 19:39:57 -0700

On Oct 10, 2009, at 3:08 PM, Adam Dunkels wrote:

>> mostly sleeping nodes to be listening for requests much more  
>> frequently than their application processing requirements alone  
>> would need them to wake up."
>> payloads possible in LoWPANs.  Their interpretation and generation  
>> may require too much code for the 8-bit and 16-bit processors  
>> dominating LoWPAN nodes."
> I very much agree with this. By so quickly dismissing the  
> opportunities with existing protocols such as HTTP and SIP, this  
> group may miss reaching widespread adoption of its protocols.

I also agree that we should not be so quick to come to conclusions  
without presentation of real empirical evidence.

> resource-constrained systems, yet today we take IP for granted for  
> these systems despite the overheads of IP. We need to a pay price  
> for interoperability and flexibility, but we think it is worth it.

I'd be careful carrying the interoperability argument towards the  
upper layers.  IP was designed to be the narrow waist.  Today, that  
has evolved to UDP and TCP being the narrow waist [1].  We have far  
more flexibility and opportunity at the application layer than we do  
with TCP/UDP/IP and we should not be afraid to exploit that.  I do  
support the argument for evaluating existing protocols/architectures,  
but more because they are established, vetted, and widely understood,  
less so because we actually want to utilize the existing technical  
infrastructure that surrounds it.

[1] http://tools.ietf.org/html/draft-rosenberg-internet-waist-hourglass

--
Jonathan Hui

---

From: "[email-masked]" <[email-masked]>
Date: Sun, 11 Oct 2009 14:59:04 +0300

> [1] http://tools.ietf.org/html/draft-rosenberg-internet-waist-hourglass

I totally agree. 

- Zach

---

From: "[email-masked]" <[email-masked]>
Date: Sun, 11 Oct 2009 15:17:05 +0300

> lead to building everything with a hammer.

I think there is confusion about the word REST in the charter. This 
is only meant to mean that if you want to proxy with HTTP and support 
the design of web-services reusing existing techniques - then we need to 
support GET, PUT, POST, DELETE methods. Maybe it is better to 
remove the word REST if that implies anything more than that to people? 

The purpose of the application protocol work item description was exactly
to 
say that we are not limiting ourselves to one single paradigm. Instead we 
need to integrate features from suitable protocols that allows us to 
innovate yet at the same time easily proxy into HTTP, SIP and SNMP.

My view is: 

- For applications where full HTTP, SIP, SNMP etc. 
can be used - then you are lucky - and you don't necessarily need this 
working group. If you just need to optimize one of the above, then 
there are appropriate working groups for that already.

- There are definitely applications, devices and networks for which 
current solutions are not suitable. 60 byte payloads, real problems with 
TCP over wireless and mobile networks and very limited bandwidth over 
multiple wireless hops in e.g. a LoWPAN are very real problems. The 
goal of 6lowapp should be to provide an application protocol totally 
suitable to this domain - not just a band-aid.

What I would like to see is that the solution we aim at in 6lowapp is 
able to easily be proxied into protocols that dominate applications in 
energy, building automation and M2M. Today this is mainly HTTP, but we 
see some SIP and SNMP as well. For this reason the work item features 
include both features from HTTP (Req-Res, REST methods) as well as from 
e.g. SIP (eventing) and e.g. peer-to-peer which is key for OASIS and
building
automation in general. The support of URIs and content-types are common 
to all. 

> use cases as main deciding factors. We will definitely be submitting an
ID

Great - and any more feedback on improving the charter text would be 
welcome too!

Zach

frequently

---

From: Brian Frank <[email-masked]>
Date: Sun, 11 Oct 2009 09:29:45 -0400

> I also agree that we should not be so quick to come to conclusions without
> presentation of real empirical evidence.
>

I agree.  One of the first things we need to do is justify that a new
protocol is required using real empirical data that shows TCP and text based
HTTP do not work for the problem domain.  For example, how does HTTP over
TCP stack up against binary UDP protocols with regard to latency, bandwidth,
and power consumption of battery powered sensors?

---

From: "Don Sturek" <[email-masked]>
Date: Sun, 11 Oct 2009 15:18:35 -0700

Hi Shidan,

One more issue:     We have existing smart energy messaging using a
proprietary binary protocol.  We then created an XML structure for these
messages then used W3C EXI to tokenize.  Here is what we ended up with:

Binary Protocol Message Size:    6 Bytes

XML Message Size:  376 Bytes

W3C EXI Tokenized XML Message Size:  12 Bytes.

The issue you run into is this:  If you have a data collector model where
many messages are forwarded from leaf nodes up to a central node (like the
ROLL DAG root for example), then you find that the nodes close to the root
have dramatically lower throughput due to congestion from packet forwarding
from the leaf nodes towards the popular data collector device.  We have many
deployments where the IEEE 802.15.4 effective throughput of 140Kpbs turns
into 9Kpbs as you forward towards the data collector.  These deployments are
using a binary protocol where nearly all messages fit into a single packet.
Now, consider what happens where each and every message turns into MANY
packet exchanges.  You will quickly see this does not work..

I agree with what Zach wrote in a later note.    I don't want to dissuade
anyone who finds HTTP or SIP working over IEEE 802.15.4 from using it.
However, I think the problem statement in 6LowAPP stands at least for the
smart energy application I am familiar with.

Don

From: Shidan [mailto:[email-masked]] 
Sent: Saturday, October 10, 2009 6:41 PM
To: [email-masked]
Cc: Adam Dunkels; [email-masked]
Subject: Re: [6lowapp] HTTP and SIP

Having said that Carsten's note is encouraging and I'm pretty convinced this
committee has in mind creating a base that will evolve without stifling
innovation and take real results and proof of concepts of use cases as main
deciding factors. We will definitely be submitting an ID for Hiroshima.

--

Shidan Gouran

Hi Adam.

Don

-----Original Message-----

/adam

---

From: Shidan <[email-masked]>
Date: Sun, 11 Oct 2009 19:18:49 -0400

Thanks for this detailed explanation Don and your figures with regards to
XML is very sound here.
However for the purposes of the project I'm involved with, we've looked at
the other side of the meter and the sessions we are defining with SIP are
C12.19 sessions and very simple SIP messaging. The great thing is we can
meld the control of meters with advanced communications infrastructure and
take "unified communications", the concept of device mobility, security and
accountability to another level. One of our partner companies has written an
informative whitepaper on this architecture and if we get their permission
we will send a link to the list if you are interested.

Now going back to the home, and this is just my personal opinion, I don't
believe standards like ZigBee are the right choice because they do indeed
lock one to a particular layer 2 technology. However, I think they've done
some great work on the application side and the application objects,
profiles and such they have defined would take care of the issue you had
without having to necessarily modify SIP or HTTP.

Again, I'm not saying that a need for a whole new application protocol is
warranted or not warranted as a fact, I'm just saying this is not evident to
me and isn't a trivial assumption or something that can be reasoned without
empirical evidence and considering some clear use cases. Lets change/modify
what really needs to be changed based on the empirical evidence.

Also unrelated, In response to Zach's note, we will be more than happy to
lend a hand in helping define a standard that can map cleanly to SIP which
might be useful for interfacing with even other applications such as XMPP
which some leading demand reponse companies like Enernoc are using on the
industrial side already. I will be engaging people in our group early next
week and I'm sure they will be interested in working on this as well.

On Sun, Oct 11, 2009 at 6:18 PM, Don Sturek <[email-masked]> wrote:

> packet exchanges.  You will quickly see this does not work……
>
>
>
> I agree with what Zach wrote in a later note.    I don’t want to dissuade
> *From:* Shidan [mailto:[email-masked]]
> *Sent:* Saturday, October 10, 2009 6:41 PM
> *To:* [email-masked]
> *Cc:* Adam Dunkels; [email-masked]
>
> *Subject:* Re: [6lowapp] HTTP and SIP
>
>
>
> The key here is to actually back up any ideas and opinions with real
> results and this is what we are in the process of doing.
>
>
>
> Having said that Carsten's note is encouraging and I'm pretty convinced
> this committee has in mind creating a base that will evolve

---

From: Shidan <[email-masked]>
Date: Sun, 11 Oct 2009 20:58:47 -0400

And just to be specific, when I'm talking about ZigBee, I'm referring in
specific to their defined data models and representations, not the whole
application architecture.

On Sun, Oct 11, 2009 at 7:18 PM, Shidan <[email-masked]> wrote:

>> Hi Don, I certainly don't disagree with you that we need better latency
>> and I'm pretty sure Adam doesn't either as he said this is without ANY
>> I'm pretty convinced that with the RIGHT sort of optimizations, any
>> latency issues can be handled without having to dictating general
>> architectural standards like using REST, SNMP, SIP, etc. which can stifle
>> innovation and lead to building everything with a hammer.
>>
>>
>>

---

From: Adam Dunkels <[email-masked]>
Date: Mon, 12 Oct 2009 13:14:22 +0200

[email-masked] wrote:
> latency

Using the acronym REST may be a little difficult because many strongly 
associate it with HTTP, even if the actual architectural principles do 
not necessarily have to be implemented over HTTP.

/adam

---

From: Adam Dunkels <[email-masked]>
Date: Mon, 12 Oct 2009 13:18:41 +0200

Brian Frank wrote:
> protocol is required using real empirical data that shows TCP and text 
> based HTTP do not work for the problem domain.  For example, how does 
> HTTP over TCP stack up against binary UDP protocols with regard to 
> latency, bandwidth, and power consumption of battery powered sensors?

HTTP/TCP is actually quite lean in terms of latency and bandwidth 
compared to a protocol with similar functionality run on top of UDP - at 
least if we remove most of the HTTP headers that modern HTTP clients and 
servers send.

It is easy to view TCP as a complex beast and that something else would 
be more lightweight. But upon closer inspection, TCP isn't such a 
resource hog as it initially seems.

That said, the HTTP and/or TCP interaction models may not be right for 
all applications in 6lowapp, but we should be wary of reinventing TCP 
over UDP for those cases where TCP-like functionality is needed.

/adam

---

From: Adam Dunkels <[email-masked]>
Date: Mon, 12 Oct 2009 13:19:40 +0200

> can be expanded/compressed.

I agree with this, and that's also why I think the 6lowapp initiative is 
important.

As a sidenote, though: the 0.5 seconds and 2 seconds are not latencies, 
but completion time. Latency about 0.3s per hop in this case, which is 
due to the power-saving radio mechanism we use, which switches the radio 
off for 99% of the time, while waking up twice per second. So the 
HTTP/TCP does not incur the significant part of the latency in this case.

/adam

---

From: Adam Dunkels <[email-masked]>
Date: Mon, 12 Oct 2009 13:22:30 +0200

Jonathan Hui wrote:
>> these systems despite the overheads of IP. We need to a pay price for 
> I'd be careful carrying the interoperability argument towards the upper 
> layers.  IP was designed to be the narrow waist.  Today, that has 
> evolved to UDP and TCP being the narrow waist [1].  We have far more 
> flexibility and opportunity at the application layer than we do with 
> TCP/UDP/IP and we should not be afraid to exploit that.  I do support 
> the argument for evaluating existing protocols/architectures, but more 
> because they are established, vetted, and widely understood, less so 
> because we actually want to utilize the existing technical 

I fully agree with you in that these arguments are stronger than the 
interoperability argument, and the application layer is the place were 
we are free to do what we please, without being constrained from the 
network infrastructure, and that we should use that fact to our 
advantage. (Even though the waist nowadays sometimes is IP/TCP port 80 
or even IP/TCP/HTTP port 80.)

But we shouldn't overlook the interoperability aspects with existing 
application-layer infrastructure such as SNMP, for which there are 
numerous tools and systems available, and HTTP, that has both 
infrastructure (e.g. caching proxies) and deployed systems.

I am not implying that we must necessarily use existing protocols for 
everything in 6lowapp. It is dangerous (and easy) to become too fixated 
with existing mechanisms, particularly in cases where they clearly are 
not applicable. But rejecting them prematurely based on perceived 
qualities and not empirical data is not good.

/adam

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 12 Oct 2009 13:22:34 +0200

On Oct 12, 2009, at 13:14, Adam Dunkels wrote:

> Using the acronym REST may be a little difficult because many  
> strongly associate it with HTTP, even if the actual architectural  
> principles do not necessarily have to be implemented over HTTP.

I think that is indeed the interesting part, nicely isolated as such  
by Roy Fielding's dissertation.
So, the $64000 question:
What are the architectural principles of SIP, SNMP, XMPP, etc. that we  
are interested in for a constrained-node/-network transfer protocol?

Gruesse, Carsten

---

From: Zach Shelby <[email-masked]>
Date: Mon, 12 Oct 2009 14:28:47 +0300

On Oct 12, 2009, at 14:18 , Adam Dunkels wrote:

> That said, the HTTP and/or TCP interaction models may not be right  
> for all applications in 6lowapp, but we should be wary of  
> reinventing TCP over UDP for those cases where TCP-like  
> functionality is needed.

I totally agree. This is why we are aiming at support for both UDP and  
TCP as transports for 6lowapp in the charter. There are interaction  
models for embedded devices where UDP is great. For example for very  
short transactions, multicast, or even where no response is needed.  
There are also models where TCP is great, and situations where it  
really works too.

In the future this activity hopefully spins off a transport area  
working group that can really make an improvement/alternative to TCP  
for these types of applications and networks.  We already identified  
this in this first Bar BOF - but it was made clear that this is a long  
term process and something to do in parallel.

Zach

-- 
http://www.sensinode.com
http://zachshelby.org - My blog “On the Internet of Things”
Mobile: +358 40 7796297

Zach Shelby
Head of Research
Sensinode Ltd.
Kidekuja 2
88610 Vuokatti, FINLAND

This e-mail and all attached material are confidential and may contain  
legally privileged information. If you are not the intended recipient,  
please contact the sender and delete the e-mail from your system  
without producing, distributing or retaining copies thereof.