Subject: [RSN] Re: [sp100] interoperability problems at ISA Houston

From: Kris Pister <[email-masked]>
Date: Tue, 30 Oct 2007 09:18:43 -0700

Here's a thread from ISA that may be of interest.  6lowpan had some 
email a few months ago on the topic of coexistence, and the level of 
safety afforded by the dispatch byte.  If you read Gerry's email below, 
you'll see that once again this approach has proven hazardous in a 
public forum.  I'd say at least half of the WSN companies that I've 
talked to have fallen victim to this (Dust included - back in 2003!).

ksjp

Kris Pister wrote:
> This is an example of a very common problem seen in the early stages of software development in sensor networks.  Wireless HART does not, and SP100.11a should not, suffer from it.
> The general problem is that those unfamiliar with working in a regulated but unlicensed band are not used to the idea that there is a lot of traffic flying around at 2.4GHz, employing a wide range of MAC protocols.  More subtly, even well-formed packets in your own network can be corrupted in flight in such a way that the frame check sequence (FCS) does not catch the errors.  In both cases, the naive system will try to execute the instructions in a packet that is essentially random, with disastrous results.
> In TSMP (the basis of Wireless HART and SP100.11a data link layers), the solution to this problem is to use a message integrity code (MIC) on every packet.  This 32 bit number is a cryptographic hash of the entire packet, and prevents corrupted or foreign packets from being accepted at the lowest layer of the communication stack, with a probability of 99.99999998% (fewer than one in 4 billion get through on average). For a reliable network, even this is not enough, since undetected errors might occur on the order of once per year per network.
> Another 32 bit MIC is used at the application layer to verify the authenticity of the sender of the message.  Combined with the DLL MIC, this reduces the probability of executing random packets to something like once per age-of-the-universe per network.  Even with energy scavenging, that's a long time! :)
>  
> The lesson here is simple: never *ever* turn off crypto in your networks.  Use well-known published keys if you want to, but keep the integrity codes in place.
>  
> ksjp
> --
> Prof. EECS, UC Berkeley
> Founder & CTO, Dust Networks
>  
>
> ________________________________
>
> From: Gerry Nadler [mailto:[email-masked]]
> Sent: Mon 10/29/2007 10:31 AM
> To: ISA100
> Subject: [sp100] interoperability problems at ISA Houston
>
>
> Everyone:
>
>  
>
> On the evening before the ISA exhibition (Monday night) I visited the show floor to see the ISA-100 booth.   Most all the booths were unmanned and all was quiet except for a small group in one booth that was having problems.  They were using Zigbee.   Their Zigbee system was not functioning due to some type of interference.  (The name of the company is not mentioned here because for the purpose of this note it is not necessary).  I walked by the booth and the people were very frustrated as to the cause of their problem.   I tried to help.  We set up a 15.4 packet sniffer to look at the packets to see what was the cause of the problem.  The vendor was using Ember chips and software stack.  We got the sniffer running and looked at the packets and we saw the problem.  There was nothing we could do because the 15.4 stack was being corrupted by an illegal packet(or so I thought at the time).  The packets were sent to the Ember software team for analysis and they made modification to their stack and the vendor was up and running by the time the show opened on Tuesday.
>
>  
>
> I walked around the show floor at about 11 PM at night and looked to see who might be transmitting 15.4 packets.  I checked the ISA-100 booth and the surrounding area and everything seemed to be shut down.  I walked up and down every aisle looking for a possible transmission site.   I didn't see anything except for the wireless HART booth where there was "flashing LEDs", a possible sign of RF transmissions.  It could have been another Zigbee/SP-100 transmission site as well.
>
>  
>
> On Tuesday I stopped by the booth and Bob LeFort the president of Ember was at the booth.  We had a nice chat and when I got back home I asked him if he could have his software people write up what they found.  Below is a short analysis of the problem written by the VP of engineering at Ember.  They did a great job and had the problem diagnosed and fixed before Houston show opened the next day.
>
>  
>
> I have included this because depending on your point of view this may be harbinger of things to come or it  can be dismissed..  I thought the group should know what we may or may not expect in the future.
>
>  
>
> Gerry
>
>  
>
> Ember report by Skip Ashton, Ember VP Engineering. 
>
>  
>
> The cause of the problem was an Ember device receiving a coordinator realignment packet.  Under the 15.4 specification, this packet is intended to be unicast to an orphan device, or broadcast to a network that is changing its parameters.  Receipt of this packet was causing the Ember device to change channels and PAN ID.  The 15.4 specification requires this packet be sent with a 64 bit extended address for the source.  However, in this case the 16 bit coordinator address for the source was being used making this an illegal packet.  This illegal packet to change PAN ID was being generated by another network device at a regular interval.
>
>  
>
> Reception and processing of the coordinator realignment command is mandatory within 802.15.4.  However, it should be ignored by devices that are using higher levels of control over items such as channel change.  A ZigBee network device is not expecting a coordinator realignment.  This command is not used to change channels within ZigBee, because a higher layer network command is used instead.  As such, these type of MAC commands should be ignored.
>
>  
>
> A change was made to our stack to ignore such MAC command frames once the ZigBee stack is operational.  
>
>  
>
> --- You are currently subscribed to sp100 as: [email-masked] To unsubscribe send a blank email to [email-masked] 
>   

_______________________________________________
RSN mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rsn

---

From: Jonathan Hui <[email-masked]>
Date: Tue, 30 Oct 2007 10:01:29 -0700

Kris, I think you're a bit confused on the purpose of the 6lowpan header 
type. The header type is completely orthogonal to whether people choose 
to utilize strong integrity checks. The dispatch type is analogous to 
the Next Header field in IP packets. It gives flexibility in 
constructing different header stacks, yet is easy to parse. Its what 
enables IPv6 to support extension headers.

Robust networks always need strong integrity checks, but integrity 
checks don't replace dispatch types.

--
Jonathan Hui

Kris Pister wrote:
>> This is an example of a very common problem seen in the early stages 
>> of software development in sensor networks.  Wireless HART does not, 
>> and SP100.11a should not, suffer from it.
>> The general problem is that those unfamiliar with working in a 
>> regulated but unlicensed band are not used to the idea that there is a 
>> lot of traffic flying around at 2.4GHz, employing a wide range of MAC 
>> protocols.  More subtly, even well-formed packets in your own network 
>> can be corrupted in flight in such a way that the frame check sequence 
>> (FCS) does not catch the errors.  In both cases, the naive system will 
>> try to execute the instructions in a packet that is essentially 
>> random, with disastrous results.
>> In TSMP (the basis of Wireless HART and SP100.11a data link layers), 
>> the solution to this problem is to use a message integrity code (MIC) 
>> on every packet.  This 32 bit number is a cryptographic hash of the 
>> entire packet, and prevents corrupted or foreign packets from being 
>> accepted at the lowest layer of the communication stack, with a 
>> probability of 99.99999998% (fewer than one in 4 billion get through 
>> on average). For a reliable network, even this is not enough, since 
>> undetected errors might occur on the order of once per year per network.
>> Another 32 bit MIC is used at the application layer to verify the 
>> authenticity of the sender of the message.  Combined with the DLL MIC, 
>> this reduces the probability of executing random packets to something 
>> like once per age-of-the-universe per network.  Even with energy 
>> scavenging, that's a long time! :)
>>  
>> The lesson here is simple: never *ever* turn off crypto in your 
>> networks.  Use well-known published keys if you want to, but keep the 
>> integrity codes in place.
>>  
>> ksjp
>> -- 
>> Prof. EECS, UC Berkeley
>> On the evening before the ISA exhibition (Monday night) I visited the 
>> show floor to see the ISA-100 booth.   Most all the booths were 
>> unmanned and all was quiet except for a small group in one booth that 
>> was having problems.  They were using Zigbee.   Their Zigbee system 
>> was not functioning due to some type of interference.  (The name of 
>> the company is not mentioned here because for the purpose of this note 
>> it is not necessary).  I walked by the booth and the people were very 
>> frustrated as to the cause of their problem.   I tried to help.  We 
>> set up a 15.4 packet sniffer to look at the packets to see what was 
>> the cause of the problem.  The vendor was using Ember chips and 
>> software stack.  We got the sniffer running and looked at the packets 
>> and we saw the problem.  There was nothing we could do because the 
>> 15.4 stack was being corrupted by an illegal packet(or so I thought at 
>> the time).  The packets were sent to the Ember software team for 
>> analysis and they made modification to their stack and the vendor was 
>> up and running by the time the show opened on Tuesday.
>>
>>  
>>
>> I walked around the show floor at about 11 PM at night and looked to 
>> see who might be transmitting 15.4 packets.  I checked the ISA-100 
>> booth and the surrounding area and everything seemed to be shut down.  
>> I walked up and down every aisle looking for a possible transmission 
>> site.   I didn't see anything except for the wireless HART booth where 
>> there was "flashing LEDs", a possible sign of RF transmissions.  It 
>> could have been another Zigbee/SP-100 transmission site as well.
>>
>>  
>>
>> On Tuesday I stopped by the booth and Bob LeFort the president of 
>> Ember was at the booth.  We had a nice chat and when I got back home I 
>> asked him if he could have his software people write up what they 
>> found.  Below is a short analysis of the problem written by the VP of 
>> engineering at Ember.  They did a great job and had the problem 
>> diagnosed and fixed before Houston show opened the next day.
>>
>>  
>>
>> I have included this because depending on your point of view this may 
>> be harbinger of things to come or it  can be dismissed..  I thought 
>> the group should know what we may or may not expect in the future.
>>
>>  
>>
>> Gerry
>>
>>  
>>
>> The cause of the problem was an Ember device receiving a coordinator 
>> realignment packet.  Under the 15.4 specification, this packet is 
>> intended to be unicast to an orphan device, or broadcast to a network 
>> that is changing its parameters.  Receipt of this packet was causing 
>> the Ember device to change channels and PAN ID.  The 15.4 
>> specification requires this packet be sent with a 64 bit extended 
>> address for the source.  However, in this case the 16 bit coordinator 
>> address for the source was being used making this an illegal packet.  
>> This illegal packet to change PAN ID was being generated by another 
>> network device at a regular interval.
>>
>>  
>>
>> Reception and processing of the coordinator realignment command is 
>> mandatory within 802.15.4.  However, it should be ignored by devices 
>> that are using higher levels of control over items such as channel 
>> change.  A ZigBee network device is not expecting a coordinator 
>> realignment.  This command is not used to change channels within 
>> ZigBee, because a higher layer network command is used instead.  As 
>> such, these type of MAC commands should be ignored.
>>
>>  
>>
>> --- You are currently subscribed to sp100 as: [email-masked] 
>> To unsubscribe send a blank email to [email-masked]   
> 
> 
> 6lowpan mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/6lowpan

---

From: Kris Pister <[email-masked]>
Date: Tue, 30 Oct 2007 14:31:40 -0700

Jonathan - funny, that's just what I was saying a few months ago :)

As long as everyone agrees with Jonathan, all is well.

ksjp

Jonathan Hui wrote:
>
> Kris, I think you're a bit confused on the purpose of the 6lowpan 
> header type. The header type is completely orthogonal to whether 
> people choose to utilize strong integrity checks. The dispatch type is 
> analogous to the Next Header field in IP packets. It gives flexibility 
> in constructing different header stacks, yet is easy to parse. Its 
> what enables IPv6 to support extension headers.
>
>> safety afforded by the dispatch byte.  If you read Gerry's email 
>> below, you'll see that once again this approach has proven hazardous 
>> in a public forum.  I'd say at least half of the WSN companies that 
>> I've talked to have fallen victim to this (Dust included - back in 
>> 2003!).
>>
>> ksjp
>>
>> Kris Pister wrote:
>>> regulated but unlicensed band are not used to the idea that there is 
>>> a lot of traffic flying around at 2.4GHz, employing a wide range of 
>>> MAC protocols.  More subtly, even well-formed packets in your own 
>>> network can be corrupted in flight in such a way that the frame 
>>> check sequence (FCS) does not catch the errors.  In both cases, the 
>>> naive system will try to execute the instructions in a packet that 
>>> is essentially random, with disastrous results.
>>> the solution to this problem is to use a message integrity code 
>>> (MIC) on every packet.  This 32 bit number is a cryptographic hash 
>>> of the entire packet, and prevents corrupted or foreign packets from 
>>> being accepted at the lowest layer of the communication stack, with 
>>> a probability of 99.99999998% (fewer than one in 4 billion get 
>>> through on average). For a reliable network, even this is not 
>>> enough, since undetected errors might occur on the order of once per 
>>> year per network.
>>> authenticity of the sender of the message.  Combined with the DLL 
>>> MIC, this reduces the probability of executing random packets to 
>>> something like once per age-of-the-universe per network.  Even with 
>>> energy scavenging, that's a long time! :)
>>>  
>>> networks.  Use well-known published keys if you want to, but keep 
>>> the integrity codes in place.
>>>  
>>> ksjp
>>> -- 
>>> Prof. EECS, UC Berkeley
>>> On the evening before the ISA exhibition (Monday night) I visited 
>>> the show floor to see the ISA-100 booth.   Most all the booths were 
>>> unmanned and all was quiet except for a small group in one booth 
>>> that was having problems.  They were using Zigbee.   Their Zigbee 
>>> system was not functioning due to some type of interference.  (The 
>>> name of the company is not mentioned here because for the purpose of 
>>> this note it is not necessary).  I walked by the booth and the 
>>> people were very frustrated as to the cause of their problem.   I 
>>> tried to help.  We set up a 15.4 packet sniffer to look at the 
>>> packets to see what was the cause of the problem.  The vendor was 
>>> using Ember chips and software stack.  We got the sniffer running 
>>> and looked at the packets and we saw the problem.  There was nothing 
>>> we could do because the 15.4 stack was being corrupted by an illegal 
>>> packet(or so I thought at the time).  The packets were sent to the 
>>> Ember software team for analysis and they made modification to their 
>>> stack and the vendor was up and running by the time the show opened 
>>> on Tuesday.
>>>
>>>  
>>>
>>> booth and the surrounding area and everything seemed to be shut 
>>> down.  I walked up and down every aisle looking for a possible 
>>> transmission site.   I didn't see anything except for the wireless 
>>> HART booth where there was "flashing LEDs", a possible sign of RF 
>>> transmissions.  It could have been another Zigbee/SP-100 
>>> transmission site as well.
>>>
>>>  
>>>
>>> Ember was at the booth.  We had a nice chat and when I got back home 
>>> I asked him if he could have his software people write up what they 
>>> found.  Below is a short analysis of the problem written by the VP 
>>> of engineering at Ember.  They did a great job and had the problem 
>>> I have included this because depending on your point of view this 
>>> may be harbinger of things to come or it  can be dismissed..  I 
>>> thought the group should know what we may or may not expect in the 
>>> future.
>>>
>>>  
>>>
>>> Gerry
>>>
>>>  
>>>
>>> intended to be unicast to an orphan device, or broadcast to a 
>>> network that is changing its parameters.  Receipt of this packet was 
>>> causing the Ember device to change channels and PAN ID.  The 15.4 
>>> address for the source.  However, in this case the 16 bit 
>>> coordinator address for the source was being used making this an 
>>> illegal packet.  This illegal packet to change PAN ID was being 
>>> generated by another network device at a regular interval.
>>>
>>>  
>>>
>>> --- You are currently subscribed to sp100 as: 
>>> [email-masked] To unsubscribe send a blank email to 
>>> [email-masked]   
>>
>>

---

From: Kris Pister <[email-masked]>
Date: Wed, 31 Oct 2007 15:05:22 -0700

Hmm.  I guess you're right after all - I am confused.  I completely 
agree with your statements below, but then I re-read RFC4944 and it says:
"Other non-LoWPAN protocols that wish to coexist with LoWPAN nodes 
should include a byte matching this pattern immediately following the 
802.15.4. header."
And I read the Arch Rock 6LoWPAN tutorial (which is really well done, by 
the way), and it says "dispatch: coexistence with other protocols over 
the same link".  These statements do not appear to be aligned with what 
you wrote below.

As I see it, there are only three options:
1) we assume that MAC-layer MICs are always used for 6LoWPAN, perhaps 
with a well-known 6LoWPAN key, and then there is never any question 
about coexistence.  If you're parsing the dispatch byte, you already 
know that this is a 6LoWPAN frame.
2) we assume 1, but plan to share MAC-layer crypto with other non-LoWPAN 
protocols.  This sounds like a  bad idea from a crypto standpoint.
3) we believe that non-6LoWPAN frames may get through to the network 
layer, talk about "coexsistence", and cross our fingers that 15.4 toy 
designers are all familiar with and abide by the IETF recommendations.  
Down this path lies mysteriously failed networks, broken deployments, 
and more of the same semi-functionality that has held WSN 
commercialization back for the last 5 years.

So which is it?  My vote is #1.

ksjp

---

From: Philip Levis <[email-masked]>
Date: Wed, 31 Oct 2007 15:18:24 -0700

On Oct 31, 2007, at 3:05 PM, Kris Pister wrote:

> should include a byte matching this pattern immediately following  
> the 802.15.4. header."
> And I read the Arch Rock 6LoWPAN tutorial (which is really well  
> done, by the way), and it says "dispatch: coexistence with other  
> protocols over the same link".  These statements do not appear to  
> be aligned with what you wrote below.
>
> 1) we assume that MAC-layer MICs are always used for 6LoWPAN,  
> perhaps with a well-known 6LoWPAN key, and then there is never any  
> question about coexistence.  If you're parsing the dispatch byte,  
> you already know that this is a 6LoWPAN frame.
> 2) we assume 1, but plan to share MAC-layer crypto with other non- 
> LoWPAN protocols.  This sounds like a  bad idea from a crypto  
> standpoint.
> 3) we believe that non-6LoWPAN frames may get through to the  
> network layer, talk about "coexsistence", and cross our fingers  
> that 15.4 toy designers are all familiar with and abide by the IETF  
> recommendations.  Down this path lies mysteriously failed networks,  
> broken deployments, and more of the same semi-functionality that  
> has held WSN commercialization back for the last 5 years.
>

I think it's #2.

The idea is that you can design protocols that safely operate  
alongside 6lowpan. If you're concerned about people just grabbing  
NALP identifiers willy-nilly, well, that's what IANA is for. To give  
an example, chances are TinyOS 2.1 will introduce an NALP byte for  
its protocols so TinyOS research protocols can coexist on a node also  
running 6lowpan.

Phil

---

From: Philip Levis <[email-masked]>
Date: Wed, 31 Oct 2007 20:44:56 -0700

On Oct 31, 2007, at 7:11 PM, Geoff Mulligan wrote:

> Phil,
>
> If the TinyOS research protocol would be possibly using 6lowpan
> designated other headers, such as HC1 or HC1g or the Frag header,  
> then I
> would think that rather than using NALP, they could request a non-NALP
> dispatch value.

>
> I would like to see devices that not attempting to utilize any IP and
> 6lowpan headers/functions assign the first 2 bits after the 15.4  
> header
> to Zero.  What they do with the other 6 bits then are up to them.  I
> would recommend that IANA stay out of that fray.

I don't understand; NALP is the set of all dispatch bytes whose first  
two bits are zero.  If IANA isn't responsible for the NALP values,  
then I assume no-one is and it's a free-for-all of collision?

Phil