Subject: Local Cloud Node

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 3 Oct 2014 09:11:48 -0400

As someone with far too many IP addressable devices in my house, I am
getting rather fed up of every new device demanding to be hooked up to
a service in the cloud.

Yes, I get the fact that the providers are trying to razor and blade
me with their business model. But that model only works for Gillette
because the blade is the most expensive, difficult part to make.

As a consumer, these services in the cloud represent a lot of risk and
a painful integration nightmare. The point of installing network
addressable switches to control the lights is so that they can be
turned on and off automatically, not so that I can control them from
an iPhone where I have to press six buttons and wait a minute for the
app to load. Then another quarter hour while it insists on updating
itself.

There is therefore a real need for a standards based device that
provides the same services these devices expect from 'the cloud' and
make them available in the local net. And it should be really easy to
connect a newly purchased device to such a cloud.

---

From: John C Klensin <[email-masked]>
Date: Fri, 03 Oct 2014 14:12:32 -0400

--On Friday, October 03, 2014 09:11 -0400 Phillip Hallam-Baker
<[email-masked]> wrote:

> As someone with far too many IP addressable devices in my
> house, I am getting rather fed up of every new device
> demanding to be hooked up to a service in the cloud.
> 
> Yes, I get the fact that the providers are trying to razor and
> blade me with their business model. But that model only works
> for Gillette because the blade is the most expensive,
> difficult part to make.
> 
> As a consumer, these services in the cloud represent a lot of
> risk and a painful integration nightmare. The point of
> installing network addressable switches to control the lights
> is so that they can be turned on and off automatically, not so
> that I can control them from an iPhone where I have to press
> six buttons and wait a minute for the app to load. Then
> another quarter hour while it insists on updating itself.
> 
> There is therefore a real need for a standards based device
> that provides the same services these devices expect from 'the
> cloud' and make them available in the local net. And it should
> be really easy to connect a newly purchased device to such a
> cloud.

While I agree that there is a problem, and largely agree with
your analysis of it, I don't see work here for which the IETF
has the right expertise, at least unless there were some chance
of convincing a WG like Homenet that their boundary router specs
should be expanded adequately to act as a controller for these
families of devices. 

As a more general observation, I think there are a number of
areas where performance and efficiency as seen by end users, as
well as security and privacy, would be well-served by moving
services from shared centralized facilities (e.g., "the cloud")
[back] onto more local networks that fell into the user's
administrative domain and control.    The class of devices
described above are, IMO, just one set of instances of the more
general issue.

    john

Disclaimer: I've been doing some "day job" work that is
consistent with some of the ideas suggested above and that
assumes significantly more powerful edge devices for relatively
small networks than we have gotten used to.  To the best of my
knowledge, there is no plan to bring any aspect of that work to
the IETF so this is not an IPR disclosure, merely an observation
about external discussions that may be influencing my thinking
and comments.

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 03 Oct 2014 19:29:53 +0100

> described above are, IMO, just one set of instances of the more

(No hats and all that.)

I agree. (And I'm also doing a little work on a side-project
along these lines - wonder how many of us are at stuff like
that:-)

And fwiw, I've not yet hit places so far where there's obvious
IETF work to be done in the short term. Longer term there could
be but I think it'd depend on some of these activities getting
popular first.

Cheers,
S.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 3 Oct 2014 14:39:16 -0400

Yes, I agree with the replies so far, more or less.

What I think would be rather sad is if we end up with a situation
where the emergent market for devices that connect these types of gear
end up having to become a middlebox so that they can intercept the
obnoxiously routed traffic.

I see a gap here in the Internet architecture. It should be easy to
hook up new appliances but it isn't easy at all. We could make it easy
by defining an appropriate protocol. But we have not seen that as our
job to date. And nor has IEEE which is still blissfully unaware that
their WiFi model still isn't fit for purpose.

I think that the way a solution is going to emerge is by people
putting together schemes that meet the need and making them available
as open source.

---

From: James Woodyatt <[email-masked]>
Date: Tue, 7 Oct 2014 11:11:44 -0700

On Fri, Oct 3, 2014 at 11:39 AM, Phillip Hallam-Baker <[email-masked]
> wrote:

> Yes, I agree with the replies so far, more or less. [...]
>

As general principle, my preference is for networked home devices to
*request* access to its maker's online service, with the owner having the
option to decline and to still have a functional device with the basic
features all working as a normal person would naturally expect.  When I
think about arguments for *demanding* access to Internet service, I can
think of precisely one for which I am forced to admit there are personally
convincing reasons for doing it: emergency firmware update.  Even there, I
still squirm, and I can sympathize with people who disagree.

Really, if my personal preferences are to rule the day, then everything
else ought to be in the category of "you bought a $DEVICE, and it does
$FUNCTION just fine, but if you let it call it's mother periodically, then
it will also do $OPTION as well, and won't that be nice.  Okay? [y/N]"
(FWIW, I'm reasonably sure my current employers hold a compatible view on
this topic, but— you know— I can't speak for them, of course.)  Enabling
more local autonomy would make me happier, and my hunch is that this may
actually be a minority view in the Internet engineering community, but I'm
happy to represent yo.  For reals.

This is to say that I agree with Mr. Klensin: there are some places where
the current HOMENET drafts are less than optimal in this regard. I can
think of ways to improve them to make engineering the local autonomy we're
discussing more feasible.  (Yes, I'll elaborate in forthcoming messages to
the HOMENET list.)

-- 
james woodyatt <[email-masked]>
Nest Labs, Communications Engineering

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 7 Oct 2014 14:38:54 -0400

On Tue, Oct 7, 2014 at 2:11 PM, James Woodyatt <[email-masked]> wrote:
> On Fri, Oct 3, 2014 at 11:39 AM, Phillip Hallam-Baker
> Really, if my personal preferences are to rule the day, then everything else
> ought to be in the category of "you bought a $DEVICE, and it does $FUNCTION
> just fine, but if you let it call it's mother periodically, then it will
> also do $OPTION as well, and won't that be nice.  Okay? [y/N]" (FWIW, I'm
> reasonably sure my current employers hold a compatible view on this topic,
> but— you know— I can't speak for them, of course.)  Enabling more local
> autonomy would make me happier, and my hunch is that this may actually be a
> minority view in the Internet engineering community, but I'm happy to
> represent yo.  For reals.

I think the starting point would be for the device to tell my system
what it is and what version of firmware it is running.

Automatic firmware updates have already become a source of nuisance
rather than sanity in the house. When I turn something on I want it to
work NOW! NOW! NOW! About one time in ten a given device will turn on
and then start downloading stuff for ten minutes.

What really irritates is the approach typified by my DVR set top box
which is on 24/7 and connected to the net. When an update notification
is sent it posts a note on the splash screen then waits for someone to
attempt to use it. At that point and only at that point does it begin
the mandatory update. There is no option to cancel, the machine just
waits until it can inconvenience someone.

Right now I have a house with three Internet thermostats and six smoke
alarms, all of which have temperature sensors and occupancy sensors.
It would seem like a no brainer to connect these up in an intelligent
way so that lights get switched off when there is nobody in a room and
using the temperature in the rooms that are occupied to set the
furnace temperature rather than the ones that are not.

---

From: Ray Bellis <[email-masked]>
Date: Thu, 9 Oct 2014 12:57:09 +0000

On 7 Oct 2014, at 19:38, Phillip Hallam-Baker <[email-masked]<mailto:[email-masked]>> wrote:

With the key part being that all this should work exactly the same even when the household 'net connection is offline.

Ray

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 9 Oct 2014 09:40:47 -0400

On Thu, Oct 9, 2014 at 8:57 AM, Ray Bellis <[email-masked]> wrote:
>
> On 7 Oct 2014, at 19:38, Phillip Hallam-Baker <[email-masked]> wrote:
>
> With the key part being that all this should work exactly the same even when
> the household 'net connection is offline.

Precisely.

We had a storm come through and take out the electricity line and
almost took the primary Internet as well. Out here they supply
electricity from telephone poles! So when a tree brings down a line
there is live 240V supply.

It would of course be a bad thing if taking out the Internet shut the
house down.

---

From: David Morris <[email-masked]>
Date: Thu, 9 Oct 2014 07:32:25 -0700 (PDT)

> It would of course be a bad thing if taking out the Internet shut the

We've had many more and longer internet failures than power failures.
Never has either failure been due to a down pole (here phone and cable
service is delivered by renting space on power poles).

---

From: joel jaeggli <[email-masked]>
Date: Mon, 13 Oct 2014 23:14:41 -0700

> service is delivered by renting space on power poles).

Last week the power was out here for about 4 hours due to an upstream
transformer replacement, servers went down immediately due to their ups
being sized for mometary outages. around hour two the UPS protecting the
cable modem and the router finally ran down. The cellular  router I have
configured with the same ssid still had plenty of devices connected to
it around hour 4 when the power came back on. I sat in my living-room
and worked, took calls at so on. The gradual paring back of capabilities
as both network and power went away is to my mind the expected and
graceful way for a homenet to respond to power and network outages. in
many respects the utility of these devices depends on them being
connected either to each other or the outside world so it should not be
entirely unsurprising that they became less useful either without
connectvity to each other or the internet.

---

From: Jim Gettys <[email-masked]>
Date: Tue, 14 Oct 2014 15:57:05 -0400

On Tue, Oct 7, 2014 at 2:38 PM, Phillip Hallam-Baker <[email-masked]>
wrote:

> else
​There is a serious issue lurking here: it is *not* safe for devices to be
without software updates. And it isn't safe to presume the upstream
manufacturer is being diligent in providing those updates.  And nagging end
users to do something that they don't understand is also not a solution.

Those of you unaware of the "Honeymoon effect" should read the following
paper:

http://www.acsac.org/2010/openconf/modules/request.php?module=oc_program&action=view.php&a&id=69&type=2

Both heartbleed and shellshock are good examples of this phenomena
(thankfully, shellshock does not happen to be present in most of these
devices, but in systems which are upgraded on an ongoing basis.

Worse yet are the "hidden monocultures" we have: binary blobs throughout
our systems that may make update impossible (on the scale of the lifetime
of these devices).

See:
http://gettys.wordpress.com/2014/10/06/bufferbloat-and-other-challenges/

The other challenges (security) dwarf bufferbloat.

I believe we need to rethink how we build software for these devices, in a
pretty fundamental way.

- Jim

 ​

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 14 Oct 2014 16:44:42 -0400

On Tue, Oct 14, 2014 at 3:57 PM, Jim Gettys <[email-masked]> wrote:
>
> There is a serious issue lurking here: it is *not* safe for devices to be

I think we need to divide divide devices into 'simple enough to not
need updates' and 'make use of a standard update process'.

My car has 30 computers in it (and a newer model would likely have
60). There is one on every wheel counting the rotations for the ABS
system. Do I really want them all to be updatable?

In general I only want devices to have an update capability if they 1)
have sufficient CPU power to authenticate the replacement code and 2)
have enough memory to hold the old and new code in memory while the
new code is being verified.

---

From: Jim Gettys <[email-masked]>
Date: Tue, 14 Oct 2014 16:53:48 -0400

On Tue, Oct 14, 2014 at 4:44 PM, Phillip Hallam-Baker <[email-masked]

> need updates' and 'make use of a standard update process'.

​There are few network connected devices "simple enough to not need
updates", IMHO. Distinguishing those that do from those that don't is just
about impossible.

Courtesy of Moore's law, even "simple" devices are often/usually based on
millions of lines of code.
​

> system. Do I really want them all to be updatable?

​I think those devices just emit signals, and we don't "talk" to them. I
can see sensors just being "output only" devices (though that creates a
different problem: network pollution.
​

> new code is being verified.

​Yup; there are some minimum hardware requirements.  One of the key
enablers sadly missing
at today is write protection on the flash.  What we did at OLPC is quite
cheap (one D flop + separate serial ROM for boot, cost of < $.30);
potential cost is zero, if we can get flash vendors to put the right hooks
into the flash (and if we can trust that flash to work properly, which
given recent experiences seems possibly questionable).

Jim
​

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 14 Oct 2014 17:20:53 -0400

On Tue, Oct 14, 2014 at 4:53 PM, Jim Gettys <[email-masked]> wrote:
>
>
> On Tue, Oct 14, 2014 at 4:44 PM, Phillip Hallam-Baker
> There are few network connected devices "simple enough to not need updates",
> IMHO. Distinguishing those that do from those that don't is just about
> impossible.
>

There are IP network devices and serial bus devices. I would like a
mechanism that would allow us to bring serial bus devices into the
Internet of things architecture without putting IP on them.

I think I could write a formal model of IP and prove an IPstack
correct. I certainly would not want to though. And I certainly don't
think I could go much more complex.

The sort of things I think need to be network addressable but not
updatable are things like temperature sensor drivers, motor speed
controllers, PID controllers and the like.

> I think those devices just emit signals, and we don't "talk" to them. I can
> see sensors just being "output only" devices (though that creates a

A light switch is a writable device.

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 15 Oct 2014 01:26:26 +0100

On 14/10/14 20:57, Jim Gettys wrote:
>>> butâ€” you knowâ€” I can't speak for them, of course.)  Enabling more local
> â€‹There is a serious issue lurking here: it is *not* safe for devices to be

If someone wanted to write an I-D describing a useful thing the IETF
could do in this space, I'd be very happy to see that and to try move
it along however is best done. (When I say "this space" I mean at
least s/w update for devices like sensors of various kinds and
DSL-modem like things.)

I think its possible, but not a slam-dunk, that there is good work
that the IETF could do on that topic.

S.

>  â€‹

---

From: Jim Gettys <[email-masked]>
Date: Wed, 15 Oct 2014 08:33:28 -0400

On Tue, Oct 14, 2014 at 5:20 PM, Phillip Hallam-Baker <[email-masked]

> >> > manufacturer is being diligent in providing those updates.  And
> nagging
> >> > end

​Hmmm... and they talk to what? with what code?  And with BadUSB out there
(unprotected writable flash is dangerous)?

> think I could go much more complex.

​The IP stack is the least of your worries.  Go look at
802.11g/802.11n/802.11ac drivers/stack and fear, and despair
at issues such as shellshock and heartbleed that run in user space (but let
you 0wn the devices).
The moral of the honeymoon effect is that zero-day
vulnerability discovery is *very* different that normal "bugs", and
​ occur​
much later.

I already know of one serious widespread issue in a binary blob device
driver that will
likely never leave the ecosystem.  It's bad enough we'd update the devices,
if only we could (and we can't; the devices only support manual upgrade).
And by the time the vulnerability was discovered, it's not clear that the
build systems/expertise still existed in the silicon vendor to do the fix,
and certainly not in the downstream ecosystem that used that silicon.
These "hidden monocultures" mean that your ecosystem, which you thought was
vibrant with many sources of supply, actually are often close to
monopolies, and those companies make no money out of fixing your security
nightmare.

> A light switch is a writable device.

​The switch is a "read" device.  It's the light bulb(s) that is(are) the
writable device(s).

​                      - Jim
​

​

---

From: Jim Gettys <[email-masked]>
Date: Wed, 15 Oct 2014 08:49:02 -0400

On Tue, Oct 14, 2014 at 8:26 PM, Stephen Farrell <[email-masked]>
wrote:

> >>> convincing reasons for doing it: emergency firmware update.  Even
> there,
> >> I

​Since writable flash must be protected (and cheaply!), and
software/firmware still updated in such devices
one area may be to describe such mechanisms, along with the issues of key
management
best practices which are far from the minds of most vendors who have never
given thought to long lived network systems.
Without such guidance, we'll live in a sea of vulnerability.

Looming larger is the observation that as an industry we don't think about
building systems and software with long life times; that goes well beyond
the IETF.

As to I-D's, I have to locate appropriate co-authors before I can commit to
anything.
                                     - Jim

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 15 Oct 2014 13:55:06 +0100

Hi Jim,

On 15/10/14 13:49, Jim Gettys wrote:
>> >
> â€‹Since writable flash must be protected (and cheaply!), and

Yes, a bunch of this is not really directly IETF stuff, but as you
say there could be informational/bcp documents that'd be of value
or maybe we'd find that the tools we already have (CMS, jose etc.)
might not be quite right or could need some extensions/addons that'd
help, or even that some new protocol might be needed.

> As to I-D's, I have to locate appropriate co-authors before I can commit to

Sure. I hope some folks get in touch with you on that.

S.

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 15 Oct 2014 15:50:13 +0200

On 10/15/2014 02:49 PM, Jim Gettys wrote:
>
>
> On Tue, Oct 14, 2014 at 8:26 PM, Stephen Farrell 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>
>
>     > On Tue, Oct 7, 2014 at 2:38 PM, Phillip Hallam-Baker
>     <[email-masked] <mailto:[email-masked]>>
>     > wrote:
>     >
>     >> On Tue, Oct 7, 2014 at 2:11 PM, James Woodyatt
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     >>> <[email-masked] <mailto:[email-masked]>> wrote:
>     >>>>
>     >>> convincing reasons for doing it: emergency firmware update. 
>     Even there,
>     >> I
>     >>> reasonably sure my current employers hold a compatible view on
>     this
>     >> topic,
>     nagging end
> one area may be to describe such mechanisms, along with the issues of 
> key management
> best practices which are far from the minds of most vendors who have 
> never given thought to long lived network systems.

For extra entertainment, consider that SIM cards have an independent OS 
and writable flash.

>
> Looming larger is the observation that as an industry we don't think 
> about building systems and software with long life times; that goes 
> well beyond the IETF.
>
> As to I-D's, I have to locate appropriate co-authors before I can 
> commit to anything.
>                    - Jim
>
>
>
>
>
>
>     S.
>
>     >
>     > - Jim
>     >
>     >
>     >  â€‹
>     >
>
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 15 Oct 2014 10:09:34 -0400

On Wed, Oct 15, 2014 at 9:50 AM, Harald Alvestrand <[email-masked]> wrote:

> For extra entertainment, consider that SIM cards have an independent OS and
> writable flash.

And being constrained devices, almost certain to be coded in raw C
with little reuse of common libraries and thus most likely to be
exposed to buffer overrun bugs.

Anyone want to do a pool bar BOF in Hawaii? One of my life ambitions
is to have a cocktail served in a baby pineapple.

I think there are several dimensions to the problem here. At one
level, just knowing what we have, what binaries are running on the
devices would be a good start. It might be cheaper to dispose of the
device or to isolate it in the network.

Knowing what there is would be the first step to managing updates.
Without management the update process becomes an attack in itself.
Until recently I was playing Minion Rush on my iPhone. Then they
'upgraded' to the Jelly lab edition and I suddenly lost all my
achievements just as I was about to complete the set.

I don't want my light bulb provider to Jelly-lab me. Any and all
updates have to be controlled by the network operator or their proxy.

---

From: joel jaeggli <[email-masked]>
Date: Tue, 28 Oct 2014 21:22:48 -0700

On 10/14/14 2:20 PM, Phillip Hallam-Baker wrote:
>> There are few network connected devices "simple enough to not need
>> updates", IMHO. Distinguishing those that do from those that don't
>> is just about impossible.
>> 
>> Courtesy of Moore's law, even "simple" devices are often/usually
>> based on millions of lines of code.
> 

There are whole clases of the devices  that I in fact cannot trust in
some cases if they are up datable.

That would include things like the code blocks of some kind of HSM,
hardware RNGs, identity tokens

Becomes suspect, means destroy it in those cases.  one hopes that in
general these devices are discrete enough to be replaceable in the
context in which they are used.

>>> My car has 30 computers in it (and a newer model would likely
>>> have 60). There is one on every wheel counting the rotations for
>>> the ABS system. Do I really want them all to be updatable?
>> 
>> 
>> I think those devices just emit signals, and we don't "talk" to
>> them. I can see sensors just being "output only" devices (though
>> that creates a different problem: network pollution.
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 29 Oct 2014 08:18:52 -0400

Getting back to my original concern, the maker of the hub that I
bought to integrate the 30+ devices in my house has just been bought
out. The new owner has announced that the hub is EOL and none of the
promised updates is going to be provided.

The main reason we have standards in the enterprise space is that they
reduce switching costs so that a vendor can be replaced if necessary.
The main thing holding back the home automation market right now is
that all the vendors are following a strategy designed to make them
the sole source monopoly provider. In most cases with an annual rent
on top for doing nothing more than run an unneeded cloud service.

---

From: Joshua Smith <[email-masked]>
Date: Wed, 29 Oct 2014 08:38:39 -0400

> on top for doing nothing more than run an unneeded cloud service.

+1

-- 
Joshua Smith