Subject: [homenet] writeup of my 2018 homenet experience on openwrt

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 8 Nov 2018 01:39:28 +0100 (CET)

Hi,

just to sum up what I said at the mic yesterday in the session.

OpenWrt 18.06.1 opkg installation of homenet requires that one deletes the 
existing dhcpv6 server because of a file conflict.

Configuration of interfaces is kind of hard, I believe I had to edit the 
network file directly to get everything to work. I also explicitly set the 
"homenet-external" and "internal" types, otherwise it didn't seem to 
understand which interfaces were external and internal.

But after that, hnetd and babeld seemed to work as expected. I did not try 
service discovery.

>From a user perspective, there are a few problems:

When an interface goes down and then up again, it's renumbered. This 
includes reboots. Since we still do not have session continuity across 
address changes, this is hard. Same with the separate routing domains 
between APs.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 08 Nov 2018 18:16:16 +0100

> includes reboots.

That shouldn't happen as long as there remains at least one Homenet router
to maintain the prefix (see Section 4.1 point 3 of RFC 7695).

I believe that hnetd (but not shncpd) additionally supports some mechanism
to handle the case where there are no routers left to maintain the prefix,
but I'm less sure.

-- Juliusz

---

From: Markus Stenberg <[email-masked]>
Date: Thu, 8 Nov 2018 21:28:14 +0200

> to handle the case where there are no routers left to maintain the prefix,

In hnetd there are actually two mechanisms fo ensure prefixes come up back the same they were before;

- storage of assigned prefixes (if enabled; if use of flash writes is not desired this is not used obviously), and

- ~determinstic pseudo-random assignment on interfaces

If both fail, that's probably a bug, although not sure if both options can be off if configured so. Fallback is purely random assignment out of prefix.

Anyway, getting back to topic of Ted's passionate speech about bad HNCP implementations, I'd love to see him (or someone else) provide better one :-)

Cheers,

-Markus

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 08 Nov 2018 23:07:16 +0100

> Anyway, getting back to topic of Ted's passionate speech about bad HNCP
> implementations, I'd love to see him (or someone else) provide better one :-)

Ted is busy working on his implementation of Simple Naming.  Please leave
him in peace, it's important that he convince us that Simple Naming is as
simple as he claims.

(Not that we don't trust him, but obviously nobody in this WG would ever
seriously listen to a person without an implementation.)

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 05:10:27 +0700

Markus, I tried to be really clear about what I was communicating on the
slide about implementations, but probably failed.  I am not saying your
implementation is bad.  I'm saying that it's sufficiently difficult for *me* to
modify that it's a barrier to forward progress.   The point of saying that
was not to insult your implementation, but to point out a problem that is
holding me back from making progress on this work.

One obvious solution to the problem is for us to collaborate on improving
the situation, which is something I'd like to see happen.   But I know that
this would be a hobby project for you right now, and you probably don't
have time.   That's what I was lamenting.

On Fri, Nov 9, 2018 at 2:28 AM Markus Stenberg <[email-masked]>
wrote:

> If both fail, that's probably a bug, although not sure if both options can
> be off if configured so. Fallback is purely random assignment out of prefix.
>
> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet
>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 09 Nov 2018 00:57:41 +0100

> slide about implementations, but probably failed.

Indeed.

A number of your comments about Markus' code were entirely unnecessary.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 07:03:29 +0700

The issue with the code (IIRC) is that it requires cmake to compile, for no
obvious reason, and cmake is hard to get working, so e.g. building it on
MacOS X is a major porting task.   And it depends on libraries that I don't
have.   And there's no layering of the configuration system aspect of the
architecture—it just goes and bashes on interfaces and stuff (this is my
recollection—I haven't looked at it in a year or so).

These are not bad things that Markus did.   Markus was doing what he needed
to do to get it running on OpenWRT.   But these are real issues, which are
preventing me from using the code.   For someone who is not an HNCP expert
to hack on the code is difficult, and would require substantial work.   I
could do it, but I have other things I'm working on.   If the code were
more modular, I could experiment with it.   This is the problem I was
trying to express.

---

From: Michael Thomas <[email-masked]>
Date: Thu, 8 Nov 2018 16:57:42 -0800

On 11/8/18 4:03 PM, Ted Lemon wrote:
> The issue with the code (IIRC) is that it requires cmake to compile, 
> for no obvious reason, and cmake is hard to get working, so e.g. 
> building it on MacOS X is a major porting task.   And it depends on 
> libraries that I don't have.   And there's no layering of the 
> configuration system aspect of the architecture—it just goes and 
> bashes on interfaces and stuff (this is my recollection—I haven't 
> looked at it in a year or so).

Sometimes the easiest thing to do in these cases is just give in and 
create a linux vm. they work really well these days.

Mike

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 08:07:02 +0700

It doesn’t build or work on Ubuntu either.

---

From: tjw ietf <[email-masked]>
Date: Thu, 8 Nov 2018 20:23:29 -0500

I heard of some fancy new technology. I think the kids call them “containers”

From my high tech gadget

> On Nov 8, 2018, at 20:07, Ted Lemon <[email-masked]> wrote:
>

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 09:01:12 +0700

What’s your point?

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 9 Nov 2018 07:49:40 +0100

Thanks Markus for clarifying this.

In addition, I can also confirm that in the absence of an uplink delegated prefix, 
a ULA prefix is there to allow internal communications.

So my best guess is that there is a bug around OpenWrt
telling hnetd that an interface is up or not, and internal or external.

- Pierre 

> Le 8 nov. 2018 à 20:28, Markus Stenberg <[email-masked]> a écrit :
>

---

From: [email-masked]
Date: Fri, 9 Nov 2018 08:07:16 +0100

Hello Ted,

OpenWrt cross-compilation is quite simple nowadays and works very well on Linux.
You can easily build targets for hardware, VMs, or just get the filesystem tar.

From there, you have a couple of options:
- Flash a router every time you change the code (That is what I was doing at the time. OpenWrt on VM or container was not exactly working as expected)
- Run it on Linux (iirc, Markus was testing in containers)
- Docker: There has been quite a lot of progress in containers since then. If I were you I would try to build and run docker images of OpenWrt. 
  You can tune your DockerFile to put the configuration you want so you don’t need to change it every run.
  Here is a link to get you started: https://openwrt.org/docs/guide-user/virtualization/docker_openwrt_image?s[]=docker <https://oldwiki.archive.openwrt.org/doc/howto/docker_openwrt_image>
- VMs: A bit more heavyweight but a bit faster dev cycle than hardware. At the time it was not working extremely well, but today it’s probably better.
  https://openwrt.org/docs/guide-user/virtualization/virtualbox-vm?s[]=virtualbox <https://openwrt.org/docs/guide-user/virtualization/virtualbox-vm?s%5B%5D=virtualbox>

Hope this helps.

- Pierre

> Le 9 nov. 2018 à 03:01, Ted Lemon <[email-masked]> a écrit :
> 
> What’s your point?
> 
> On Fri, Nov 9, 2018 at 8:23 AM tjw ietf <[email-masked] <mailto:[email-masked]>> wrote:
> On Nov 8, 2018, at 20:07, Ted Lemon <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> On Fri, Nov 9, 2018 at 7:58 AM Michael Thomas <[email-masked] <mailto:[email-masked]>> wrote:
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/homenet <https://www.ietf.org/mailman/listinfo/homenet>
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/homenet <https://www.ietf.org/mailman/listinfo/homenet>

---

From: Tim Wicinski <[email-masked]>
Date: Fri, 9 Nov 2018 02:42:07 -0500

Apologies, I should never use the phone to send emails, I always hit send
to do this.

I was trying to say cmake on OSX is a real pain, I had this problem a year
ago and I ended up doing it in a container.
I went back to my computer to find the container, and it's gone, and then I
remember it was a year ago because
when while @ IETF100 I had to have my have my work laptop reimaged and....
container lost.

But agree on all counts on cmake.

again apologies.

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 9 Nov 2018 09:47:54 +0200

First off, hnetd was team effort - me, Pierre Pfister and Steven Barth.

Secondly, I did not particularly want to promote hnetd but 'existing implementations are bad, boo hoo' argument gets old and I think e.g. https://github.com/jech/shncpd is also quite sufficient. I use even https://github.com/fingon/pysyma 'in production' even now (admittedly not HNCP bits of it, but DNCP + SHSP :->).

On 09.11.2018, at 2.03, Ted Lemon <[email-masked]> wrote:
> The issue with the code (IIRC) is that it requires cmake to compile, for no obvious reason, and cmake is hard to get working, so e.g. building it on MacOS X is a major porting task.   And it depends on libraries that I don't have.   And there's no layering of the configuration system aspect of the architecture—it just goes and bashes on interfaces and stuff (this is my recollection—I haven't looked at it in a year or so).

There's some distinct parts that have well defined interfaces (as far as those 'well defined' go in C anyway) - DNCP (dncp_*), /HNCP code (hncp_*), prefix assignment code (pa_*), and operating system (platform_*) are all pretty loosely coupled.

I have compiled the DNCP(/partial HCNP) bits on OS X with relatively minor modifications for my hobby project for example.

> These are not bad things that Markus did.   Markus was doing what he needed to do to get it running on OpenWRT.   But these are real issues, which are preventing me from using the code.   For someone who is not an HNCP expert to hack on the code is difficult, and would require substantial work.   I could do it, but I have other things I'm working on.   If the code were more modular, I could experiment with it.   This is the problem I was trying to express.

I have used it also unmodified on plain Linux number of times. If you want to use it on something more esotreic, I could bet on OS X port being doable but as lots of HNCP value comes from interfacing with system servers, I do not see anyone writing platform interface for it.

I am curious about what 'more modular' code looks like. Separate DLLs? Single function call API between modules? Possibly made out of Lego bricks?

Cheers,

-Markus

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 14:48:46 +0700

My edge router is an Ubuntu machine. I haven’t been able to get Marcus’
HNCP daemon to build there. It’s possible that that has changed since I
last tried it, but that was what stopped me last time.

On Fri, Nov 9, 2018 at 4:07 PM <[email-masked]> wrote:

> - Flash a router every time you change the code (That is what I was doing
> at the time. OpenWrt on VM or container was not exactly working as expected)
> - Docker: There has been quite a lot of progress in containers since then.
> If I were you I would try to build and run docker images of OpenWrt.
>   Here is a link to get you started:
> https://openwrt.org/docs/guide-user/virtualization/docker_openwrt_image?s[]=docker
> <https://oldwiki.archive.openwrt.org/doc/howto/docker_openwrt_image>
> - VMs: A bit more heavyweight but a bit faster dev cycle than hardware. At
> the time it was not working extremely well, but today it’s probably better.
>
> https://openwrt.org/docs/guide-user/virtualization/virtualbox-vm?s[]=virtualbox
>
> Hope this helps.
>
> - Pierre
>
>

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 9 Nov 2018 09:52:15 +0200

On 09.11.2018, at 9.48, Ted Lemon <[email-masked]> wrote:
> My edge router is an Ubuntu machine. I haven’t been able to get Marcus’ HNCP daemon to build there. It’s possible that that has changed since I last tried it, but that was what stopped me last time. 

It built fine on Debian/Ubuntu last I tried it (late 2016 so my guess is Debian 8 + Ubuntu 14.04 LTS), but obviously you need the dependencies (libubox, cmake, etc).

Since that I have moved back to IPv4 ISP and my motivation and therefore time for that particular hobby project died.

Pull requests accepted ;-)

Cheers,

-Markus

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 14:52:30 +0700

Made out of LEGO bricks. What you’re describing doesn’t match my experience
last time I looked at it.  I will look again. This is why I asked: because
I need help. This is the most information I’ve gotten about it. I’m sorry
my comments come across as saying the software is bad. That’s not what i
was trying to communicate.

On Fri, Nov 9, 2018 at 4:47 PM Markus Stenberg <[email-masked]>
wrote:

> Secondly, I did not particularly want to promote hnetd but 'existing
> implementations are bad, boo hoo' argument gets old and I think e.g.
> https://github.com/jech/shncpd is also quite sufficient. I use even
> https://github.com/fingon/pysyma 'in production' even now (admittedly not
> HNCP bits of it, but DNCP + SHSP :->).
>
> > The issue with the code (IIRC) is that it requires cmake to compile, for
> no obvious reason, and cmake is hard to get working, so e.g. building it on
> There's some distinct parts that have well defined interfaces (as far as
> those 'well defined' go in C anyway) - DNCP (dncp_*), /HNCP code (hncp_*),
> prefix assignment code (pa_*), and operating system (platform_*) are all
> pretty loosely coupled.
>
> I have compiled the DNCP(/partial HCNP) bits on OS X with relatively minor
> modifications for my hobby project for example.
>
> > These are not bad things that Markus did.   Markus was doing what he
> needed to do to get it running on OpenWRT.   But these are real issues,
> which are preventing me from using the code.   For someone who is not an
> HNCP expert to hack on the code is difficult, and would require substantial
> work.   I could do it, but I have other things I'm working on.   If the
> code were more modular, I could experiment with it.   This is the problem I
> was trying to express.
>
> I have used it also unmodified on plain Linux number of times. If you want
> to use it on something more esotreic, I could bet on OS X port being doable
> but as lots of HNCP value comes from interfacing with system servers, I do
> not see anyone writing platform interface for it.
>
> I am curious about what 'more modular' code looks like. Separate DLLs?
> Single function call API between modules? Possibly made out of Lego bricks?
>
> Cheers,
>
> -Markus

---

From: [email-masked]
Date: Fri, 9 Nov 2018 09:21:36 +0100

I guess what you want to do is publish and receive TLVs using DNCP, and probably get a sense of the topology.
To do so, there mostly is a single header to look at: dncp.h <https://github.com/sbyx/hnetd/blob/master/src/dncp.h>

Here is a simple example of Wifi autoconfig we did for one of IETF’s hackathon: https://github.com/Oryon/hnetd/blob/hackathon/src/hncp_wifi.c <https://github.com/Oryon/hnetd/blob/hackathon/src/hncp_wifi.c>
As a more advanced example, I think the service discovery part will be relevant to you: https://github.com/sbyx/hnetd/blob/master/src/hncp_sd.c <https://github.com/sbyx/hnetd/blob/master/src/hncp_sd.c>

If you want to add configuration parameters available in OpenWrt, I’d recommend you look at platform-openwrt.c <https://github.com/sbyx/hnetd/blob/master/src/platform-openwrt.c> 

- Pierre

> Le 9 nov. 2018 à 08:52, Ted Lemon <[email-masked]> a écrit :
> 
> Made out of LEGO bricks. What you’re describing doesn’t match my experience last time I looked at it.  I will look again. This is why I asked: because I need help. This is the most information I’ve gotten about it. I’m sorry my comments come across as saying the software is bad. That’s not what i was trying to communicate. 
> 
> On Fri, Nov 9, 2018 at 4:47 PM Markus Stenberg <[email-masked] <mailto:[email-masked]>> wrote:
> Secondly, I did not particularly want to promote hnetd but 'existing implementations are bad, boo hoo' argument gets old and I think e.g. https://github.com/jech/shncpd <https://github.com/jech/shncpd> is also quite sufficient. I use even https://github.com/fingon/pysyma <https://github.com/fingon/pysyma> 'in production' even now (admittedly not HNCP bits of it, but DNCP + SHSP :->).
> 
> On 09.11.2018, at 2.03, Ted Lemon <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 17:32:15 +0900

Yes, this is exactly the sort of information I am looking for—thanks.   I
just did a pass through the code to refresh my memory, and the issue I have
with it is that there's no architecture document, so you kind of have to
figure that out by reverse engineering it, and there aren't a lot of
comments.   hncp_sd.c is actually pretty good—I don't think I'd looked at
it before.   E.g., one of the modules I looked at before I got this mail
from you was pa_core.c.   Actually it looks like there are some useful
comments in the header files.   But an introduction that explains how the
code basically works would really help.   Not the protocol—just what the
concepts are and how they fit together.   And debugging messages aimed at
users would help as well—one of the easiest ways to learn how something
works is to just run it with debugging maxed and see what it logs.   But
the log messages appear to be more about hncp than about what has been
learned through hncp.

Anyway, I'll give it another look—you've already helped me quite a bit.

On Fri, Nov 9, 2018 at 5:21 PM <[email-masked]> wrote:

> To do so, there mostly is a single header to look at: dncp.h
> <https://github.com/sbyx/hnetd/blob/master/src/dncp.h>
>
> Here is a simple example of Wifi autoconfig we did for one of IETF’s
> hackathon: https://github.com/Oryon/hnetd/blob/hackathon/src/hncp_wifi.c
> As a more advanced example, I think the service discovery part will be
> relevant to you: https://github.com/sbyx/hnetd/blob/master/src/hncp_sd.c
>
> If you want to add configuration parameters available in OpenWrt, I’d
> recommend you look at platform-openwrt.c
> <https://github.com/sbyx/hnetd/blob/master/src/platform-openwrt.c>
>
> - Pierre
>
> Made out of LEGO bricks. What you’re describing doesn’t match my
> experience last time I looked at it.  I will look again. This is why I
> asked: because I need help. This is the most information I’ve gotten about
> it. I’m sorry my comments come across as saying the software is bad. That’s
> not what i was trying to communicate.
>
>> https://github.com/fingon/pysyma 'in production' even now (admittedly
>> not HNCP bits of it, but DNCP + SHSP :->).
>>
>> for no obvious reason, and cmake is hard to get working, so e.g. building
>> it on MacOS X is a major porting task.   And it depends on libraries that I
>> don't have.   And there's no layering of the configuration system aspect of
>> the architecture—it just goes and bashes on interfaces and stuff (this is
>> my recollection—I haven't looked at it in a year or so).
>>
>> I have compiled the DNCP(/partial HCNP) bits on OS X with relatively
>> minor modifications for my hobby project for example.
>>
>> I have used it also unmodified on plain Linux number of times. If you
>> want to use it on something more esotreic, I could bet on OS X port being
>> doable but as lots of HNCP value comes from interfacing with system
>> servers, I do not see anyone writing platform interface for it.
>>

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 9 Nov 2018 10:44:34 +0200

I may or may not have some architectural/flow diagrams, but I am bit leery of putting them anywhere (if I were to have them that is) as they were not officially licensed to be distributed (as the code was) and as this was Cisco-funded research project back in the day.

I could draw some basic ones from scratch one of these days and stick them in the repository if there is sufficient demand :-)

Today's hobby project will be Babel bis draft support for pybabel though.

-Markus

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 17:48:27 +0900

Noted. Good luck with the Babel work!

On Fri, Nov 9, 2018 at 5:44 PM Markus Stenberg <[email-masked]>
wrote:

> I may or may not have some architectural/flow diagrams, but I am bit leery
> of putting them anywhere (if I were to have them that is) as they were not
> officially licensed to be distributed (as the code was) and as this was
> Cisco-funded research project back in the day.
>
> I could draw some basic ones from scratch one of these days and stick them
> in the repository if there is sufficient demand :-)
>

---

From: [email-masked]
Date: Fri, 9 Nov 2018 09:51:02 +0100

hnetd is actually able to produce quite an insane amount of debug logs (e.g. about anything happening on DNCP).
Those are meant for developers (like trying to debug DNCP protocol).
I don’t think this is what you want to do, since DNCP is actually fairly reliable… But since you ask, here is how to sink under DNCP debug messages.
- Enable ‘debug’ log level in hnetd.h (HNETD_DEFAULT_L_LEVEL).
- And then set hnetd runtime log level to debug with --loglevel option (As you probably tried already).

- Pierre

> Le 9 nov. 2018 à 09:32, Ted Lemon <[email-masked]> a écrit :
> 
> Yes, this is exactly the sort of information I am looking for—thanks.   I just did a pass through the code to refresh my memory, and the issue I have with it is that there's no architecture document, so you kind of have to figure that out by reverse engineering it, and there aren't a lot of comments.   hncp_sd.c is actually pretty good—I don't think I'd looked at it before.   E.g., one of the modules I looked at before I got this mail from you was pa_core.c.   Actually it looks like there are some useful comments in the header files.   But an introduction that explains how the code basically works would really help.   Not the protocol—just what the concepts are and how they fit together.   And debugging messages aimed at users would help as well—one of the easiest ways to learn how something works is to just run it with debugging maxed and see what it logs.   But the log messages appear to be more about hncp than about what has been learned through hncp.
> 
> On Fri, Nov 9, 2018 at 5:21 PM <[email-masked] <mailto:[email-masked]>> wrote:
>> Le 9 nov. 2018 à 08:52, Ted Lemon <[email-masked] <mailto:[email-masked]>> a écrit :
>>

---

From: Ted Lemon <[email-masked]>
Date: Fri, 9 Nov 2018 17:52:34 +0900

I’ll try that. I don’t remember if I did last time. My recollection was
that the log messages were useful, but not sufficient, pretty much for the
reason you’ve stated.

On Fri, Nov 9, 2018 at 5:51 PM <[email-masked]> wrote:

> I don’t think this is what you want to do, since DNCP is actually fairly
> reliable… But since you ask, here is how to sink under DNCP debug messages.

---

From: Michael Thomas <[email-masked]>
Date: Fri, 9 Nov 2018 10:47:50 -0800

> Pull requests accepted ;-)
This is why G*d invented chef/puppet/ansible :)

Readme's are good too.

Mike