Subject: [homenet] hnet-full & LEDE bug report

From: Rich Brown <[email-masked]>
Date: Fri, 22 Jul 2016 23:31:27 -0400

Folks,

I submitted this report to the LEDE bug tracker. I realize that LEDE is slightly different from OpenWrt, but they still track pretty closely, so I wonder if anyone here has thoughts about what might be happening. Many thanks.

Rich

---- https://bugs.lede-project.org/index.php?do=details&task_id=58 ---
I have always wanted to try out homenet. In light of the recommendation expressed in http://lists.infradead.org/pipermail/lede-dev/2016-July/001772.html, it looks as if LEDE would be a good platform for further experimentation.

I followed Tore Anderson’s excellent instructions at: https://blog.toreanderson.no/2015/10/11/making-a-homenet-router-out-of-openwrt.html The following is based on: LEDE r1006 and the default packages for hnet-full (ipset 6.24-1, and hnet-full 2016-06-28-606d7e9046..e-1).

The problem came when “Step 3 - Converting WAN interface to Homenet” didn’t work as described. I clicked Delete for the WAN interface. The entry lost its address info, but did not to disappear as expected. Repeated clicks made no further difference. Here’s a screenshot of what I saw. (WAN and WAN6 looked similar before clicking the WAN Delete button.)

http://richb-hanover.com/wp-content/uploads/2016/07/PastedGraphic-1.png

Subsequently clicking Delete for WAN6 caused the WAN6 entry to disappear. The WAN entry remained, keeping only its RX & TX info. Here’s the resulting screenshot.

http://richb-hanover.com/wp-content/uploads/2016/07/PastedGraphic-2.png

At this point, I should have bailed. (I did try to go further, but nothing that happened was encouraging.)

Other observations: 
- I tried the same process with OpenWrt 15.05.1, ipset 6.24-1, and hnet-full 2015-07-10-ea2bd2bc2d..9-1. These worked as expected.
- I did not try LEDE r1006 with the older ipset & hnet-full.

My questions:

- What other troubleshooting information would be valuable?
- Is the list the right venue for this kind of report?

Thanks!

Rich

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 11:34:20 +0200

> The problem came when “Step 3 - Converting WAN interface to Homenet”
> didn’t work as described. I clicked Delete for the WAN interface.

Don't use the web interface.  Ssh into the router, and use vi on
/etc/config/network, it's easier and more reliable.

Here's how we make a Homenet router:

  opkg update
  opkg install hnet-full ip tcpdump strace
  vi /etc/config/system
  vi /etc/config/network
  vi /etc/config/wireless
  vi /etc/config/firewall
  /etc/init.d/network restart
  (check the output for errors)
  sync
  reboot

I'll publish Dorine's configuration files when I'm back in Paris, they're
a little beyond what you're interested in (they set up two mesh interfaces
and one AP on each node), but they're a good example.

-- Juliusz

---

From: Tim Coote <[email-masked]>
Date: Sat, 23 Jul 2016 11:10:04 +0100

> On 23 Jul 2016, at 10:34, Juliusz Chroboczek <[email-masked]> wrote:
> 

Configuring and smoke testing >1 router, > once starts to look like a devops problem. Not least because getting consistency becomes very error prone and difficult to reconstruct what worked/failed.  fwiw, I found that tools like R/make + git on a controlling computer make the work more repeatable.  I see that others are using more mainstream tools such as Ansible.

tim

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 12:25:05 +0200

> Configuring and smoke testing >1 router, > once starts to look like
> a devops problem.

I think you're overthinking it.  Once your routers are up and reachable
from outside, propagating a new version of the config files is easily done
using scp.  Matthieu tends to automate that using a throw-away shell
script, I prefer to do it by hand so I know when something breaks.

-- Juliusz

---

From: Tim Coote <[email-masked]>
Date: Sat, 23 Jul 2016 16:31:02 +0100

> On 23 Jul 2016, at 11:25, Juliusz Chroboczek <[email-masked]> wrote:
> 

scp + control (bash is fine, if that’s what comes to hand) was all that I had in mind, with git to manage variation over time and so that I could see what I’d actually set up. Smoke tests just catch the intermittent errors, that either I’d typically miss or get frustrated restarting a bunch of devices until they came up in the right order.  

Maybe I’m just much less good at getting the configurations right first time ;-)

Whatever works.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 23 Jul 2016 21:25:17 +0200

> Maybe I’m just much less good at getting the configurations right first
> time

In my experience, it's only the first two or three years that are difficult.

-- Juliusz

---

From: Rich Brown <[email-masked]>
Date: Tue, 26 Jul 2016 23:17:21 -0400

> On Jul 23, 2016, at 5:34 AM, Juliusz Chroboczek <[email-masked]> wrote:
> 

OK. I read all the documentation I can find (I note that http://www.homewrt.org is off the air at this time), and have come up with what I thought would be a good process for configuring Homenet. Unfortunately, it doesn't quite work.

Rather than describe the symptoms, I think it might be more efficient for someone to read through the steps outlined at https://gist.github.com/richb-hanover/ec88b851c4da074e48003e6fe9276901 and tell me if I've got it right? 

I would appreciate corrections, suggestions, edits, etc. Many thanks.

Rich

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 27 Jul 2016 13:02:45 +0200

> Rather than describe the symptoms, I think it might be more efficient
> for someone to read through the steps outlined at
> https://gist.github.com/richb-hanover/ec88b851c4da074e48003e6fe9276901
> and tell me if I've got it right?

Looks good to me.  You've correctly removed the software bridge and run
everything on the right interfaces.

Note that the firewall configuration is probably not necessary, hnetd does
weird things to the firewall.  But it doesn't harm, and it allows you to
easily switch back to a static configuration if Homenet doesn't meet your
needs.

If I were you, I'd explicitly tell hnetd that the E0 interface is external
("option mode external"), since I don't trust the edge detection mechanism.

I suggest adding two things:

  - before you start hacking, write down the IPv6 link-local address of
    the LAN interface, to make sure you can log in;
  - you should review /etc/config/upnpd, and list the interfaces you want
    to allow NAT-PMP on.

-- Juliusz

---

From: Rich Brown <[email-masked]>
Date: Wed, 27 Jul 2016 09:14:52 -0400

Thanks for these comments. I'll review them, try them out on my routers, and come back with further questions.

Rich

> On Jul 27, 2016, at 7:02 AM, Juliusz Chroboczek <[email-masked]> wrote:
>

---

From: Rich Brown <[email-masked]>
Date: Thu, 28 Jul 2016 19:32:25 -0400

Thanks for reading this through.

> everything on the right interfaces.

OK. I updated the gist as a result of these comments. https://gist.github.com/richb-hanover/ec88b851c4da074e48003e6fe9276901

> easily switch back to a static configuration if Homenet doesn't meet your

Should the procedure remain silent on the firewall? Or should people just put all interfaces into the lan zone? Or something else?

> ("option mode external"), since I don't trust the edge detection mechanism.

I think you mean E1 - that's what the instructions use for the wide area interface. And 'option mode external' goes in /etc/config/network? 

And is there a man page for hnetd that gives other options? The Googles didn't help me...
>    the LAN interface, to make sure you can log in;

Done. The newest draft also tells people to make backups :-)

>  - you should review /etc/config/upnpd, and list the interfaces you want

I'm not quite sure how that affects things. Where would NAT-PMP be important in my home net? 

And finally, a prosaic question: There is a wide variety of single quote usage in config files. Are there any places that quotes are mandatory? What is the best practice here?

Many thanks!

Rich

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 29 Jul 2016 13:12:04 +0200

> Should the procedure remain silent on the firewall? Or should people
> just put all interfaces into the lan zone? Or something else?

Pleae leave it as it is, it's fine.  I'd remove the bits about making E0
internal, it just confuses things -- if the router has a 4-port switch,
it's not likely that getting an extra internal interface is likely to be
a critical feature.

> I think you mean E1 - that's what the instructions use for the wide area
> interface.

It looks like one of us is confused.

> And 'option mode external' goes in /etc/config/network?

Yes, in the interface section.

> And is there a man page for hnetd that gives other options?

https://wiki.openwrt.org/doc/uci/network#protocol_hnet_self-managing_home_network_hncp

> Done. The newest draft also tells people to make backups :-)

Please tone it down -- no need to frighten people, and if things go wrong,
you just boot into failsafe mode and either fix things or reset the router:

  https://wiki.openwrt.org/doc/howto/generic.failsafe

Please use "sysupgrade -b backup.tar.gz" to do the backup -- I'd rather
you didn't mention the web interface at all, I've found it to be more
hassle than it's worth, and it's not included by default in the snapshot
(unstable) builds.

> important in my home net?

It allows clients to perform port redirection automatically.  Firewalls
are evil.

I also add the following to the firewall config:

config rule
        option target 'ACCEPT'
        option src 'wan'
        option name 'Accept-v6'
        option family 'ipv6'
        option dest 'lan'
        option dest_port '1024-65535'

Of course, since you've renamed wan and lan, you'll need to tweak the
relevant tweakanda.

> And finally, a prosaic question: There is a -wide variety of single
> quote usage in config files. Are there any places that quotes are
> mandatory?

The syntax is the same as the shell's.  Actually, the config files are
shell script fragments -- 'config' and 'option' are shell functions,
defined in /lib/functions.sh.

> What is the best practice here?

Put single quotes everywhere, it avoids having to think about special
characters.

Thanks again for your work, Rich.

-- Juliusz

---

From: Rich Brown <[email-masked]>
Date: Fri, 29 Jul 2016 10:42:37 -0400

Thanks again for your comments. They're reflected in the new draft: https://gist.github.com/richb-hanover/ec88b851c4da074e48003e6fe9276901

> On Jul 29, 2016, at 7:12 AM, Juliusz Chroboczek <[email-masked]> wrote:
> 

Done.

> It looks like one of us is confused.

It looks like it was me. :-) 

> Yes, in the interface section.

I have updated E0 /etc/config/network to mention "option 'mode' 'external'"

> https://wiki.openwrt.org/doc/uci/network#protocol_hnet_self-managing_home_network_hncp

Added to the External References section along with a link to RFC7788

> Please tone it down -- no need to frighten people,

Actually, I'm writing this for a less knowledgeable audience. (Me, for example. I'm learning as I go along, with your kind assistance :-) 

My goal is to produce an approachable document that has a good balance between background information and "Just Do This". I want to produce the "guide" that I wish I had when I started with Homenet. That's why I'm also adding in bits of knowledge (in the External References, Troubleshooting Procedures, etc.) that might help other people get themselves out of trouble.

This is a corollary of your assertion at IETF the other day ("if it’s not implemented, it didn’t happen") It's also true that if no one can install it, it's not going to be tested in the real world. :-)

> and if things go wrong,

This is useful advice. There's now a link to the page in the Troubleshooting Procedures. 

<rant> That OpenWrt page is a primary example of the style of documentation I abhor. It's enough to make the shields come down over my eyes. Huge blocks of text. Starts with a description of what it does ("bypasses all configuration...") without initially saying *why* you might want to use it. Three alternative methods, giving equal prominence to a) the most common case that's easy and will work 99% of the time, b) a super-techie process (packet sniffer) and c) using a serial connection that most people don't have. In fact, I had never looked at that page until you recommended it, since it was so inscrutable, and I wasn't sure it would ever apply to me...</rant>

> hassle than it's worth, and it's not included by default in the snapshot

I have added sysupgrade to the Backup procedure. But I like to recommend the LuCI GUI because:
	a) it gets the backup off the device onto my computer
	b) it date-stamps the backup
	c) it adds the router's name to the filename

> Of course, since you've renamed wan and lan, you'll need to tweak the

I have added both these to a "Next Steps" section since they're not core to switching to hnet. I don't know much about them, and will study them more when I get time.

> Put single quotes everywhere, it avoids having to think about special

Done.

> Thanks again for your work, Rich.

And thank you for your patience. 

Rich

---

From: Rich Brown <[email-masked]>
Date: Fri, 29 Jul 2016 11:16:07 -0400

Also... I'm aware that the Installing Homenet guide elides the reasons for using it. Do we have a short paragraph that tells *why* a lay person would want to use Homenet? 

If I'm not mistaken, these are the big attributes:

- No configuration - Homenet routers figure out how things are connected and do the right thing
- Local devices (printers, servers, etc) can be discovered easily (using mDNS, et al, but I don't want to use those acronyms...)
- Is it true that if a vendor shipped a fully-Homenet router, you could take it out of the box, plug it in, and you'd be 'on the air'?

I looked to see if there was anything on homewrt.org (it's currently down, but the Wayback Machine archived a few pages - https://web.archive.org/web/20150912135701/http://www.homewrt.org/doku.php ) Nothing there that I could use. 

Thanks.

Rich

> On Jul 29, 2016, at 10:42 AM, Rich Brown <[email-masked]> wrote:
>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 29 Jul 2016 22:06:21 +0200

> It looks like it was me. :-)

Why not rename the interfaces to "internal" and "external"?

> This is a corollary of your assertion at IETF the other day ("if it’s
> not implemented, it didn’t happen")

The next slide said "If it's deployed, it did happen".  With HNCP, we're
in the gray area between the two.

> <rant> That OpenWrt page is a primary example of the style of
> documentation I abhor.

Please speak to the LEDE guys.  Have a chat with "jow" and "HentaiMonster"
on Freenode, either on #openwrt or on #lede-dev.  "Nbd" is also extremely
competent, but he's more difficult to catch.

> I have added sysupgrade to the Backup procedure. But I like to recommend
> the LuCI GUI...

De gustibus non disputandum.  That's Latin for "you're wrong and I'm right" ;-)

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 29 Jul 2016 23:28:09 +0200

> - No configuration - Homenet routers figure out how things are connected

Agreed, but see below.

> - Local devices (printers, servers, etc) can be discovered easily (using
>   mDNS, et al, but I don't want to use those acronyms...)

Yes, although I understand this is still in flux.

> - Is it true that if a vendor shipped a fully-Homenet router, you could
>   take it out of the box, plug it in, and you'd be 'on the air'?

That's just restating point (1) above in more user-friendly terms.

I think that the above is a little dry, you need to speak of user-visible
features:

 - you can extend your wireless range by just connecting an extra router.
   As the network expands, the routers will automatically compute the best
   paths and route packets accordingly;

 - new routers can be connected wirelessly -- you can put a new router in
   any place that's in range of an existing router.  Note that this
   requires some configuration (putting an interface in client mode, or
   putting a bunch of interfaces in ad-hoc mode);
pp
 - ability to use multiple providers at the same time -- connect your
   Homenet routers to two providers, and the network will automatically
   route to one, the other, or both, depending on whether you have
   sufficiently smart host software.  This requires no cooperation from
   the provider.

The Commission for Truth in Advertising requires me to mention that the
last feature is not quite ready yet -- with current software, when one
provider goes down, you might need to manually disable the router that's
connected to that provider.

-- Juliusz

---

From: Gert Doering <[email-masked]>
Date: Sat, 30 Jul 2016 16:46:07 +0200

Hi,

>    paths and route packets accordingly;
[..]
> connected to that provider.

The Commission for Truth in Advertising also remarks that the "just
expand your wireless range by hooking up additional routers" is not working
very nicely with (wifi-)roaming clients yet, as the clients today seem
to have funny ideas about TCP sessions and new v6 addresses when changing
AP...

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 30 Jul 2016 17:29:22 +0200

> to have funny ideas about TCP sessions and new v6 addresses when changing

The Commission for Truth in Advertising has been doing some excellent work
of late.

-- Juliusz

---

From: Rich Brown <[email-masked]>
Date: Sat, 30 Jul 2016 12:54:16 -0400

> On Jul 29, 2016, at 4:06 PM, Juliusz Chroboczek <[email-masked]> wrote:
> 

Or, follow the pattern of the W24 and W5 interface names, and use EINT and EEXT
Or W-24, W-5, E-INT, E-EXT
Or an "h" prefix, for homenet: hW24, hW5, hInternal, hExternal
Or, because names seem to be converted to uppercase, use a "H-" prefix: H-24, H-W5, H-Internal, H-External
I'm not in love with any of these: they all kind of hurt my eyes. But I will try one variation in my next draft.

> in the gray area between the two.

If a package is available in the woods, but nobody tries it out, does it make a sound? (That's one of my motivations for this new guide - to broaden the test base.)

> competent, but he's more difficult to catch.

I have permissions edit the OpenWrt wiki. However, I have never taken the time to become sufficiently knowledgeable on the subject to feel comfortable that my rewriting would improve its quality. (Thus far - touch wood - I have successfully avoided needing to know about failsafe booting...)

> De gustibus non disputandum.  That's Latin for "you're wrong and I'm right" ;-)

:-)

re: my dry description of homenet. 

I agree - it's dry. I was hoping to trick someone on the list into blurting out something memorable (sometimes it works :-) 

I'll draft a 30-second "elevator pitch" about homenet and post it to the list. It'll definitely include:
	- no/minimal configuration; 
	- naming; 
	- some use cases (repeating wireless). 
	- oh, and IPv6. 
	- Anything else?

Thanks again.

Rich