Subject: [homenet] ISIS wifi testing

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 16 Oct 2015 15:59:55 +0200 (CEST)

Hello.

I have spent a large chunk of today doing wifi testing with the quagga 
implementation of homenet isis. 
https://plus.google.com/110006881111138232905/posts/UwYS9h2n7eM

I have been using iperf3 sending 2 megabit/s of UDP:

iperf3 -l 1400 -R -u -b 10M -t 6000 -c 10.0.58.140

The setup is that I set up a WDR4900 with one connection to the Internet 
(not really relevant), and one wired connection to an Ubuntu machine. I 
then set up two additional WDR4900 on my sons tricycle, plus a laptop.

           +----W1-----+R1
C1 +----+R3            +
           +            +
           +----W0-----+R2+---+C2

C1-R3, R2-C2, R1-R2 are wired connections.

W0 is 5GHz radio.
W1 is 2.4GHz radio.

I'm running all radios at 10mW.

If I position the setup just outside the room the R3 is located in, W0 has 
better SNR and lower metric, and is thus used. As I move further away from 
R3, W1 will start to get better SNR compared to W0, as W0 is degrading 
more per physical distance compared to W1.

Generally I only see very little packet loss as long as at least one of 
the radios has decent radio performance. I can go back and forth between 
W0 and W1 being the best radio with usually just 0-10 packets lost out of 
893 packets per second, usually it's 0-3.

I spent part of the day doing testing between laptop and a VM on my normal 
laptop, but I just in the past hour discovered that this VM causes packet 
loss. I replaced it with another computer and now all the spurious packet 
loss I was seeing before even using cable, is gone.

So this is a very simple setup, and it's also loop-free at least in one 
direction, traffic R3->C2 is loop-free either via R2 and R1, whereas 
R2->C1 can loop at R1 until R1 has converged its routing table due to a 
change received from R2.

Also, the above UDP test is only in one direction. How should I record 
the testing, should I have two sessions, one in each direction, and just 
log the results to file, so we see per-second result of packets 
sent/received and packet delay variance (iperf3 will give a value there).

I mean, from setting up everything and then just powering it up and moving 
it around, it basically "just works". I can move the rig out of coverage, 
it'll connect and start working as soon as the radios are up, and when 
there is a lower SNR radio, it'll move to it without any major packet 
loss.

I could for instance make a screen shot video of 10 minutes of testing 
with all the values scrolling, on the screen including the homenet web 
"bubble" diagram in the corner somewhere, and "ip monitor" running so the 
metrics can be seen continously.

Suggestions for tests greatly appreciated.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Dave Taht <[email-masked]>
Date: Fri, 16 Oct 2015 16:07:26 +0200

flent.org, dude, has tons of tests, lovely graphs, and so on.The rrul
test was the one intended for 802.11e in the first place. the
rtt_fairness tests are good for testing what happens when routing
happens.

On Fri, Oct 16, 2015 at 3:59 PM, Mikael Abrahamsson <[email-masked]> wrote:
> (not really relevant), and one wired connection to an Ubuntu machine. I then
> set up two additional WDR4900 on my sons tricycle, plus a laptop.
>
>           +----W1-----+R1
> C1 +----+R3            +
>           +            +
>           +----W0-----+R2+---+C2
>
> R3, W1 will start to get better SNR compared to W0, as W0 is degrading more
> per physical distance compared to W1.
>
> Generally I only see very little packet loss as long as at least one of the
> radios has decent radio performance. I can go back and forth between W0 and
> W1 being the best radio with usually just 0-10 packets lost out of 893
> packets per second, usually it's 0-3.
>
> direction, traffic R3->C2 is loop-free either via R2 and R1, whereas R2->C1
> can loop at R1 until R1 has converged its routing table due to a change
> received from R2.
>
> Also, the above UDP test is only in one direction. How should I record the
> testing, should I have two sessions, one in each direction, and just log the
> results to file, so we see per-second result of packets sent/received and
> packet delay variance (iperf3 will give a value there).
>
> it'll connect and start working as soon as the radios are up, and when there
> is a lower SNR radio, it'll move to it without any major packet loss.
>
> I could for instance make a screen shot video of 10 minutes of testing with
> all the values scrolling, on the screen including the homenet web "bubble"
> diagram in the corner somewhere, and "ip monitor" running so the metrics can
> be seen continously.
>
> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet

-- 
Dave Täht
Do you want faster, better, wifi? https://www.patreon.com/dtaht

---

From: Michael Richardson <[email-masked]>
Date: Fri, 16 Oct 2015 15:35:42 -0400

Mikael Abrahamsson <[email-masked]> wrote:
    > The setup is that I set up a WDR4900 with one connection to the
    > Internet (not really relevant), and one wired connection to an Ubuntu
    > machine. I then set up two additional WDR4900 on my sons tricycle, plus
    > a laptop.

On his tricycle, so you can move it around, I guess.
I think that it's R1/R2 + C2 that are on the tricycle.

Let me suggest two additional "challenges"

1) get a $20 hands-free phone.
   like: http://www.amazon.ca/Vtech-T2326-Cordless-Phone-Caller/dp/B000BTDLVK
   Can be found at garage sales. People get rid of them for a reason.
   Emphasis on: "Enhanced analog 2.4 GHz technology"
   meaning, when off the hook, it eats all of channels 1 and 2.
   (it also, uses 900Mhz for the other direction. Weirdly, it doesn't use them
   in the direction I'd think would reduce battery drain...)

   Repeat your experiment.

2) get a Lexmark printer, like:
   http://www.ebay.com/itm/NEW-Lexmark-X4650-Wireless-Multifunction-3-in-1-Printer-Scanner-Copier-/321760765408?hash=item4aea6fa5e0:m:mZzBHEAZOlVGrAthONtP6yw

And well, just plug it in.
My experience with them is that every 3-5 minutes, it announces itself on all
802.11b channels (or maybe it's just straight 802.11g without any care about
listening for b-type users first).  There is no way to disable the wifi that
I found short of the power cord, even if you only care about USB connection.
My neighbour had one of these: its why we switched to 5Ghz.
   (I might not have the exact model right)
In the end, their kid consumed too much ink, they bought something else,
and they lock the power cord up so it can't be used without permission.

You might want to create a network like:

          +----W1-----+R1+++++R4---W3---R6
C1 +----+R3            +      +         +
          +            +      +         +
          +----W0-----+R2+++++R5---W2---R++C2

Or perhaps R1/R4 are the same router with two ESSIDs configured.
Maybe seperate R1/R4, R2/R5 (they are connected by cables), so that they are
in different broadcast domains...

And also make W1==W3 ESSID, and W0==W2, just in different places, and walk
the tricycle around.  Do you have some metals doors?
{and don't forget to deduct the price of the tricycle as a "research cost"! :-)}

That's particularly interesting because there are only a single set of TxOps.
Note that this is essentially very similiar to the problem outlined in (physical)
traffic planning by Brian Hayes at:
        https://www.americanscientist.org/issues/pub/2015/4/playing-in-traffic

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Chris Elliott <[email-masked]>
Date: Fri, 16 Oct 2015 17:27:59 -0400

Mikael,

Any chance you can repeat the tests using multi and broadcast traffic? May
not be enough clients...

Thanks!
Chris.

On Friday, October 16, 2015, Mikael Abrahamsson <[email-masked]> wrote:

> the radios has decent radio performance. I can go back and forth between W0
> and W1 being the best radio with usually just 0-10 packets lost out of 893
> testing, should I have two sessions, one in each direction, and just log
> the results to file, so we see per-second result of packets sent/received
> and packet delay variance (iperf3 will give a value there).
>
> there is a lower SNR radio, it'll move to it without any major packet loss.
>

-- 
Chris Elliott
CCIE # 2013

“You and I are mirages that perceive themselves”
--Douglas Hofstadter

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 17 Oct 2015 12:03:36 +0200 (CEST)

> Any chance you can repeat the tests using multi and broadcast traffic? May

Yes, I can do both the "p2mp" and multicastcast modes, that's fairly easy 
to test.

However, as you say, I don't expect there to be much multicast loss 
because 2.4GHz isn't hugely congested where I live.

Now that I have a better feel for the setup and most of the bugs worked 
out (I hope), on monday I'll do quite a few more tests and I also intend 
to use babel and try a few similar test cases with it and compare.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 23 Oct 2015 14:38:40 +0200 (CEST)

>          +----W0-----+R2+++++R5---W2---R++C2

I ended up doing this instead (I didn't have enough routers to do what you 
suggested above, and I thought the below test case would better test a 
real-world scenario with multuple APs and devices fading or moving between 
them).

                                  C2
                                  |
          +---------W0-------+R1==R2+--W2-+
C1 =====R3                               |
          |                               |
          |                               +
          =============R4+-----W1--------+R5

So the R1/R2/C2 combination can be rolled around so that the best 
wifi is either R3 or R5. I am running an iperf3 TCP session between C1 and 
C2.

I tried both with a cable between R4 and R5 (shortcut) and only with the 
above wifi. I tested both with and without cable, and both with Babel and 
Quagga ISIS. Below are my findings. I have several hours of video, but the 
last test I am basing my below findings on is a 47 minute video. I'm 
uploading it to Youtube now. https://youtu.be/VXE5_cOcTvw (it's still 
processing if you check it right now). I kept a window open to the right 
where I wrote what I did, as I did it.

I'd say both protocols/implementations are doing a decent job. Babel, not 
knowing anything about the actual radio parameters, tends to stick around 
on a going-bad wifi, until it's really bad and basically stops working, 
then it switches around. Both protocols really prefers single wifi hop 
instead of 2 wifi hops, even if the 2 wifi hops are a lot better than the 
single wifi hop. Babel tends to switch fairly slowly, taking a 
conservative approach to path selection (which is to be expected 
considering the design criteria as I understand them).

With just single wifi hops (cabled connection R4-R5), ISIS is able to 
re-route from one wifi to the next a bit earlier than Babel, as the one 
it's on is going really bad.

In order to actually optimize wifi, I'd say both protocols need to know 
more about the speed the wifi is averaging at, and SNR would also be of 
interest (currently IS-IS only uses SNR). None of them are doing a great 
job, but to do a great job, I'd imagine the solution would be a lot more 
complicated. Description of the metric daemon for IS-IS is here:

https://git.netdef.org/projects/OSR/repos/openwrt-isis-hnet/browse/etxrd

I still do not know what topologies we're expecting to see. Both protocols 
are today behaving decently when it comes to wifi connections coming and 
going, getting worse, getting better, at least trying to avoid the really 
bad wifi connections. Babel tends to be "stickier". Both suffer from 3-10 
seconds outages when some things happen, this might even be related to 
hnetd changing things around, I don't know for sure. It's very frustrating 
that by default there are no loopback addresses for the routers that are 
the addresses used for DNS entries. Also, in current implementation there 
is no stickyness for addresses on interfaces, if it goes down, it'll come 
back up again with new prefix.

I haven't seen anything that indicates prolonged microloops for IS-IS 
during my testing. I've kept ping sessions (one per second) going and 
always just seen the expected TTL on replies.

For now I have only tested IS-IS with "ipv6 and P2MP" setup. I'm going to 
test in broadcast mode now and see if there are some obvious differences, 
I don't believe there will be since my radio environment is not extremely 
crowded and they're mostly used for point-to-point link.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 23 Oct 2015 14:57:13 +0200 (CEST)

> uploading it to Youtube now. https://youtu.be/VXE5_cOcTvw (it's still

https://youtu.be/j3-651rmClc instead, Youtube didn't do what I expected it 
to do... It'll still processing the higher resolution videos, it's 
unreadable in 360p.x

---

From: Gabriel Kerneis <[email-masked]>
Date: Fri, 23 Oct 2015 16:47:14 +0200

Hi Mikael,

thanks for the testing.

On Fri, Oct 23, 2015 at 2:38 PM, Mikael Abrahamsson <[email-masked]>
wrote:

> single wifi hop. Babel tends to switch fairly slowly, taking a conservative
> approach to path selection (which is to be expected considering the design
> criteria as I understand them).
>

Are all your AP on the same frequency? Did you enable diversity-sensitive
routing for babel? If not, you should add in /etc/config/babeld:

config general
option diversity true

Best,
Gabriel

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 23 Oct 2015 17:50:02 +0200 (CEST)

On Fri, 23 Oct 2015, Gabriel Kerneis wrote:

> Are all your AP on the same frequency? Did you enable

No, I made sure they're all on different frequencies. One of the hops is 
on 2.4GHz, the two others are on two different 5GHz 20MHz bands.

> diversity-sensitive routing for babel? If not, you should add in 
> /etc/config/babeld:
>
> config general
> option diversity true

What does that do?

---

From: Gabriel Kerneis <[email-masked]>
Date: Fri, 23 Oct 2015 17:53:19 +0200

> diversity-sensitive routing for babel? If not, you should add in
Prefer paths that avoid interference:
https://tools.ietf.org/html/draft-chroboczek-babel-diversity-routing-00
Supposedly useful in the precise case of multiple AP on different
frequencies.

Gabriel

---

From: Dave Taht <[email-masked]>
Date: Fri, 23 Oct 2015 18:05:52 +0200

Dave Täht
I just lost five years of my life to making wifi better. And, now...
the FCC wants to make my work, illegal for people to install.
https://www.gofundme.com/savewifi

>>> diversity-sensitive routing for babel? If not, you should add in

Except! that there is a bug in sensing the actual channel in babel on
some radios, when last I looked. (the linux api for detecting it
changed)

Add also:

option channel X

Yes this is pita.

---

From: Henning Rogge <[email-masked]>
Date: Fri, 23 Oct 2015 18:21:19 +0200

> some radios, when last I looked. (the linux api for detecting it

The nl80211 API changed?

Henning Rogge

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 23 Oct 2015 18:23:13 +0200 (CEST)

> Supposedly useful in the precise case of multiple AP on different 

But since I manually optimized this, it would not have changed anything, 
right?

---

From: Gert Doering <[email-masked]>
Date: Fri, 23 Oct 2015 18:52:12 +0200

Hi,

On Fri, Oct 23, 2015 at 02:38:40PM +0200, Mikael Abrahamsson wrote:
> It's very frustrating 

I assume that this is just an implementation issue - and in that case,
would indeed second Mikael's complaints :-) - having a sticky address
on the router to talk to, no matter which interface happens to be up or
down just now, and have interfaces that keep their subnet for the 
"easy" things ("user unplugs and replugs his laptop") that do not
require persistant storage would certainly be appreciated.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Gabriel Kerneis <[email-masked]>
Date: Mon, 26 Oct 2015 10:03:12 +0100

> But since I manually optimized this, it would not have changed anything,

I think that's right, yes. The diversity optimization can only help babel
make smarter choices if there is an actual choice to make between two
radios for the same path.

Gabriel

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 29 Oct 2015 14:36:34 +0100

Thanks a lot for doing that, Mikael.  Dynamically computed metrics are an
essential aspect of zero-configuration routing, sadly overlooked by the
IS-IS community in the past.

> I'd say both protocols/implementations are doing a decent job.

Interesting.  I didn't expect SNR to be such a good predictor of usable
throughput.  I'd be interested to see something similar in a crowded
environment (see the "cross-traffic" note at the bottom of this mail).

> Babel[d], not knowing anything about the actual radio parameters, tends to
> stick around on a going-bad wifi, until it's really bad and basically
> stops working, then it switches around.

(I'm trying to distinguish babeld, the implementation, from Babel, the
protocol.)

Yes.  Babeld applies hysteresis before route selection in order to avoid
excessive oscillations in highly congested environments.  You can disable
the hysteresis mechanism by setting

  smoothing-half-life 0

in the config file.  The default is 4, which means that we'll switch 8 to
12 seconds later than expected from ETX computation, or when ETX yields
infinity.

> Both protocols really prefers single wifi hop instead of 2 wifi hops,
> even if the 2 wifi hops are a lot better than the single wifi hop.

[...]

> interest (currently IS-IS only uses SNR).

Right.  Babeld should multiply the result of ETX by 1/rate, where rate is
the data rate chosen by the link layer.  That's a little tricky to do
right (it requires consulting the ND cache in order to map link-local IPv6
addresses to MAC addresses), and I have more urgent stuff on the table.

> None of them are doing a great job, but to do a great job, I'd imagine
> the solution would be a lot more complicated.

As far as I know, nobody's doing a great job.  Radio is weird, and finding
a set of heuristics to accurately predict throughput is an open research
problem.  There's been a lot of papers about "cross-layer approaches", but
most researchers are only evaluating in simulation.

(Meraki might have something, but since they've gone proprietary and
secretive, they've excluded themselves from the community.  But then,
they appear to be concentrating on manageability nowadays, so perhaps not.)

> I still do not know what topologies we're expecting to see.

Experience with my users -- admittedly a biased sample, and not necessarily
representative of the Homenet userbase -- shows that they care about
worst-case behaviour in a stationary network.  They have no problem
waiting for a few seconds after a mobility event; on the other hand, they
do object if the routing exhibits instability in the presence of
congestion and interference.  Hence the hysteresis algorithm, and its
conservative default tuning of a half-life of 4s.

> test in broadcast mode now and see if there are some obvious differences,

I'm very much interested in that.

> crowded and they're mostly used for point-to-point link.

You should test with cross traffic:

    A .................. B
    .                    .
    .                    .
    .                    .
    C .................. D

Measure from A to D while simultanously running iperf from C to B.  You
should measure both ping (low traffic) and iperf packet loss (high throughput).

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 29 Oct 2015 14:44:35 +0100

> routing for babel? If not, you should add in /etc/config/babeld:

I disagree, Gabriel -- since IS-IS doesn't do diversity routing, the
comparison is more interesting if babeld doesn't either.  I'd much rather
have two comparisons:

  - IS-IS compared to non-diversity Babel;
  - diversity Babel compared to non-diversity Babel.

We did some work on that with Gabriel a few years ago, and we found out
that the results are somewhat paradoxical: diversity Babel appears to beat
plain Babel in uncongested networks, but yields similar results in
congested ones.  My interpretation is that in a congested network packet
loss is a good indicator of interference, and therefore plain Babel
converges to a non-interfering set of routes without explicit interference
measurements.

The tests need repeating in a more formal setting, but they are extremely
time consuming, and more urgent stuff came up.

-- Juliusz