Subject: [homenet] Reachability of distributed prefixes

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 09:22:46 +0200

Hi,

I am experimenting with SHNCPD at the moment and wonder about a detail
in the Homenet prefix distribution to attached hosts.

Does HNCP somehow make sure that there is a route towards the prefix
it distributes? While D/HNCP checks that there is a path of links, the
routing protocol might decide that one of the links is too
unstable/slow for traffic and does not use it for routing.

What is the preferred way to deal with this situation?

Henning Rogge

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 26 Aug 2015 09:42:14 +0200 (CEST)

> What is the preferred way to deal with this situation?

Hm, there must be some preconception here that I do not understand.

Why would a routing protocol choose to decide not to use a "bad" link if 
there are no other alternatives available? Bad links should be avoided if 
there are better available, but if this is the only one available it 
should be used anyway?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Steven Barth <[email-masked]>
Date: Wed, 26 Aug 2015 09:45:26 +0200

Hello Henning,

> What is the preferred way to deal with this situation?

HNCP has the following requirement wrt. (applied) assigned prefixes:

[...] each router connected to said [Common] link:
MUST forward traffic destined to said prefix to the respective link.

which should ensure that once traffic for a prefix reaches any router
adjacent to a link where it is assigned, is delivered to the link.

Everything else wrt. routing and interactions is more or less declared
out of scope for HNCP.

Cheers,

Steven

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 11:26:17 +0200

On Wed, Aug 26, 2015 at 9:42 AM, Mikael Abrahamsson <[email-masked]> wrote:
> there are better available, but if this is the only one available it should
> be used anyway?

It is not uncommon for wireless links to use some kind of hysteresis
on a routing protocol. The problem/feature of D/HNCP is that it is
independent of the routing protocol... so it does not know.

Henning Rogge

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 11:30:37 +0200

> MUST forward traffic destined to said prefix to the respective link.

Here's the mechanism in the shncpd implementation, Steven will hopefully
tell me if that's what's intended:

  - for each locally delegated prefix P, we install:
     - a source-specific default unreachable route (::/0, P);
     - an unreachable route to P;
  - for each prefix Q assigned (and applied) to an interface E, we install:
     - a route to Q through E.

All of those routes are announced over Babel, even the unreachable routes.
Babel propagates them in their usual manner.  If at a given point in the
Homenet the Babel metric for Q is finite, longest-prefix routing will call
it to be routed to the nearest (according to Babel's metric) router
attached to link E.  If the link quality is so bad that the metric for
Q is infinite, packets destined for Q will follow the covering route to
P and reach the router delegating P, where they will be dropped by the
unreachable route.  If both routes have infinite metric, then the network
is so bad that I really don't care.

Note by the way that the non-Homenet routes are *not* announced over
Babel, unlike in the hnetd implementation, which announces all routes.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 11:33:59 +0200

> independent of the routing protocol... so it does not know.

I'm not sure I'm following you.

All that shncpd does is to announce attached prefixes over the routing
protocol.  It is then the routing protocol's business to pick a path to
one of the routers advertising the prefix (or to drop all such routes, if
the link quality has collapsed too much).

-- Juliusz

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 11:39:15 +0200

My problem is not with the prefixes assigned to the interfaces of HNCP
routers itself, my problem is with the prefixes provided to attached
hosts.

If I understand HNCP right then the "uplink" will announce a prefix
which should be used by all routers for the attached hosts.

The problem will appear if HNCP learns about this prefix through DNCP
but the routing protocol cannot provide a route to the uplink (because
hysteresis decided one of the links is too bad).

I wonder if HNCP routers should apply all addresses/prefixes to its
local interfaces, but should check for an existing route to the HNCP
router that announce the prefix before providing it via DHCP or RA to
hosts.

This would not create a dependency loop between routing protocol and
HNCP because the routing protocol does only advertise the
addresses/prefixes configured on the HNCP interfaces. But it would
prevent HNCP to announce a prefix that is not reachable via the
routing protocol.

Henning Rogge

---

From: Markus Stenberg <[email-masked]>
Date: Wed, 26 Aug 2015 12:50:30 +0300

> router that announce the prefix before providing it via DHCP or RA to

I think this ties it too tightly to a RP; routes _can_ appear and disappear over time after all, so if you follow this logic to it’s logical conclusion, you would have to make decision that e.g. 95% of the time, this delegated prefix has functioned and it does not flap frequently, so we can subdelegate it. Leading to relatively much state duplicated both within RP (hysteresis) and HNCP daemon (long-term route validity/flap statistics/..).

I would personally rather just say ’too bad’ in such a network :) YMMV of course. I would not personally mind _implementations_ that did this, given they knew they used RP with hysteresis, but I disagree with sticking this in base spec itself.

Cheers,

-Markus

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 26 Aug 2015 11:50:58 +0200 (CEST)

> router that announce the prefix before providing it via DHCP or RA to 

Let me rephrase what I think you're saying and tell me if I'm correct:

Your worry is that HNCP will determine neighbors and speak HNCP well 
enough, but the routing protocol thinks the link is so bad, that it's 
effectively has partitioned the network (because this was the only link 
connecting the two (now) parts), and since there might be two uplinks, you 
want HNCP to detect this partition, and only hand out ISP1 prefixes on the 
side where ISP1 works, and only ISP2 prefixes, on the ISP2 side that 
works. Also, if the network is partitioned, you want prefixes no longer 
reachable to be sent corresponding RAs with zero lifetime etc, to make 
hosts no longer use them for new connections?

So one way of doing this would be for hnetd to check if the ISPx prefix 
(for instance /56) is in the routing table, and if it isn't, it should 
not use it to put addresses on interface etc, and if it goes away (at 
least for any duration of time), stop using it.

I don't remember seeing this discussed anywhere, but it might very well be 
something that should be done. I would imagine HNCP is reacting on ISP1 
WAN link going down and stopping to use the ISP1 prefix, but if there is a 
break within the homenet (routing protocol wise), nothing is done as long 
as HNCP is up.

One way of solving this would be to create fate-sharing between HNCP and 
the routing protocol. Ie, HNCP wouldn't do anything unless there is a 
valid routing protocol adjacency with that neighbor (=fate sharing).

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 11:52:46 +0200

> which should be used by all routers for the attached hosts.

Er... no.  The "uplink", or delegating router in Homenet parlance, only
announces a source-specific route.  It's the routers performing the
assignment that announce a route to the prefix.

> hysteresis decided one of the links is too bad).

That's not how Babel's hysteresis works.  But yes, the link quality
estimator and routing metric being used could decide that none of the
routes are usable.

> router that announce the prefix before providing it via DHCP or RA to

This would cause renumbering whenever the routing protocol has a hiccup.
Instead, what HNCP does is that it assigns one prefix for each delegated
prefix, and instructs hosts to grab one address for each assigned prefix.
Hosts are then expected to do the happy eyeballs/ICMP unreachable dance.

Or to run Babel.  (Ducks.)

-- Juliusz

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 11:58:49 +0200

On Wed, Aug 26, 2015 at 11:50 AM, Mikael Abrahamsson <[email-masked]> wrote:
>> I wonder if HNCP routers should apply all addresses/prefixes to its local
>> interfaces, but should check for an existing route to the HNCP router that
>> announce the prefix before providing it via DHCP or RA to hosts.
>
>
> Your worry is that HNCP will determine neighbors and speak HNCP well enough,
> but the routing protocol thinks the link is so bad, that it's effectively
> has partitioned the network (because this was the only link connecting the
> two (now) parts), and since there might be two uplinks, you want HNCP to
> detect this partition, and only hand out ISP1 prefixes on the side where
> ISP1 works, and only ISP2 prefixes, on the ISP2 side that works. Also, if
> the network is partitioned, you want prefixes no longer reachable to be sent
> corresponding RAs with zero lifetime etc, to make hosts no longer use them
> for new connections?

This is a good example.

> So one way of doing this would be for hnetd to check if the ISPx prefix (for
> instance /56) is in the routing table, and if it isn't, it should not use it
> to put addresses on interface etc, and if it goes away (at least for any
> duration of time), stop using it.

Yes. If DHCP server and radvd wait until the route to the prefix is
available in the routing table, we keep the decision about
"reachability" to the routing protocol without having a hard
dependency on it.

> something that should be done. I would imagine HNCP is reacting on ISP1 WAN
> link going down and stopping to use the ISP1 prefix, but if there is a break
> within the homenet (routing protocol wise), nothing is done as long as HNCP
> is up.
>
> One way of solving this would be to create fate-sharing between HNCP and the
> routing protocol. Ie, HNCP wouldn't do anything unless there is a valid
> routing protocol adjacency with that neighbor (=fate sharing).

Yes.

HNCP has its own "control plane" (and it needs this control plane),
but if we use the neighbor information within HNCP to decide if a
prefix is reachable we might create a different view than the routing
protocol, which has its own idea what is reachable and what not.

Henning Rogge

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 12:06:20 +0200

> "reachability" to the routing protocol without having a hard dependency
> on it.

So if a route is flapping, hosts get or don't get an IP depending on the
exact time when they send a DHCPREQUEST or NS?  Is that better than always
assigning an IP to hosts, and expecting ICMP to signal route flapping in
real time?

-- Juliusz

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 14:04:14 +0200

> assigning an IP to hosts, and expecting ICMP to signal route flapping in

Are you talking about a route that is created and vanishes every few
seconds or minutes?

I would guess some kind of hysteresis would be okay. Activate a prefix
if you had a route for X seconds... deactivate it if you loose it for
X seconds.

Still, assigning a source IP address without a route to it is a bad
thing, some client applications might just keep trying instead of
moving to a different source address.

Henning Rogge

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 14:31:49 +0200

> Are you talking about a route that is created and vanishes every few

What I'm saying is that neither DHCPv4 nor IPv6 RA are designed to deal
with prefixes that last less than a few hours.  See for example RFC 4862
Section 5.5.3 paragraph e2.  (That's just an example, there are other
reasons why yo-yo RAs are a bad idea.)

Short-term reachability indications are sent to hosts in a reactive manner,
using ICMP unreachables.  If any applications are unable to do the right
thing with ICMP unreachables, we should fix the applications.

-- Juliusz

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 15:07:50 +0200

> thing with ICMP unreachables, we should fix the applications.

I am not aware of any application doing anything more than "try to
open the connection again".

How do you propose the application to react? Most applications leave
the source-IP selection to the operation system...

does any OS currently change the preference order of IPv6 source
prefixes when it gets ICMP unreachable messages?

Henning Rogge

---

From: Philip Homburg <[email-masked]>
Date: Wed, 26 Aug 2015 15:27:19 +0200

>prefixes when it gets ICMP unreachable messages?

I never bothered to automate, but I regularly switch prefixes by changing
the preference times in RAs. That works very well.

Linking general ICMP unreachable messages to a source prefix sounds very hard
to me.

---

From: Henning Rogge <[email-masked]>
Date: Wed, 26 Aug 2015 15:36:08 +0200

> Linking general ICMP unreachable messages to a source prefix sounds very hard

RA's are generated by the HNCP router, so the generation could take
the routing table into account (put prefixes WITH routes in preferred
places).

If the routing protocol would export the "distance" to the destination
in form of the routing table metric value, the RA generation could
even make sure that the "closest" gateway is always the preferred one.

So we can manipulate the published RAs in a way that the hosts will
take normally one the HNCP router prefers?

Henning Rogge

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 26 Aug 2015 15:52:55 +0200

> the source-IP selection to the operation system...

Right.  That's a good point.

-- Juliusz

---

From: Gert Doering <[email-masked]>
Date: Sun, 30 Aug 2015 23:27:19 +0200

Hi,

> prefixes when it gets ICMP unreachable messages?

And that's a well-known issue that the IETF needs to finally tackle:
source-address failover.  This is way bigger than just homenet-related,
as it will basically affect every host that has multiple equally-scoped
IPv6 addresses where one of them has intermittent failures.

Hosts need to learn to cope - the failure could be in the ISP1 network,
so there is nothing HNCP could do inside the homenet to signal "uh, btw,
do not use prefix1 for connections to ISP_Z, it's broken right now" (first,
it does not know the scope, second, it has no mechanism to tell).

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 31 Aug 2015 13:16:28 +1200

On 31/08/2015 09:27, Gert Doering wrote:
> source-address failover. 

We did - it's called shim6. The trouble is that we can't deploy it
because of stupid firewalls blocking the necessary extension headers.

   Brian

> This is way bigger than just homenet-related,

---

From: joel jaeggli <[email-masked]>
Date: Sun, 30 Aug 2015 22:03:47 -0700

>> And that's a well-known issue that the IETF needs to finally tackle:

So long as you don't invoke the prospect of either extremely expnesive
overlay networks, or globably route scalability go right ahead those are
both in play already.

Hosts in in absence of state as turns out are rather good at
instantaneous renumbering. e.g. as they roam  between networks it
remains a mystery to me that networks  containy hosts are less able to
cope. I may be in fact that that they are not less able to cope.

> because of stupid firewalls blocking the necessary extension headers.

Give it a rest. we work with have we have, not might want other people
to deploy at great expense for our benfit.

> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet
>

---

From: Gert Doering <[email-masked]>
Date: Mon, 31 Aug 2015 15:20:26 +0200

Hi,

> because of stupid firewalls blocking the necessary extension headers.

And that it solves a different problem.

But otherwise, yes, shim6 would have been nice to see...

Gert Doering
        -- NetMaster

---

From: Gert Doering <[email-masked]>
Date: Mon, 31 Aug 2015 15:24:25 +0200

Hi,

> cope. I may be in fact that that they are not less able to cope.

No, that wasn't what I was talking about.  Not "I get a new source address
and need to cope it", but "I have two source addresses with global scope,
tried one according to source-selection rules, it did not work, so I 
should maybe try the other one now?"

No network dynamics of any kind involved, just "a normal homenet 
multi-ISP scenario" (with some breakage "somewhere upstream", not 
something the host can easily see) - I thought that this would have been 
obvious from the thread and WG list context.

The advertised IETF solution for "(SoHo) multihoming" is "multiple global 
IPv6 addresses", but this does not work yet as well as it could, partly 
due to this missing piece.

Gert Doering
        -- NetMaster

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 1 Sep 2015 08:44:13 +1200

> should maybe try the other one now?"

Yes, you're correct that shim6 doesn't do that, and has other deployability
issues that I noted. But unless we want every application and/or transport
protocol to contain its own variant of happy eyeballs, this functionality
(suck-it-and-see-until-it-works) would need to be inserted as a shim in the
socket layer or somewhere around there.

I'm not convinced there's a protocol involved, though. It's more a matter
of "history", in the sense used in draft-baker-6man-multi-homed-host.

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 1 Sep 2015 09:02:04 +1200

Joel,

> cope. I may be in fact that that they are not less able to cope.

As we observed in 6renum, today's homenets simply don't have a renumbering
problem. Every power cycle causes renumbering, even if you end up with the
same addresses again. Roaming hosts manage pretty well too, as we all
demonstrate at scale during every IETF meeting.

> to deploy at great expense for our benfit.

There's no particular deployment expense involved in a hypothetical world
where IPv6 stacks include shim6 and firewalls respect RFC7045. Trouble is,
we don't live in that world.

Fixing the problem that Gert raised isn't possible without host stack
updates. Shim6's mistake was to also require firewall updates. MPTCP
avoids that problem but only helps TCP applications.

    Brian