Subject: Support for RFC 7084 on shipping devices...

From: Ted Lemon <[email-masked]>
Date: Thu, 3 Oct 2019 17:39:15 -0700

(If you got this as a Bcc, it’s because I am hoping you can contribute to the discussion, but might not be on the mailing list to which I sent the question, so please answer on-list if you are willing.)

I’ve been involved in some discussions recently where the question has come up: how good is support for RFC7084 in shipping routers?   And what gaps exist in RFC7084 that could cause problems?   And in cases where RFC7084 support either isn’t present, or isn’t useful because no IPv6 or because ISP is delegating a /64, what things might work and what things might not, if we want bidirectional reachability between two separate network links in the home.

So for example, suppose we have "CE Router," which supports RFC7084, including prefix delegation.  And we have "Internal Router" on that network requests a delegation, and gets a prefix from the CE router.  Presumably that prefix is out of a larger prefix that CE Router got from the ISP.  Great so far.  Let’s call the network on the southbound interface of Internal Router “South Network”. Let’s call the network on its northbound interface, which is also the network on CE router’s southbound interface, “North Network.”

viz:

                                                ISP
                                                 |
                                             CE Router
                                                 |
North Network    |-------------------------------+--------------+-----------------|
                                                 |              |
                                           Internal Router      +---- Node A
                                                 |
South Network    |-----------+-------------------+--------------------------------|
                             |
                   Node B ---+

If I want hosts on South Network to communicate with hosts on North Network, what do I have to do?   Should Internal Router publish an RA on its northbound interface?   What is the likelihood of that being filtered by the network?   If packets for South Network are forwarded through CE Router, will it forward them on to Internal Router, forward them north, or drop them?

Similarly, suppose we have a network where unfortunately PD Isn’t available internally, but IPv6 is present on the northbound interface of the internal node and southbound interface of the CE router.   Suppose further that Internal Router allocates itself a ULA prefix and advertises that as reachable and on-link on its southbound interface, and as reachable but not on-link on its northbound interface.   Will that be blocked at layer 2 by CE Router?   I’m sort of assuming here that the CE router is managing the North Network link, which is probably WiFi.

Okay, now what if there’s no IPv6 support on CE Router or being provided by CE router on North Network.   Suppose Internal Router allocates a ULA and allocates two /64s out of the ULA, one of which is advertised as reachable on its northbound interface and on-link on its southbound interface, and a second of which is advertised as on-link on its northbound interface and reachable on its southbound interface.

Fourth possibility: Node A is manually configured with an IPv6 address on a prefix that Internal router is advertising as reachable on its southbound interface, but which is not advertised on South Network because of filtering.  Node B has an address on a prefix that Internal Router is advertising as on-link on its southbound interface.   Node A has a static route configured through Internal Router to the second prefix.   Is there any reason to think that traffic between Node A and Node B will be filtered at layer 2 by CE Router, assuming that traffic on North Network is all going through CE Router?

The goal here is to have bidirectional reachability between the two nodes on IPv6 using either a global prefix or a ULA.  The concern is that something could prevent each of these cases from working.   What I’m really curious about is whether people have experience with doing communications of this type using actual routers that ISPs are shipping.   Is this “internal network” scenario part of acceptance testing for these routers?  Is this all a big question mark?   In principle this should all work, unless RA guard is hyperactive in CE Router.   But what about in practice?

---

From: Ole Troan <[email-masked]>
Date: Fri, 4 Oct 2019 07:44:12 +0200

Ted,

[top posting]

RFC7084 does not have any support for internal routers. 

Futher:
It might just be the way you describe the use cases, there seems to be a misconception about how routers work with regards to ND “advertisements”. ND is not a routing protocol. 

Hierarchical PD which you also allude to, was proposed, has limitations and was not standardized. That why HNCP was done. 

If you have a set of rfc7084 routers I believe you are left with manual configuration of prefixes and either manually configured static routing or RIP. 

Cheers 
Ole

> On 4 Oct 2019, at 02:40, Ted Lemon <[email-masked]> wrote:
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 4 Oct 2019 08:20:32 +0200 (CEST)

> RFC7084 does not have any support for internal routers.

While this is true, OpenWrt does support DHCPv6-PD within the home, out of 
the box. I also have a report of AVN Fritzbox supporting sub-PD without 
additional configuration.

In all devices I've looked at the WAN is WAN, it comes up with firewalls 
on, requests PD etc, and if it doesn't get it then there is no GUA IPv6 on 
LAN.

In my opinion the work in homenet could be leveraged into an operational 
document where recommendations on what parts of homenet could be easily 
implemented to make it work within a home (without implementing 
everything), thinking primarily of "firewall off" and "service discovery 
proxy on". If no PD was available, turn ethertype 0x86dd bridging on 
between LAN and WAN. I guess we would still need to do NAT44 because 
without HNCP there wouldn't be a route to the IPv4 network on LAN of the 
"sub-router".

It would however mean that a printer on the sub-router LAN could be 
reached over IPv6. In order for this to happen without HNCP then this 
sub-router would need to send RAs on its WAN announcing reachability to 
its LAN IPv6 prefix (either GUA+ULA if PD is available, otherwise just 
ULA). I have never seen RA guard or similar functions in residential 
equipment, so I would expect this to work.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Fri, 04 Oct 2019 10:19:01 +0200

Ted Lemon <[email-masked]> wrote:
    > I’ve been involved in some discussions recently where the question has
    > come up: how good is support for RFC7084 in shipping routers?   And
    > what gaps exist in RFC7084 that could cause problems?   And in cases
    > where RFC7084 support either isn’t present, or isn’t useful because no
    > IPv6 or because ISP is delegating a /64, what things might work and
    > what things might not, if we want bidirectional reachability between
    > two separate network links in the home.

I see it (7084) in most every router at pubs in Ottawa.  
They are connected by one of the incumbents that also does TV (think sports
channels in bars). There isn't always an IPv6 uplink (30% of them have IPv6),
but there is consistently an IPv6 ULA visible.
Less often in coffee shops (WPA is on chalkboard), where it seems that they
tend to either buy from smaller ISPs (and provide their own crappy router),
or they are a multinational with hostile portals.

    > So for example, suppose we have "CE Router," which supports RFC7084,
    > including prefix delegation.  And we have "Internal Router" on that
    > network requests a delegation, and gets a prefix from the CE router.
    > Presumably that prefix is out of a larger prefix that CE Router got
    > from the ISP.  Great so far.  Let’s call the network on the southbound
    > interface of Internal Router “South Network”. Let’s call the network on
    > its northbound interface, which is also the network on CE router’s
    > southbound interface, “North Network.”

But 7084 has no requirements for DHCPv6-PD server.

    > Similarly, suppose we have a network where unfortunately PD Isn’t
    > available internally, but IPv6 is present on the northbound interface
    > of the internal node and southbound interface of the CE router.
    > Suppose further that Internal Router allocates itself a ULA prefix and
    > advertises that as reachable and on-link on its southbound interface,
    > and as reachable but not on-link on its northbound interface.   Will
    > that be blocked at layer 2 by CE Router?   I’m sort of assuming here
    > that the CE router is managing the North Network link, which is
    > probably WiFi.

That would probably work.

    > The goal here is to have bidirectional reachability between the two
    > nodes on IPv6 using either a global prefix or a ULA.  The concern is
    > that something could prevent each of these cases from working.   What
    > I’m really curious about is whether people have experience with doing
    > communications of this type using actual routers that ISPs are
    > shipping.   Is this “internal network” scenario part of acceptance
    > testing for these routers?  Is this all a big question mark?   In
    > principle this should all work, unless RA guard is hyperactive in CE
    > Router.   But what about in practice?

I have never tried it, but I'm keen to.

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Timothy Winters <[email-masked]>
Date: Fri, 4 Oct 2019 07:34:40 -0400

Hi Ted,

For the testing that we have conducted at the lab, must typical CE Router
don't support DHCPv6 PD on the LAN as Ole pointed out.   There are a couple
that have this as an additional feature.   I'm not aware of RA-Guard or
Layer-2 filtering being placed on Ethernet networks and haven't seen it but
I must admit it's not something that I have paid attention super close
attention too.   Wireless has a different set of rules, which is a longer
conversation.

Regards,
Tim

On Fri, Oct 4, 2019 at 4:18 AM Michael Richardson <[email-masked]>
wrote:

> _______________________________________________
> homenet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homenet
>

---

From: Ole Troan <[email-masked]>
Date: Fri, 4 Oct 2019 14:10:17 +0200

Mikael,

> While this is true, OpenWrt does support DHCPv6-PD within the home, out of the box. I also have a report of AVN Fritzbox supporting sub-PD without additional configuration.
> 
> In all devices I've looked at the WAN is WAN, it comes up with firewalls on, requests PD etc, and if it doesn't get it then there is no GUA IPv6 on LAN.
> 
> In my opinion the work in homenet could be leveraged into an operational document where recommendations on what parts of homenet could be easily implemented to make it work within a home (without implementing everything), thinking primarily of "firewall off" and "service discovery proxy on". If no PD was available, turn ethertype 0x86dd bridging on between LAN and WAN. I guess we would still need to do NAT44 because without HNCP there wouldn't be a route to the IPv4 network on LAN of the "sub-router".
> 
> It would however mean that a printer on the sub-router LAN could be reached over IPv6. In order for this to happen without HNCP then this sub-router would need to send RAs on its WAN announcing reachability to its LAN IPv6 prefix (either GUA+ULA if PD is available, otherwise just ULA). I have never seen RA guard or similar functions in residential equipment, so I would expect this to work.

There has been multiple proposals in that direction.
None of them succeeded, because they left too many corner cases "undefined".

Homenet has solved the problem of self-configuring networks in arbitrary topologies.

Unless someone goes to lengths describing exactly how this is supposed to work, the idea of "simplifying" the homenet solution sounds like a recipe for failure. How are you going to tell the customers that you can only plug in devices in particular topologies and in particular ways.

- single router and bridging (7084)
- arbitrary topology plug and play (homenet)
- manually configured (meaning ansible, scripts or whatever automation solution external to the routers themselves).

Cheers,
Ole

---

From: Ted Lemon <[email-masked]>
Date: Fri, 4 Oct 2019 06:06:09 -0700

> Homenet has solved the problem of self-configuring networks in arbitrary topologies.

If that were true, I wouldn’t be asking this question.  We’re still chugging along, but we don’t have something that nay router vender could even consider shipping right now.  There isn’t enough participation in homenet anymore for us to really iron out the kinks.  Certainly if I could count o homenet being present in routers in the home, we wouldn’t be having this conversation! :)

> Unless someone goes to lengths describing exactly how this is supposed to work, the idea of "simplifying" the homenet solution sounds like a recipe for failure.

I tend to agree, but wasn’t proposing that.   I was really just trying to solve the very specific problem of how I do IPv6 routing in the case I described, given the hardware that is likely to be present in the home.

> How are you going to tell the customers that you can only plug in devices in particular topologies and in particular ways.

The customer already has a router.   Suppose I want to add a router with a bunch of devices behind it, and  I know the router I’m adding will never have a second layer behind it.   Can I get it to work?   That’s the problem.

> - single router and bridging (7084)

If it doesn’t filter RA, and I control the second router, I’m okay, right?

> - arbitrary topology plug and play (homenet)

Yes, that’s homenet.   It would really help if folks who think homenet is done would try to deploy it in their home using OpenWRT and see how it goes.   Seriously.   This is not a solved problem.   We’re maybe 50-60% of the way there at this point.   I’ve been trying to make progress on this ever since HNCP was declared done, and occasionally get collaboration, and indeed we may be begining to make progress agan, but we could definitely use more participation if there are folks in 6man who want this to work and imagine that it already does.

> - manually configured (meaning ansible, scripts or whatever automation solution external to the routers themselves).

Yech.   That’s not an option.  :]

---

From: Ole Troan <[email-masked]>
Date: Fri, 4 Oct 2019 15:28:27 +0200

Ted,

> If that were true, I wouldn’t be asking this question.  We’re still chugging along, but we don’t have something that nay router vender could even consider shipping right now.  There isn’t enough pa…

Since this was sent to an IETF list I assumed you were focused on standards. (And presumably also running code). :-)

> If it doesn’t filter RA, and I control the second router, I’m okay, right?

Well, yes it isn't a router at that point, it's a bridge.

> Yes, that’s homenet.   It would really help if folks who think homenet is done would try to deploy it in their home using OpenWRT and see how it goes.   Seriously.   This is not a solved problem.…

The notion of "permissionless extensions of the network" is still somewhat unresolved. HNCP allows for that, but as we know it's not well supported/deployed. I had some hopes for multi-link subnet routing (implemented how it was originally intended, not as spanning tree in ND). But I have never gotten around to flesh that out in detail. Pascal has done various stuff in this space for lowpan, which may be something to build on.

The MLSR idea is basically a /64 shared across all links. Hosts register with routers using ARO (or DHCP). Routers advertise host routes in a routing protocol between themselves. The tricky parts are of course detecting when hosts go away, detecting movement and so on.

A side meeting in Singapore perhaps?

Cheers,
Ole

---

From: Ted Lemon <[email-masked]>
Date: Fri, 4 Oct 2019 08:59:09 -0700

> Since this was sent to an IETF list I assumed you were focused on standards. (And presumably also running code). :-)

It’s the running code that’s the problem.  What I’m hunting around for is whether there are some things we need to do that we haven’t yet done.   I wish I could say “just turn on Homenet and everything will be fine,” but we aren’t in a position to say that yet.   If people are feeling satisfied that homenet is “done,” we (the IETF, and the Internet community in general) have a problem.

We’d been tempted to punt on this because there are perfectly good commercial solutions that solve this problem with L2 bridging, but those don’t work when you need to route between WiFi and e.g. 6lowpan networks.

> Well, yes it isn't a router at that point, it's a bridge.

The reason I mentioned this particular problem is that I’ve heard reports of 7084 routers implementing RA guard, which sort of makes sense, but it totally breaks this use case.

> The MLSR idea is basically a /64 shared across all links. Hosts register with routers using ARO (or DHCP). Routers advertise host routes in a routing protocol between themselves. The tricky parts a…

I’ve seen this.   It’s how the 6lowpan folks seem to have defined their “routing."   But scalability is horrible, and when I’ve looked at doing an implementation, it’s obviously ridiculously harder than just publishing an RA.   And it’s particularly hard if there’s no IPv6 on North Network.    So if there’s a way to get IPv6 to work in this case, that seems like a better option.

> A side meeting in Singapore perhaps?

I’m not going to be in Singapore—I just can’t justify the carbon footprint.   I do think we ought to have a deeper discussion about this, though.   Maybe a “virtual interim”?   If there’s interest, I’d be happy to organize this.   I’d also be happy to attend a side meeting in Singapore remotely, but our experience with that thus far has been pessimal, and I don’t think a fix is planned for Singapore (unless I just haven’t heard about it).

---

From: Ole Troan <[email-masked]>
Date: Sun, 6 Oct 2019 19:58:27 +0200

> We’d been tempted to punt on this because there are perfectly good commercial solutions that solve this problem with L2 bridging, but those don’t work when you need to route between WiFi and e.g. 6…

Are you saying there might be gaps in HNCP? Or things we could do to make it more deployable?
If it's just a matter of running code missing, I'm not sure defining anything else new in the IETF would help that problem.

I know why I haven't implemented HNCP on software I work on... and I also know that there aren't any very realistic alternatives either.

> The reason I mentioned this particular problem is that I’ve heard reports of 7084 routers implementing RA guard, which sort of makes sense, but it totally breaks this use case.

RA guard isn't applicable in a RFC7084 context. RFC7078 talks about routing and routers. I.e. what happens at the network layer.
If you are talking about what happens at the often integrated bridge CE devices have, then sure, they could implement RA Guard.
But having your additional router sending RAs across the homenet might not be a particularly good idea anyway.
Sounds like you need to set it up as a NAT.

> I’ve seen this.   It’s how the 6lowpan folks seem to have defined their “routing."   But scalability is horrible, and when I’ve looked at doing an implementation, it’s obviously ridiculously harder…

I wasn't thinking of doing it exactly like the 6lowpan does it.
Regardless I don't see why scaling should be problematic, are you planning to have millions of rapidly moving hosts on your shared /64 network?

Cheers,
Ole

---

From: Ted Lemon <[email-masked]>
Date: Sun, 6 Oct 2019 14:41:06 -0700

> If it's just a matter of running code missing, I'm not sure defining anything else new in the IETF would help that problem.

There are definitely missing features from the protocol that I’d like to add.   But I think the fact that the protocol isn’t deployed on a _single_ commercially available router, and is not really usable on OpenWRT by a non-expert, is an indication that there is substantial remaining work to do.   Operations work is not out of scope for IETF; maybe I should have asked this on v6ops.   We have historically said “not our problem,” but I don’t agree that that’s the right answer.   If HNCP had really convincingly solved the problem, we would not be seeing what we are seeing.

> I know why I haven't implemented HNCP on software I work on... and I also know that there aren't any very realistic alternatives either.

Can you say why that is?

> RA guard isn't applicable in a RFC7084 context. RFC7078 talks about routing and routers. I.e. what happens at the network layer.

You mean at the “internet layer,” I assume?

> But having your additional router sending RAs across the homenet might not be a particularly good idea anyway.

Why not?   What would be a better idea?   I don’t mean to say that using RAs in this way is ideal, but what’s the alternative that doesn’t involve the vast complexity of per-host routing?

> Sounds like you need to set it up as a NAT.

I really hope you are just making a funny joke here.   But it’s not very funny.   What I want is something that’s operationally simple, not something with lots of moving parts that have to be kept track of.   NATs in particular suck for any UDP-based protocol.

> Regardless I don't see why scaling should be problematic, are you planning to have millions of rapidly moving hosts on your shared /64 network?

If you have N devices on a single link on the other side of the router, then when using either RA or a routing protocol, the amount of information you need to propagate to get things working is very small: just a prefix and a router.   If you can’t do that, then the amount of information you need to propagate is at a minimum N units, and possibly K*N, for some not insignificant factor K.

To be clear, the reason I’m concerned about this is that I’ve looked at implementing it on OpenWRT, and it’s at least roughly doubling the complexity of the work required, even if you can depend on using IPv6.   If you have to use IPv4 on one side, then it’s even more complexity.   And it’s utterly stupid complexity—it adds no value over subnetting.   Why add that to the network?

This is why I’m asking people if they have knowledge of what is actually deployed.   I think this is the right place to ask, but if you disagree, I’m open to suggestions.

---

From: Mark Smith <[email-masked]>
Date: Mon, 7 Oct 2019 09:34:14 +1100

Perhaps ANIMA is an alternative? It has seemed to me that home networks
might be just a more specific case of autonomic networks.

For example, they've been defining a Generic Autonomic Signalling Protocol
(GRASP).

https://tools.ietf.org/wg/anima/

Brian Carpenter has been working on an implementation.

https://github.com/becarpenter/graspy

On Mon, 7 Oct 2019, 08:41 Ted Lemon, <[email-masked]> wrote:

> If it's just a matter of running code missing, I'm not sure defining
> anything else new in the IETF would help that problem.
>
>
> There are definitely missing features from the protocol that I’d like to
> add.   But I think the fact that the protocol isn’t deployed on a _single_
> commercially available router, and is not really usable on OpenWRT by a
> non-expert, is an indication that there is substantial remaining work to
> do.   Operations work is not out of scope for IETF; maybe I should have
> asked this on v6ops.   We have historically said “not our problem,” but I
> don’t agree that that’s the right answer.   If HNCP had really convincingly
> solved the problem, we would not be seeing what we are seeing.
>
> I know why I haven't implemented HNCP on software I work on... and I also
> know that there aren't any very realistic alternatives either.
>
>
> Can you say why that is?
>
> RA guard isn't applicable in a RFC7084 context. RFC7078 talks about
> routing and routers. I.e. what happens at the network layer.
>
>
> If you are talking about what happens at the often integrated bridge CE
> devices have, then sure, they could implement RA Guard.
> Why not?   What would be a better idea?   I don’t mean to say that using
> RAs in this way is ideal, but what’s the alternative that doesn’t involve
> the vast complexity of per-host routing?
>
> I really hope you are just making a funny joke here.   But it’s not very
> funny.   What I want is something that’s operationally simple, not
> something with lots of moving parts that have to be kept track of.   NATs
> in particular suck for any UDP-based protocol.
>
> Regardless I don't see why scaling should be problematic, are you planning
> to have millions of rapidly moving hosts on your shared /64 network?
>
>
> If you have N devices on a single link on the other side of the router,
> then when using either RA or a routing protocol, the amount of information
> you need to propagate to get things working is very small: just a prefix
> and a router.   If you can’t do that, then the amount of information you
> need to propagate is at a minimum N units, and possibly K*N, for some not
> insignificant factor K.
>
> To be clear, the reason I’m concerned about this is that I’ve looked at
> implementing it on OpenWRT, and it’s at least roughly doubling the
> complexity of the work required, even if you can depend on using IPv6.   If
> you have to use IPv4 on one side, then it’s even more complexity.   And
> it’s utterly stupid complexity—it adds no value over subnetting.   Why add
> that to the network?
>
> This is why I’m asking people if they have knowledge of what is actually
> deployed.   I think this is the right place to ask, but if you disagree,
> I’m open to suggestions.
>
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 7 Oct 2019 11:46:55 +1300

On 07-Oct-19 11:34, Mark Smith wrote:
> Perhaps ANIMA is an alternative? It has seemed to me that home networks might be just a more specific case of autonomic networks.

...for professionally managed networks. So there would be new work to do, if we wanted to expand the scope.

> 
> For example, they've been defining a Generic Autonomic Signalling Protocol (GRASP).
> 

Which is plastered with warnings that it's not fit for real-life usage, as a prototype written in Python.

   Brian

> 
> On Mon, 7 Oct 2019, 08:41 Ted Lemon, <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     On Oct 6, 2019, at 10:58 AM, Ole Troan <[email-masked] <mailto:[email-masked]>> wrote:
>     [email-masked] <mailto:[email-masked]>

---

From: Gyan Mishra <[email-masked]>
Date: Mon, 7 Oct 2019 00:05:50 -0400

In line 

Thanks 

Sent from my iPhone

> On Oct 6, 2019, at 6:46 PM, Brian E Carpenter <[email-masked]> wrote:
> 
     [Gyan] Good 6man question related to polling of what as far has home net HNCP RFC 7788 has been deployed.

Reading through both RFC 7788 and 7084 there does seem to be gaps and clarity as far as auto provisioning of IPv6.

I think both documents really need to be updated to be helpful and also have concrete solution as to how the IPv6 auto provisioning is going to work.

For IPv4 in general across the board all broadband routers openWrt or otherwise pnp setup have a dhcp wan IP and LAN side is RFC 1918 192.xx and nat overload is done from inside to outside.  Pretty basic and it works well.

For IPv6 we have the option of hiding the internal space doing a ULA on the lan side but only caveat that would require NAT64 DNS64 as well added complexity but that would work.

The other option is PD on the wan side get a GUA /56 from the SP allocated dynamically to the home broadband router and then a /64 is doled our to lan interface via PD and all hosts in the home network sit on the GUA /64 and get a slaac address.

The later  I thin is the easiest as no nat requirement.

You could build in some python or ansible automation scripts to build out the pnp config to make it cookie cutter for any deployment so the routers would ship with the script and get auto provisioned.  

CIsco has NSO product for programmability for enterprises and service providers for auto provisioning day 1 configurations and deployment which would be perfect for this use case for broadband home net deployments.

If everyone agrees that those RFCs need to be updated for home net per what I mentioned I could take the lead on that for 6man.

Gyan

---

From: Ole Troan <[email-masked]>
Date: Mon, 7 Oct 2019 08:36:46 +0200

Hi Ted,

> There are definitely missing features from the protocol that I’d like to add.   But I think the fact that the protocol isn’t deployed on a _single_ commercially available router, and is not really…

I believe HNCP has solved the technical problem it set out to do. Allow for an automatically configured, arbitrary topology network with multiple exits.
The deployment challenge of that is that every router must support HNCP and must support SADR.

>> I know why I haven't implemented HNCP on software I work on... and I also know that there aren't any very realistic alternatives either.

Quite a few reasons, some which might be not relevant to your case of course.
 - the pendulum between distributed algorithms and centralized controllers is for the problems I'm working on largely towards the centralized side
 - it's quite a lot of work. it requires a new routing protocol, it requires a changed forwarding paradigm, and it requires integration with a lot of features
 - it does not support the case "permissionless extensions of the network".

> You mean at the “internet layer,” I assume?

No, I mean the network layer. RA guard operates as a layer violating feature at the data-link layer.

> Why not?   What would be a better idea?   I don’t mean to say that using RAs in this way is ideal, but what’s the alternative that doesn’t involve the vast complexity of per-host routing?

I don't understand how it would work. Add another router with it's own link. How would you get addresses for that link? Make them up from ULA? And then advertise that in an RA upstream with the MSR option?
That would put a lot of responsibility on the hosts of getting things "right".
And what if you added two of your routers, or connected them differently?

Per-host routing is in itself trivial and likely scales orders of magnitude further than you need. But there are problems with MLSR that are unsolved / not solved to my liking yet there.

> I really hope you are just making a funny joke here.   But it’s not very funny.   What I want is something that’s operationally simple, not something with lots of moving parts that have to be kept…

for "permissionless extensions of the network" there isn't much else than NAT.
Your other likely option is an ND proxy. If you are very sure that nothing else can be added to the network (we don't want to build a spanning tree protocol out of ND).

> To be clear, the reason I’m concerned about this is that I’ve looked at implementing it on OpenWRT, and it’s at least roughly doubling the complexity of the work required, even if you can depend on…

Because you don't like the mechanisms for automated subnet assignment? ;-)
And no, I'm not suggesting we should do MLSRv2 as a competitor for HNCP. MLSRv2 would also require an update to every router, and a network operator allowing extensions to the network.

> This is why I’m asking people if they have knowledge of what is actually deployed.   I think this is the right place to ask, but if you disagree, I’m open to suggestions.

I do not disagree. I think having that feedback loop from implementation/deployment/operational experience is important.

Best regards,
Ole

---

From: Alexandre Petrescu <[email-masked]>
Date: Mon, 7 Oct 2019 11:31:54 +0200

> South Network    |-----------+-------------------+--------------------------------|

If somebody makes a good solution and easily deployed for the topology
in the above figure, then I am willing to consider it for vehicular
networks as well.  In them, the CE Router is a Mobile Router in the car
and the Internal Router is one of many other routers in the car.  The
ISP is a cellular network and, potentially, an 802.11-OCB network.

At this time, the furthest we can do is a CE Router that runs DHCPv6-PD
and then forms /64s for the Internal Router, and then some 64share (
draft-shytyi-v6ops-danir-04 with URL to implementation).  But it has
inconvenients: it introduces a new class (/56), because it is hardcoded
as much as the exiting class /64 is.

(we tried as much as we could to remove the IPv6 class only to come up
with a new one :-)

Alex

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 7 Oct 2019 12:07:25 +0200 (CEST)

> The deployment challenge of that is that every router must support HNCP 

Yes, there is indeed a problem here with incremental deployment.

That's why I think there might be upside in "homenet lite" which drops the 
arbitrary topology requirement and keeps the "routed home" requirement, 
but also brings with it the service discovery part. Perhaps it's an easier 
chunk of code to deploy if vendors do not need to implement full homenet 
but instead implement RFC7084 plus a few other things?

High level would be to use DHCPv6-PD, turning sub-router WAN firewall off 
and enabling service discovery proxies, as I outlined in previous email.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Oct 2019 10:37:58 +0000

> Your other likely option is an ND proxy. If you are very sure that nothing else can be added to the network (we don't want to build a spanning tree protocol out of ND).

Agreed that ND proxy is not an alternative for routing. It is a one hop extension around a transit link and nothing more. For deeper ”mesh” extensions there are millions of nodes deployed out there that use RPL quite successfully.

The design allows to combine the 2 with a high speed backbone (Ethernet) and RPL meshes ganging off ND proxies. 

But inside a subnet RPL is used to route to hosts so there is no subnet assignment just SLAAC or DHCP. Still those networks scale to ~ten thousand. Good enough for home for a long time. 

>>> I wasn't thinking of doing it exactly like the 6lowpan does it.

Too bad then... I still fail to see why the model cannot be generalized to more powerful nodes.

Anyway a transit and one level of proxy already takes us a long way.

All the best,

Pascal

---

From: Michael Richardson <[email-masked]>
Date: Mon, 07 Oct 2019 11:49:15 +0200

> For example, they've been defining a Generic Autonomic Signalling

GRASP has some overlap with HNCP, but HNCP isn't really the issue.
Homenet has a routing protocol (BABEL).  It was going to be OSPFv3, and we'd
add the information flooding that HNCP wound up doing into OSPFv3, but that
"didn't work" and the WG (mysteriously) abandonned that idea.

ANIMA's GRASP is also not a routing protocol: ANIMA builds an overlap control
layer with IP over IPsec over IPv6LL tunnels, and runs RPL (RFC6550) as
the routing protocol for the control plane.  While the resulting ACP shares
some properties with the desired HOMENET routing goals, it's not the same
thing at all.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2019 06:18:22 -0700

> The deployment challenge of that is that every router must support HNCP and must support SADR.

The question I have about this is: if nobody is shipping this, how do we know that the problem is solved?   We have zero operational experience for the intended use case.   HNCP experts setting it up for themselves or people they know is not the target use case.  At present everybody who’se tried seriously to use this stuff has looked at the source code.

> - the pendulum between distributed algorithms and centralized controllers is for the problems I'm working on largely towards the centralized side

This makes sense.

> - it's quite a lot of work. it requires a new routing protocol, it requires a changed forwarding paradigm, and it requires integration with a lot of features

Yes.   This is a big problem.

> - it does not support the case "permissionless extensions of the network".

That seems to contradict your first point—if you want permissionless extension, isn’t that by definition not centralized?

> No, I mean the network layer. RA guard operates as a layer violating feature at the data-link layer.

Hm, okay, fair point.   But I think that we need to say how RA guard works on home networks if we don’t want to have to guess.   Possibly this is not 6man work, but it feels relevant to me.

> Per-host routing is in itself trivial and likely scales orders of magnitude further than you need. But there are problems with MLSR that are unsolved / not solved to my liking yet there.

If you’re adding this to an RFC 7084 network, there are things that you can do that can work as long as RA guard isn’t present, and that achieve the limited goal of being able to have devices on one network communicate with devices on the other.   Indeed, a ULA would work well for this use case.   Allocate a single ULA, and then allocate prefixes out of it for the various networks.

I think that if you want to add more than one router, HNCP+Babel is a sensible way to do it.   But this is only required for leaf-to-leaf communication.   A device on the home network will have an RA advertising transit to each leaf prefix, and will either have IPv6 connectivity from RFC 7084, or from another prefix advertised on the link.

The problem with per-host routing is that it means I have to implement and deploy per-host routing, which I never want.   And as you add leaf networks, the number of routes being propagated starts to snowball.   I believe that this can be made to work for use cases I care about, but it’s in no way a “good” solution.  The main problem with it is that it means I have to write code to do it.   I’d rather not write that code if I don’t have to.

> Your other likely option is an ND proxy. If you are very sure that nothing else can be added to the network (we don't want to build a spanning tree protocol out of ND).

Yeah.  That is the exact situation I would most like to avoid, so I don’t want to be walking in that direction.

> Because you don't like the mechanisms for automated subnet assignment? ;-)

On the contrary, if HNCP were widespread, that’s what I’d be using.   HNCP would be my first choice for this.   I’d still like that to be the long-term solution.   But what can I build into a leaf router _today_ to make this work when the edge router doesn’t run HNCP?  :)

> And no, I'm not suggesting we should do MLSRv2 as a competitor for HNCP. MLSRv2 would also require an update to every router, and a network operator allowing extensions to the network.

BGP?   ;)

> I do not disagree. I think having that feedback loop from implementation/deployment/operational experience is important.

Okay.   Thanks for the deep engagement on this.   I’m not sure we agree on what to do, but it’s very useful to hear what you are thinking about this.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2019 06:27:13 -0700

> Too bad then... I still fail to see why the model cannot be generalized to more powerful nodes.

Because it is maximally complex?   :]

You say that RPL has scaled to millions of nodes.   Where is this deployed in production?   What are the leaf nodes doing?   With what are they communicating?

---

From: Michael Richardson <[email-masked]>
Date: Mon, 07 Oct 2019 15:47:40 +0200

> arbitrary topology requirement and keeps the "routed home" requirement, but
    > also brings with it the service discovery part. Perhaps it's an easier chunk
    > of code to deploy if vendors do not need to implement full homenet but
    > instead implement RFC7084 plus a few other things?

So, 7084++ or HomeNet-Lite.  

    > High level would be to use DHCPv6-PD, turning sub-router WAN firewall off and
    > enabling service discovery proxies, as I outlined in previous email.

I agree. 7084bis should suggest downstream DHCPv6-PD be supported, and we
need to explicitely signal there when HNCP is available so that we don't wind
up double allocating things, etc.

---

From: RayH <[email-masked]>
Date: Mon, 07 Oct 2019 16:15:09 +0200

On 7 Oct 2019 15:47, Michael Richardson <[email-masked]> wrote:

Mikael Abrahamsson <[email-masked]> wrote:

>> The deployment challenge of that is that every router must support HNCP and

>> must support SADR.

Ughh.

I wasn't enamoured with the idea of 7084 being part of Homenet at all, but concensus was reached in the WG to spin out that document as the last-ever do-it-as-a-quick-win "single router" solution, after pressure from operators.

IMHO there's no future in updating 7084, because the goal of that document was purely a single router solution.

Anything else requires either full Homenet (yes, sorry, throw away or reflash your old router and embrace the required complexity),

Or it inevitably fails by either prolonging the lifetime of NAT, or being so broken in the limited topologies that it supports as to be nearly useless.

This goes to the decisions and discussions we had when writing the Homenet Architecture rfc7368.

My preferred path would be to look at why Homenet hasn't been rolled out.

If it's because manufacturers aren't updating boxes at all, or even ipv6 at all as per my local internet non-service provider, another standard ain't going to solve that.

So is there concensus on what's broken? And what needs fixing?

Regards,

--

homenet mailing list

[email-masked]

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 7 Oct 2019 14:36:20 +0000

From: RayH
> I wasn't enamoured with the idea of 7084 being part of Homenet at all, but concensus was reached
> in the WG to spin out that document as the last-ever do-it-as-a-quick-win "single router" solution,
> after pressure from operators.

RFC7084 was never a homenet doc. It was a v6ops doc, only. It was not "spun out" from homenet. It was intended to provide consumer router vendors with the basic requirements they needed to connect to any wireline IPv6 network (compliant with either CableLabs or BBF IPv6 network architectures). And to allow hosts attached directly to the LAN to use that IPv6 connection. It served its purpose well.

> IMHO there's no future in updating 7084, because the goal of that document was purely a single router solution.

I agree with this, but also because it was only for a single "customer edge" router solution. The customer edge part of this was the goal. That's why it was a v6ops doc.

Barbara

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2019 09:37:14 -0500

> So is there concensus on what's broken? And what needs fixing?

I think it’s a lot simpler than that: they don’t have to do it, so they don’t.   There’s no upside for them in adding complexity to the network, and that’s what this looks like.   In order for homenet to see widespread adoption, there has to be a problem it solves that lots of home users have.

TBH, one of the reasons that I am not in favor of ND proxy is precisely that it kicks this can even father down the road.   IoT network transit and similar applications are a clear use case for Homenet; building a solution that’s going in entirely the wrong evolutionary direction seems like an unfortunate plan.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 07 Oct 2019 16:40:54 +0200

> You say that RPL has scaled to millions of nodes.   Where is this
    > deployed in production?   What are the leaf nodes doing?   With what
    > are they communicating?

Diagram to help:
          A               B
          |               |
     +----+-------+-------+----+ <-network 1
     |            |            |
     C            D            E
     |            |            |
   +-+-+(2)     +-+-+(3)     +-+-+(4)
   |   |        |   |        |   |
   F   G        H   I        J   K

1) the deployments I'm aware of involve many thousands (4-zeros) of
   electricity meters (Automatic Metering Infrastructure) (sometimes
   including gas or water meters as leaves). 
   I'm *unaware* of any LLNs that have 10^6 nodes in a single LLN.
   Unfortunately Pascal can't tell you who, because of NDAs he has with his
   customers.  I've been shown evidence with stuff blacked out/redacted.

2) the leaf nodes are sending readings up on a MP2P topology.  There isn't
   much cross-traffic.  So in a homenet context, this is equivalent to there
   being 30 routers in the home, and no PCs/laptops/etc, rather than 3
   routers with 10 desktops each.
   In the homenet situation, cross-traffic would be traffic that somehow goes
   from network 2(F) to network 4(K) without going across network 1.
   In an LLN there might be other paths across the radio links that would
   permit F<->K traffic without using network 1, and P2PRPL is a protocol
   that could find it.

3) the leaf nodes in AMI talk to HQ only.
   There are lighting LLNs that are all cross-traffic though.

Some other things to note:
  a) RPL is not particularly self-configuring, and probably has more
     parameters to tweak than other protocols, but getting it exactly right
     only really matters when you have to worry about bandwidth and energy.

  b) to date, RPL hasn't had a lot of "Internet Standard"-style interop
     validation.  All of the deployments I'm aware of are single-vendor, or
     involve one or two vendors working closely together.  This is sad.

  c) While RPL has a single root to which all traffic flows, in the diagram,
     both A and B would announce their own DODAG in a DODOG Information
     Object (DIO), and the DIO would include Prefix Information Object, which
     is the equivalent of an RA.
     Routes in the network are usually /128 routes, but if one used DHCPv6-PD
     or HNCP to get a /64, one could announce that equally well.
     Announcing exclusively /128s (with L=0, so offlink) does do nice things
     for wifi and mobility.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Oct 2019 14:46:13 +0000

Hello Ted :

Yes the millions I'm talking about are deployed in production. If that's so complicated then I guess we have has 2 small groups of magicians at Cisco alone ; ) I'm aware of competition as well. Plus my private pre-RPL implementation that took less than a month to code and test.

As the moment all deployed nodes are equally capable, meaning that the leaves are capable to be routers. They are all powered and always son.

The second generation that we are preparing will have low-power leaves that
- are duty cycled
- do not need to understand any of RFC 6550.

The WIP draft is https://tools.ietf.org/html/draft-ietf-roll-unaware-leaves

We've made it so that the 6LoWPAN ND interface (RFC 8505) is enough to register to abstract connectivity services  that can be ND proxy or routing. The unaware leaves spec redistributes addresses learned via ND into RPL; we have a similar operation in DC routing to inject virtual machines / containers addresses in RIFT. It should work with other IGPs as well, and even BGP/eVPN, why not?

All the best

Pascal

From: Ted Lemon <[email-masked]>
Sent: lundi 7 octobre 2019 15:27
To: Pascal Thubert (pthubert) <[email-masked]>
Cc: Ole Troan <[email-masked]>; Markus Stenberg <[email-masked]>; 6MAN <[email-masked]>
Subject: Re: [homenet] Support for RFC 7084 on shipping devices...

On Oct 7, 2019, at 3:37 AM, Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>> wrote:
Too bad then... I still fail to see why the model cannot be generalized to more powerful nodes.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Oct 2019 14:54:12 +0000

Hello Ted :

The reason why we opted for ND proxy is not the lack of a homenet solution. It is rather the fact that the wireless node keeps moving from a mesh to the next and if they are different subnets then it needs to renumber, which we did not know how to do efficiently – think dhcp or autoconf, 1x, plus reregistration to the application, a number of round trips on constrained links, some of which real deep. Bottom line is that we wanted both the benefits of routing to control/avoid broadcast and those of a single subnet to control/avoid renumbering and we built accordingly.

Cheers,

Pascal

From: ipv6 <[email-masked]> On Behalf Of Ted Lemon
Sent: lundi 7 octobre 2019 16:37
To: RayH <[email-masked]>
Cc: Michael Richardson <[email-masked]>; Markus Stenberg <[email-masked]>; 6MAN <[email-masked]>
Subject: Re: [homenet] Support for RFC 7084 on shipping devices...

On Oct 7, 2019, at 9:15 AM, RayH <[email-masked]<mailto:[email-masked]>> wrote:
My preferred path would be to look at why Homenet hasn't been rolled out.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 7 Oct 2019 14:58:25 +0000

Hello Michael

The order of order of 10^6 (hopefully 10^7) are the total deployed not a single mesh. This makes RPL quite a well-deployed protocol.
As you indicate, a single mesh can approach 10^4. A depth can be al lot more than the 10 hops that we imagined initially. Yet it keeps working.

Cheers,

Pascal

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Michael Richardson
Sent: lundi 7 octobre 2019 16:41
To: 6MAN <[email-masked]>; [email-masked]
Subject: Re: [homenet] Support for RFC 7084 on shipping devices...

Diagram to help:
          A               B
          |               |

---

From: RayH <[email-masked]>
Date: Mon, 07 Oct 2019 17:00:19 +0200

On 7 Oct 2019 16:37, Ted Lemon <[email-masked]> wrote:

Why does an ISP have to add complexity to their network in order to support Homenet?

Because they want to own and operate a 7084 cpe router on the customer's premises?

If they draw the service delivery point at an Ethernet isolation LAN that provides IA PD, RA, and pure unfiltered native IPv6 transport, it's arguably simpler for them than managing a full cpe including local DHCP, DNS resolution, NAT, wireless.....

My latest business class connection only supports ds-lite (ipv4 only, and no plans for IPv6) and not even any DHCP. So there are providers that are looking to redraw the boundary.

I agree wholeheartedly on that.

Regards,

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2019 10:03:59 -0500

> Why does an ISP have to add complexity to their network in order to support Homenet?

If the ISP is on the hook for support, then “their network” includes your home network.

---

From: RayH <[email-masked]>
Date: Mon, 07 Oct 2019 17:16:00 +0200

On 7 Oct 2019 17:03, Ted Lemon <[email-masked]> wrote:

If that's their business model, we've got nothing to say.

IMHO that quite simply isn't Homenet's scope....  and they can deploy whatever proprietary management protocols they like in that situation.

Hopefully, they'll open one port to IA-PD RA etc. for plug and play. But no-one can force them.

Regards,

---

From: Ted Lemon <[email-masked]>
Date: Mon, 7 Oct 2019 10:31:34 -0500

> If the ISP is on the hook for support, then “their network” includes your home network.

What I mean is that the ISP gets phone calls from customers whether it’s their fault the network is broken or not.  So they aren’t going out of their way to make the home network more complicated.  Although we have seen ISP participation in Homenet, I do not expect them to be the drivers for this functionality.   I’d love to be proven wrong, but I don’t think it’s realistic to assume otherwise.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 8 Oct 2019 08:53:55 +1300

> some properties with the desired HOMENET routing goals, it's not the same

No, it isn't. I didn't have time to reply fully yesterday.

First: when ANIMA was originally chartered, we took care to avoid overlap
and interference with HOMENET. In particular, ANIMA assumes there is some
degree of professional management of the network, which autonomic functions
are expected to enhance and simplify. Also they will probably do things
(like policy-driven decision taking) that are not really expected in
a home network.

Second: GRASP is not competing with HNCP. That's analyzed in
https://tools.ietf.org/html/draft-ietf-anima-grasp-15#appendix-F
(GRASP is an approved draft, stuck waiting for missing references).

Third: If there was a desire to use the ANIMA work for homenets
or unmanaged small office networks, there would be work to do.
As Michael says, we'd need to adapt the autonomic control plane
accordingly. And then some autonomic service agents would need to
be written to fulfill, guess what, the HOMENET requirements
implied by RFC 7368. For example, an agent to perform address
assignment and an agent to configure the dataplane routing protocol.

Regards
   Brian

---

From: Alexandre Petrescu <[email-masked]>
Date: Thu, 10 Oct 2019 15:16:30 +0200

Le 07/10/2019 à 16:46, Pascal Thubert (pthubert) a écrit :
> Hello Ted :
> 
> Yes the millions I’m talking about are deployed in production.

Is it in a limited-domain?

CAn I ping one of them?

Alex

  If that’s
> so complicated then I guess we have has 2 small groups of magicians at 
> Cisco alone ; ) I’m aware of competition as well. Plus my private 
> pre-RPL implementation that took less than a month to code and test.
> 
> As the moment all deployed nodes are equally capable, meaning that the 
> leaves are capable to be routers. They are all powered and always son.
> 
> We’ve made it so that the 6LoWPAN ND interface (RFC 8505) is enough to 
> register to abstract connectivity services  that can be ND proxy or 
> routing. The unaware leaves spec redistributes addresses learned via ND 
> into RPL; we have a similar operation in DC routing to inject virtual 
> machines / containers addresses in RIFT. It should work with other IGPs 
> as well, and even BGP/eVPN, why not?
> 
> All the best
> 
> Pascal
> 
> *From:*Ted Lemon <[email-masked]>
> *Sent:* lundi 7 octobre 2019 15:27
> *To:* Pascal Thubert (pthubert) <[email-masked]>
> *Cc:* Ole Troan <[email-masked]>; Markus Stenberg 
> <[email-masked]>; 6MAN <[email-masked]>
> *Subject:* Re: [homenet] Support for RFC 7084 on shipping devices...
> 
> On Oct 7, 2019, at 3:37 AM, Pascal Thubert (pthubert) 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
> deployed in production?   What are the leaf nodes doing?   With what are 
> they communicating?
> 
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 10 Oct 2019 09:39:26 -0400

> As you indicate, a single mesh can approach 10^4. A depth can be al lot more than the 10 hops that we imagined initially. Yet it keeps working.

How frequently do things change in the mesh?   Does this require any management to set up?

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 10 Oct 2019 15:58:23 +0000

Hello Ted

I do not have stats that I can expose, sorry for that, but switching parent is relatively frequent. Switching DODAG happens more often that we'd like. Either way there is nothing manual and the management is not involved. RPL handles the routing but if the other DODAG is a different subnet the node that moved needs to renumber and that is multiple round trips.

RPL is a form of L3 spanning tree when operated inside a subnet. There are advantages in forwarding at L3 within the subnet, e.g,  TTL to avoid meltdown and NECM...

Cheers,

Pascal

From: Ted Lemon <[email-masked]>
Sent: jeudi 10 octobre 2019 15:39
To: Pascal Thubert (pthubert) <[email-masked]>
Cc: Michael Richardson <[email-masked]>; 6MAN <[email-masked]>; [email-masked]
Subject: Re: [homenet] Support for RFC 7084 on shipping devices...

On Oct 7, 2019, at 10:58 AM, Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>> wrote:
As you indicate, a single mesh can approach 10^4. A depth can be al lot more than the 10 hops that we imagined initially. Yet it keeps working.