Subject: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

From: Erik Nordmark <[email-masked]>
Date: Wed, 19 Feb 2014 23:30:27 -0800

Thanks for putting this draft together. Some questions and comments below.

In section 3 you have:
* One solicited RA per host joining the network (if solicited RAs are 
sent using multicast)
That isn't entirely true. Because of the rate limiting of multicast RAs 
if a lot of hosts join at the same time, then there will be less.

In section 4.2 we can suggest an implementation that does unicast:
If we believe that the flash crowd case is still important (when we 
wrote ND the concern was around a building power failure and recovery - 
today it might be a stadium setting), then one could implement a mix as 
follows:
  - When RS received start timer.
  - If N additional RSs arrives before timer fires, then multicast one 
RA. Otherwise when timer fires unicast RAs to the received RSs.
For N=1 this is trivial.

In section 4.3 there are two suggestions: MLD snooping and L2 unicast or 
L3 multicast packets. I think there are some operational concerns around 
MLD snooping - I hope others can fill in some information in that space 
since I don't know the issues.
(Editorially it would be clearer if those two are separate sub-sections.)

The multicast-over-unicast refers to SAVI as a way to collect the state, 
but it doesn't specify where you see that state used. Presumably for DAD 
(and NS in general unless you also do section 4.7)? But SAVI doesn't 
claim to have all addresses since it is concerned with conflicts. Thus a 
host can exist and be silent - but DAD would still expect to reach it. 
Or a host could have moved to a different port and SAVI having stale 
state - yet DAD should work. My point is that the details of how the 
state is 1) maintained and 2) used to forward packets are key to 
analyzing how the neighbor discovery functionality and robustness would 
be affected by this idea.

I section 4.4 the document refers to proxy but without specifying which 
of the different proxy approaches you have in mind.
There is the proxy ND RFC, and there is the DAD proxy internet-draft in 
6man. Are you referring to one of those, or some slightly different form 
of proxy? (For example, ND proxy would respond with the LLA of the 
router/AP, but that might result in host movement looking like a 
duplicate address - again depends on the details.)

I don't have any issue with removing the somewhat arbitrary 9000 second 
max AdvDefaultLifetime in section 5.1. However, the tradeoff for what 
default lifetime to use in section 4.5 needs to take into account one 
additional factor.
The default lifetime serves to garbage collect entries from the default 
router list should a router silently disappear. Thus for links that do 
not have a fixed (set of) link-local address(es) for the router(s), 
having a high default lifetime means that after a failure the hosts 
would have one entry in the default router list which us unreachable - 
until that high lifetime expires. I don't know if there has been a study 
on the performance impact of that would have on the hosts e.g., how 
often they would re-probe the default router.

Section 4.6 has the same concern. But 4.5 and 4.6  makes lots of sense 
e.g., in a VRRP deployment where the link-local address of the virtual 
default router would always be the same. Ditto for networks with a 
single point of failure single router at a fixed address (e.g., if the 
router is always at fe80::1 or some other fixed address.) Thus I think 
we should recommend 4.5 and 4.6 that within that applicability. Added 
benefit is that the routers control it, hence the operator of the 
network can set the values higher for VRRP or single router cases.

Section 4.7 talks about limiting the table space on the hosts. But 
clearing the on-link bit in the advertised prefixes also has the benefit 
of removing any multicast NS sent from hosts for non-link-local 
destinations. And per RFC 4861 the table will be filled in by the router 
sending redirects. Thus the router can be used to control which 
addresses get in the tables on the hosts by choose whether and when to 
send the redirects (redirects with target==destination).

In think it would be good to separate out the second half of section 4.7 
(blocking link-locals) into a separate section, since it is quite 
different than clearing the on-link bit. That idea has significant 
implications since it changes the IP subnet model (RFC 4903 talks about 
this.) I not saying that we shouldn't consider this, but I do think it 
would fall in a very different category than the other ideas in this 
draft. Might even be best to have a separate draft on this radical idea 
so it can be explored fully.

4.8 seems to conflate the address assignment with DAD. Just because we 
might want to centralize the DAD checks doesn't imply that we want to 
remove the ability for the host to pick its own privacy enhanced 
interface-IDs to form its addresses.
 From a deployment perspective DHCPv6 is available for address 
assignment, but don't think we want to require that for WiFi or other 
links which have packet loss. (Packet loss occurs on wired networks as 
well, but the drop distribution is different - might happen during 
spanning tree reconvergence etc.)
Note that DHCPv6 (RFC 3315) has a SHOULD for doing DAD on the addresses 
received from the DHCP server - needed since the server could be confused.

In don't understand 4.9. Should I read it as a host shutting things down 
if it goes to sleep even for a short time, and then waking up and 
multicasting a few RSs to get RAs, then multicasting a DAD probe for 
each assigned IPv6 address? If all the hosts did that then I think there 
would be more multicasts.
Note that even if the host doesn't revert to multicasting RSs, it still 
needs to be concerned about a duplicate address having arrived on the 
link while it was asleep (and not responding to any DAD probes.)

The suggestion in section 5.2 is needs a lot more work. I tried to work 
out some of these issues in section 8.9 in 
draft-chakrabarti-nordmark-6man-efficient-nd-05.
But even if we figure out how to do that without causing lots of 
unneeded RS load on the routers, in the case of your draft wouldn't the 
router(s) still need to multicast RAs at the same frequency? The 
router(s) have no way to know whether all the hosts on the link initiate 
RS to refresh RA information.

Regards,
     Erik

---

From: Ole Troan <[email-masked]>
Date: Thu, 20 Feb 2014 15:43:15 +0100

<nochair>

> 4.8 seems to conflate the address assignment with DAD. Just because we might want to centralize the DAD checks doesn't imply that we want to remove the ability for the host to pick its own privacy enhanced interface-IDs to form its addresses.
> From a deployment perspective DHCPv6 is available for address assignment, but don't think we want to require that for WiFi or other links which have packet loss. (Packet loss occurs on wired networks as well, but the drop distribution is different - might happen during spanning tree reconvergence etc.)
> Note that DHCPv6 (RFC 3315) has a SHOULD for doing DAD on the addresses received from the DHCP server - needed since the server could be confused.

I'd like to explore the difference between DHCP address assignment and ND address registration a little more.
the state that is required in the network for "efficient ND", why cannot that be created and maintained by DHCP?

there are multiple ways of dealing with DAD in this scenario, including solutions that don't require host changes.
DHCP also supports temporary addresses btw.

cheers,
Ole

</nochair>

---

From: Andrew Yourtchenko <[email-masked]>
Date: Thu, 20 Feb 2014 17:56:02 +0100

On Wed, 19 Feb 2014, Erik Nordmark wrote:

> * One solicited RA per host joining the network (if solicited RAs are sent 
> using multicast)
> That isn't entirely true. Because of the rate limiting of multicast RAs if a 
> lot of hosts join at the same time, then there will be less.

agreed. If more than one host joins within ~ 3 seconds interval, only 
one multicast RA will be sent. I'll update the next rev with this 
clarification.

> If we believe that the flash crowd case is still important (when we wrote ND 
> the concern was around a building power failure and recovery - today it might 
> be a stadium setting), then one could implement a mix as follows:
> - If N additional RSs arrives before timer fires, then multicast one RA. 
> Otherwise when timer fires unicast RAs to the received RSs.
> For N=1 this is trivial.

Yeah, it's a very useful optimization to consider, we discussed it I think 
alread. The reason I did not include it yet, was to go over the 
tradeoffs without the pressure of the submission deadline being a few 
hours away :)

1) from the efficiency standpoint this approach is fantastic for the wired 
and for the wireless (the implementations that do the mcast->ucast 
conversion over the air the win is a bit less, but nonetheless there is a 
win since we send only one packet on the wired side instead of many and 
save the CPU on the gateway).

2) This adds an instantaneous behavior change into the network at the peak 
load conditions - so if that code path has a problem, this creates 
hard-to-debug situation.

3) There are a few trickier scenarios with L3 roaming (hosts arriving 
from other subnets onto the same AP) and that AP having a single 802.11 
group encryption key, which this behavior might make easier to 
accidentally have broken.

But, all in all it can be very useful to have a knob to allow to send a 
multicast RA at the time when the gateway can't handle the flood of 
incoming RS anyway and has to drop them.

>
> In section 4.3 there are two suggestions: MLD snooping and L2 unicast or L3 
> multicast packets. I think there are some operational concerns around MLD 
> snooping - I hope others can fill in some information in that space since I 
> don't know the issues.
> The multicast-over-unicast refers to SAVI as a way to collect the state, but 
> it doesn't specify where you see that state used. Presumably for DAD (and NS 
> in general unless you also do section 4.7)? But SAVI doesn't claim to have 
> all addresses since it is concerned with conflicts. Thus a host can exist and 
> be silent - but DAD would still expect to reach it. Or a host could have 
> moved to a different port and SAVI having stale state - yet DAD should work. 
> My point is that the details of how the state is 1) maintained and 2) used to 
> forward packets are key to analyzing how the neighbor discovery functionality 
> and robustness would be affected by this idea.

The hosts that are "quiet" are indeed a problem. However, I think it might 
be very OS-specific - the today's smartphones are anything but quiet :-)
So I think this will be much more of an issue for some special-purpose 
devices, which are also predominantly servers.

I think the smart-meter type of device would be in this category. But to 
access those, the remote party will need to know their address... so, if 
a device does some sort of service registration upon boot-up, then it 
will not be a quiet device anymore.

I haven't seen this failure in the "consumer portable devices" networks 
that I ran - so this is purely a mental construct.

I do agree that there are limitations - but I think it is very important 
to collect the data about the scope of these limitations, to keep a 
balanced approach.

The data I have as of today based on 5-6 networks with 10-20K hosts says 
it is quite minimal. But I reserve the right to be wrong and thus would be 
very interested to hear about other live networks where it is different.

(side note: if these sleepy hosts are quiet and are concerned that 
their address be known, they might consider to use DHCPv6 as well).

>
> I section 4.4 the document refers to proxy but without specifying which of 
> the different proxy approaches you have in mind.
> There is the proxy ND RFC, and there is the DAD proxy internet-draft in 6man. 
> Are you referring to one of those, or some slightly different form of proxy? 
> (For example, ND proxy would respond with the LLA of the router/AP, but that 
> might result in host movement looking like a duplicate address - again 
> depends on the details.)

I don't know the spec. I saw it implemented in a product, where two 
attached clients A and B during the ND saw the packet sequences that 
were close enough to the ND spec to achieve identical practical result 
and be compliant to the letter of it, yet allowed to perform the process 
more optimally from the broader perspective.

Arguably this kind of behavior would not require its own spec ?

>
> I don't have any issue with removing the somewhat arbitrary 9000 second max 
> AdvDefaultLifetime in section 5.1. However, the tradeoff for what default 
> lifetime to use in section 4.5 needs to take into account one additional 
> factor.
> router list should a router silently disappear. Thus for links that do not 
> have a fixed (set of) link-local address(es) for the router(s), having a high 
> default lifetime means that after a failure the hosts would have one entry in 
> the default router list which us unreachable - until that high lifetime 
> expires. I don't know if there has been a study on the performance impact of 
> that would have on the hosts e.g., how often they would re-probe the default 
> router.

This is indeed a useful consideration that is worth noting in a doc 
that would be removing the 9000 seconds max limit. Mind if sketch a -00 on 
this matter and send you the possible text ?

>
> Section 4.6 has the same concern. But 4.5 and 4.6  makes lots of sense e.g., 
> in a VRRP deployment where the link-local address of the virtual default 
> router would always be the same. Ditto for networks with a single point of 
> failure single router at a fixed address (e.g., if the router is always at 
> fe80::1 or some other fixed address.) Thus I think we should recommend 4.5 
> and 4.6 that within that applicability. Added benefit is that the routers 
> control it, hence the operator of the network can set the values higher for 
> VRRP or single router cases.

I totally agree! We probably might have a series of documents for 
different types of use cases - this could be very helpful for the folks 
deploying them. We could turn this doc into a "large-scale high-density 
WiFi (stadiums, exhibitions, campuses, etc.)" and then have couple of 
others describing other types of deployments. Let's discuss this in London 
?

>
> Section 4.7 talks about limiting the table space on the hosts. But clearing 
> the on-link bit in the advertised prefixes also has the benefit of removing 
> any multicast NS sent from hosts for non-link-local destinations. And per RFC

Yes, it's an omission by implication. Very good thing to note though, will 
fix, thanks!

> 4861 the table will be filled in by the router sending redirects. Thus the 
> router can be used to control which addresses get in the tables on the hosts 
> by choose whether and when to send the redirects (redirects with 
> target==destination).

Absolutely, thanks for this mention, I forgot to write this.
Yes, this section is incomplete without mentioning that the routers have 
to have the redirects off, indeed.

> (blocking link-locals) into a separate section, since it is quite different 
> than clearing the on-link bit. That idea has significant implications since 
> it changes the IP subnet model (RFC 4903 talks about this.) I not saying that 
> we shouldn't consider this, but I do think it would fall in a very different 
> category than the other ideas in this draft. Might even be best to have a 
> separate draft on this radical idea so it can be explored fully.

I thought RFC5942 was clarifying this (by the way I need to reference it).

My thought process was as follows: suppose we have 9000 people, each with 
a smartphone, wandering inside a large building or on campus. We want to 
limit the broadcast domains. So we will split these 9000 hosts into 256
subnets, each with 35-36 hosts. Those are small enough to be close to what 
is a typical "small network" so everything will be fine, and the protocols 
which do use link-local addresses will actually be able to function.

Good ? Seems so, but, there are problems with this approach:

1) "Lobby ambassador" problem: everyone arrives through the same entrance, 
and turns on their portable device there. Therefore, the assignment of a 
host to the subnet must happen there, not based on physical location, but 
on some loadbalancing algorithm. I have 256 subnets, so e.g. for 
predictability we can use a first byte of a hash of the host's mac 
address.

2) I need to span those 256 vlans across the entire venue, and ensure I 
place each of the hosts into the correct VLAN on each AP. A typical 
network of such a scale will have about 300 APs. I still need to send RAs 
within each vlan. I need to ensure that the hosts do not receive the 
"wrong" RAs. (and remember, there is not really a way in a naive 802.11 
imlementation to do "selective" multicast).

3) I have people that are able to see each other using multicast 
advertisements, but what relation do they have to each other ? Equality of 
the first byte of the hash of their mac address. Not very useful.

So I conclude, that while splitting the whole crowd into smaller subnets 
somewhat takes care of the floods at the expense of additional complexity, 
the resulting "working" link-local multicast protocols do not really make 
any sense - because the partition of the network does not have any other 
logic than pure loadbalancing.

So, I decide to explore the dimension of optimizing the multicast-limiting 
properties, and increase the number of subnets.

Extrapolating, we get that having a subnet per host seems to be the most 
logical approach. (Actually there is even a shipping product that does 
exactly this - a per-host unicast-only RAs with different /64s).

But since I have only one /64, I decide that not-on-link+no-redirect 
should give me the same functionality within a much smaller address space.

So, rather than have a bunch of per-host /64s, with a router forwarding 
between them, I get a bunch of per-host /128s and get a much more compact 
solution - because now I need to send only *one* RA per AP, and even if it 
needs to be a multicast one, it is not a catastrophy.

Of course, I need to disclaim: doing this in the home network with one 
AP and < 100 devices all belonging to a single person is not useful at 
all.

Doing this in a network with 9000 devices belonging to different people is 
the thing that made the most sense.

Now that we've cut off all the services, we need to figure out how to 
bring them back... And I think the further mental construct here that 
would be useful is to consider this as a giant homenet with a lot of 
guests. Thus, I think from the services discovery standpoint, the products 
of the DNS-SD workgroup will be very beneficial here.

>
> 4.8 seems to conflate the address assignment with DAD. Just because we might 
> want to centralize the DAD checks doesn't imply that we want to remove the 
> ability for the host to pick its own privacy enhanced interface-IDs to form 
> its addresses.
> From a deployment perspective DHCPv6 is available for address assignment, but 
> don't think we want to require that for WiFi or other links which have packet 
> loss.

This is more of a fallback scenario for those who want a 100% guarantee 
of address uniqueness in the network - using the existing mechanism.

Also as a trigger for another small discussion:

I think it's worth modeling the real-world experience in the networks with 
varying packet loss to see at which point it stops being usable, and go 
from there.

The classic IETF "Built-in hotel WiFi" topic: is it worth being extra-sure 
that DAD works in that scenario ? Or is it worth fixing the connectivity 
by bringing the loss down to acceptable level ? At how many retransmits do 
we declare a failure ? Could we then explore a similar approach to 
make the DAD more robust instead - keeping in mind the probability of 
failure should be quite low ?

This will be a good robustness improvement for the hosts that will 
immediately benefit them.

> (Packet loss occurs on wired networks as well, but the drop 
> distribution is different - might happen during spanning tree reconvergence 
> etc.)

yeah.

> In don't understand 4.9. Should I read it as a host shutting things down if 
> it goes to sleep even for a short time, and then waking up and multicasting a

depends on the definition of "short". Some hosts nap during the 100ms 
interval between the 802.11 beacons. Obviously would not work here.
Maybe not do it within 5 seconds of turning the screen off.
But I certainly know my gadgets stop being reachable if I do not touch 
them for a day"

> few RSs to get RAs, then multicasting a DAD probe for each assigned IPv6 
> address? If all the hosts did that then I think there would be more 
> multicasts.
> Note that even if the host doesn't revert to multicasting RSs, it still needs 
> to be concerned about a duplicate address having arrived on the link while it 
> was asleep (and not responding to any DAD probes.)
>
> The suggestion in section 5.2 is needs a lot more work. I tried to work out 
> some of these issues in section 8.9 in 
> But even if we figure out how to do that without causing lots of unneeded RS 
> load on the routers, in the case of your draft wouldn't the router(s) still 
> need to multicast RAs at the same frequency? The router(s) have no way to 
> know whether all the hosts on the link initiate RS to refresh RA information.

Sure it needs more whiteboarding. But if we assume one MAC address = one 
stack (this is a fair assumption I think?), it should be possible, here is 
a raw idea:

By the destination of the RS being unicast, the router knows the host is 
"resolicit-capable". The source address contains the MAC address, so it is 
sufficient information to create a neighbor entry with a flag 
"resolicit-capable" in it.

Subsequent neighbor entries with the same MAC address will inherit the 
"resolicit-capable" in it.

(Data structure detour: don't have to index by MAC address to do this.
Take two counting bloom filters "legacy hosts' MAC addresses" and 
"resolicit-capable hosts' MAC addresses". In case of a false positive 
match, prefer the membership in the "legacy hosts" to avoid blackout).

Keep the running counters of number of "resolicit-capable" ND entries and 
"legacy" ND entries.

When it's time to send a periodic RA, look at the counters, and if the 
count of "legacy" hosts is zero, no need to send it.

This algorithm gives multiple deployment choices:

1) "legacy" mode:

set the RA interval at 1/4 of the router lifetime. This way none of the 
hosts ever reaches the half remaining time, resolicits will never be sent, 
business as usual same as today.

2) "resolicit-capable" mode:

set the RA interval at 3/4 of the router lifetime. Then by this time we 
will have seen the RS from all the capable nodes, so we can decide whether
to send a multicast RA or not.

This of course needs to be worked on in order to avoid the synchronization 
issues (i.e. the host can not just blast an RS straight at half lifetime). 
But they are solvable: Assuming even the existing router lifetime limit 
of 2.5 hours, and RA interval of 1.5 hours, we have a space +- 0.5 hour of 
jitter.

Taking an arbitrary guess of 180000 hosts within that /64, a uniformly 
distributed jitter will give ~100pps of RSs which is more than manageable.

NB: this is a very first napkin sketch, so it is fairly simplistic. But I 
think if we were to tinker a bit more with the timers, this can be made to 
work.

There is also a question - what happen if the router has to clean up the 
legacy host ND entry due to resource constraints - this can be taken care 
of by temporarily going into "legacy only" mode for a period of a couple 
of RA lifetimes.

Of course, it requires the sleepy hosts to wake up every now and then. 
But with a lifted router lifetime limit, it will be every 9 hours.

Probably doable, but let's take a SWAG just to sanity check:
http://www.ti.com/lit/ds/symlink/cc3000.pdf and suppose they were one day 
to implement IPv6 with the similar power consumption.

I count the 802.11g, because of more efficient spectrum usage. This gives 
us maximum power consumption of 207ma, and a shutdown current of 5ua.

Now, assuming we couple it with something like 
http://www.atmel.com/Images/Atmel-2586-AVR-8-bit-Microcontroller-ATtiny25-ATtiny45-ATtiny85_Datasheet.pdf
which has standby of 0.1ua, and 12ma of active consumption (graph at page 
173 of this pdf)

Let's say 20 seconds should be enough to make all the necessary 802.11 
arrangements, send the data, send the solicitation, and receive 
advertisement.

Now, assuming we power it off the typical NiMh 1.2V elements with 1800mAh 
capacity, the continous running time off it will be 1800*3600/220 = 29454 
seconds.

Let's assume for simplicity that we wake up every 8 hours, so this gives 
60 seconds per day. This gives us the active life span of 490 days.

Given that even the low self-discharge batteries 
(http://en.wikipedia.org/wiki/Nickel%E2%80%93metal_hydride_battery#Low_self-discharge_cells) 
have retain rate of 70%-85% within a year, probably this is a reasonable 
lifecycle (also for this reason I am not accounting for the standby 
current, it's comparable to self-discharge).

This is of course also a quick napkin sketch, a properly engineered 
approach would take into account the 802.11 maintenance stuff - and, also 
it's really the "extreme" case which does not listen to the packets while 
asleep - so the lifetime would certainly vary - but I think this shows 
that this is not a totally unreasonable path.

--a

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 14:03:25 +0000

ErikN has commented on section 4.7.   I have additional comments.   ND as defined in RFC 4861 has no means to specify a prefix as off-link.  Thus the L-bit equals zero is not a valid means to signal an off-link prefix to SLAAC hosts.  ND can only specify an ipv6 destination as off-link.   Here is the reason.  The RA sets the A-bit to have hosts use SLAAC for ipv6 address assignment.  The router also sets the L-bit to zero in the RA.   After address assignment the host has an incomplete entry in the Neighbor Cache for a packet destination where the packet destination matches the host address up to 64 bits.  Due to the A-bit set, all addresses within the /64 assigned to the host are on-link.  Does the host issue an address resolution NS because the packet destination is on-link or due to the L-bit cleared in the received RA, the host sends the packet to the default router(s)?  

The only means in ND to specify all destinations as off-link is to have the RA include no PIO and have the A-bit cleared.   The hosts use DHCPv6 for address assignment.   This is the NBMA ND link model.  The router also supports a DAD Proxy.

For multicast behavior in network switches, please see RFC 4541.  Additionally MLD snooping is old and a burden for consumer devices to implement.   New protocol work is needed to mark packets for specific multicast flows so that hosts don't have to snoop MLD. 

Additionally, section 4.6 mentions the avalanche effect.  The ND protocol randomizes and adds jitter when issuing messages to the link precisely to alleviate any avalanche effect.  Thus I am not convinced of the avalanche effect.    May I also know why the wifi nodes in a 100,000 clients link will have not of entries in the Neighbor Cache?  Each client talks to one or two clients at a time.  So what traffic causes the number of entries in a client to be large?  How large is large?

Thanks,

Hemant

---

From: "Eric Levy- Abegnoli (elevyabe)" <[email-masked]>
Date: Fri, 21 Feb 2014 14:18:45 +0000

DHCP won't work without changes which would be a lot bigger that anything
envisioned for ND address registration.

DHCP does not maintain link-locals. That would be a major change to fix
that, including on hosts, since address requests are responded in unicast.
DHCP does not maintain <IP, MAC> binding. It does not have the MAC (not to
confuse with DUID).
DHCP is not on the forwarding path where we need the information quickly
Eric

On 20/02/14 15:43, "Ole Troan" <[email-masked]> wrote:

>there are multiple ways of dealing with DAD in this scenario, including
>solutions that don't require host changes.

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 15:08:09 +0000

I agree with Eric.   Without changing some IETF RFC's such as making sending DAD a MUST for every DHCPv6 acquired address or not changing RFCs we still will have to change ipv6 implementations to make DHCPv6 work.  
For this wifi services, the client and the access-point (AP) serving the client are in the same ipv6 ND link-local domain.  There are several solutions but implementations will change. 

A hack, but if the AP also includes DHCPv6 server, the AP knows from the upstream L2 frame of the DHCPv6 SOLICIT as to what the client mac-address and ipv6 link-local address is.   If the DHCPv6 server is north-bound of the AP, then the AP should support DHCPv6 relay services and the relay tracks client mac-address vs. ipv6 address(es) in the AP.  

Hemant

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Eric Levy- Abegnoli (elevyabe)
Sent: Friday, February 21, 2014 9:19 AM
To: Ole Troan; Erik Nordmark
Cc: Andrew Yourtchenko (ayourtch); 6man WG
Subject: Re: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

DHCP won't work without changes which would be a lot bigger that anything envisioned for ND address registration.

DHCP does not maintain link-locals. That would be a major change to fix that, including on hosts, since address requests are responded in unicast.
DHCP does not maintain <IP, MAC> binding. It does not have the MAC (not to confuse with DUID).
DHCP is not on the forwarding path where we need the information quickly Eric

On 20/02/14 15:43, "Ole Troan" <[email-masked]> wrote:

><nochair>
>
>> 4.8 seems to conflate the address assignment with DAD. Just because 
>>we might want to centralize the DAD checks doesn't imply that we want 
>>to remove the ability for the host to pick its own privacy enhanced 
>>spanning tree reconvergence etc.)  Note that DHCPv6 (RFC 3315) has a 
>>SHOULD for doing DAD on the addresses received from the DHCP server - 
>>needed since the server could be confused.
>
>the state that is required in the network for "efficient ND", why 
>cannot that be created and maintained by DHCP?
>

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Ole Troan <[email-masked]>
Date: Fri, 21 Feb 2014 16:39:56 +0100

Eric,

> envisioned for ND address registration.

can you elaborate?

> that, including on hosts, since address requests are responded in unicast.

do we need to care about link-locals? are they part of the problem we need to solve?

> DHCP does not maintain <IP, MAC> binding. It does not have the MAC (not to

true, but an onlink DHCP server or snooping relay could do an address discovery request to populate the ND table.

> DHCP is not on the forwarding path where we need the information quickly

what do you mean by that? if the function is on the first-hop router, then that's a deployment choice.
ND register also needs to solve how to deal with multiple routers. I believe -05 proposes that the host maintains the state in all on-link routers, but that has quite a few implications too.

cheers,
Ole

---

From: Andrew Yourtchenko <[email-masked]>
Date: Fri, 21 Feb 2014 16:52:50 +0100

On Fri, 21 Feb 2014, Hemant Singh (shemant) wrote:

> ErikN has commented on section 4.7.   I have additional comments.   ND 
>as defined in RFC 4861 has no means to specify a prefix as off-link. 
>Thus the L-bit equals zero is not a valid means to signal an off-link 
>prefix to SLAAC hosts.  ND can only specify an ipv6 destination as 
>off-link.   Here is the reason.  The RA sets the A-bit to have hosts use 
>SLAAC for ipv6 address assignment.  The router also sets the L-bit to 
>zero in the RA.   After address assignment the host has an incomplete 
>entry in the Neighbor Cache for a packet destination where the packet 
>destination matches the host address up to 64 bits.  Due to the A-bit

hmm I'd be really interested to see which host implementation behaves this 
way. I explicitly tested Linux, MacOS X, iOS - none of them do. I glanced 
over the shoulder at the behavior of the Windows box - and it was behaving 
in a similar fashion which does not match what you describe.

>set, all addresses within the /64 assigned to the host are on-link.  Does

RFC4861 says this:

    on-link     - an address that is assigned to an interface on a
                  specified link.  A node considers an address to be on-
                  link if:

                     - it is covered by one of the link's prefixes (e.g.,
                       as indicated by the on-link flag in the Prefix
                       Information option), or

                     - a neighboring router specifies the address as the
                       target of a Redirect message, or

                     - a Neighbor Advertisement message is received for
                       the (target) address, or

                     - any Neighbor Discovery message is received from
                       the address.

Note - that the "on-link" flag needs to be on as well, A-bit alone does 
not cut it.

The "off-link" shortcut of mine is unfortunate, since indeed it is not 
strictly speaking correct. But
"Not-on-link-but-because-ICMPv6-redirects-are-disabled-it-is-treated-as-off-link" 
is a bit too long to write. Probably a new term is due there.

>the host issue an address resolution NS because the packet destination is 
>on-link or due to the L-bit cleared in the received RA, the host sends 
>the packet to the default router(s)?

L-bit set to zero and no-redirects (Thanks Erik Nordmark for the 
correction of this important omission in the write-up!) worked fine to 
have the hosts send all the packets to the default router, thus avoiding 
performing the ND for this destination.

Of course, a better wording proposal is always welcome.

>
> The only means in ND to specify all destinations as off-link is to have 
>the RA include no PIO and have the A-bit cleared.   The hosts use DHCPv6 
>for address assignment.   This is the NBMA ND link model.  The router 
>also supports a DAD Proxy.
>
>> For multicast behavior in network switches, please see RFC 4541.

Interesting read, thanks.

>Additionally MLD snooping is old and a burden for consumer devices to 
>implement.   New protocol work is needed to mark packets for specific 
>multicast flows so that hosts don't have to snoop MLD.

The consumer switches can flood, it's not a problem there, really.

>
> Additionally, section 4.6 mentions the avalanche effect.  The ND 
>protocol randomizes and adds jitter when issuing messages to the link 
>precisely to alleviate any avalanche effect.  Thus I am not convinced of 
>the avalanche effect.    May I also know why the wifi nodes in a 100,000 
>clients link will have not of entries in the Neighbor Cache?  Each client 
>talks to one or two clients at a time.  So what traffic causes the number 
>of entries in a client to be large?  How large is large?

If you're not convinced, then you can skip this item for your 
deployment. I've seen it once, and there were multiple contributing 
factors, so I don't have a good data to convince you. Equally I don't have 
a spare lab with 9000 hosts of various operating systems, moving around. I 
left it because it does seem like a conservatively prudent thing to do.

--a

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 17:07:52 +0000

Andrew,

I'd be happy to work with you later to work out any text when I have the time.    Thanks to you and Lorenzo for taking up this work.  Please see in line below. 

-----Original Message-----
From: Andrew Yourtchenko (ayourtch) 
Sent: Friday, February 21, 2014 10:53 AM
To: Hemant Singh (shemant)
Cc: Erik Nordmark; IETF IPv6; Lorenzo Colitti
Subject: RE: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

>hmm I'd be really interested to see which host implementation behaves this way. I explicitly tested Linux, MacOS X, iOS - none of them do. I glanced over the shoulder at the behavior of the Windows box - and it was >behaving in a similar fashion which does not match what you describe.

Note I pointed out an ambiguity for the host that is still compliant to the specifications but the host forwarding model has ambiguity.   For a side note, RFC 5942 has deprecated the last two bullets of the on-link definition in RFC 4861.  That leaves two bullets and one is the Redirect bullet we can put aside for this discussion.   Then the leftover text for on-link from RFC 4861 is as follows. 

The definition starts with "an address that is assigned to an interface on a specified link".  This is it.  Every address assigned to an interface has a prefix length.  Thus the host deems any address within that prefix length to be on-link.  See also, RFC 4862 and search for "on-link" and you will find the text related to prefix lengths.  The first IPv6 implementation of a popular host implementation actually did use this interpretation and actually sent out an address resolution NS for a /64 destination matching its ipv6 address when the host was sent an RA with no PIO.   The host issued the NS for resolving a packet destination which was no in the host Neighbor Cache. 

>If you're not convinced, then you can skip this item for your deployment. I've seen it once, and there were multiple contributing factors, so I don't have a good data to convince you. Equally I don't have a spare lab with >9000 hosts of various operating systems, moving around. I left it because it does seem like a conservatively prudent thing to do.

It's not me.  Each time anyone has contacted the IETF mailers such as the 6man or the v6ops WG for an avalanche problem with IPv6 ND or DHCPv6, folks in the WG's have replied that both ND and the DHCPv6 specifications have randomization and at least ND has an additional jitter that avoids avalanches to cause any major problem in the ipv6 network.  I have personally tested routers with over 40,000 IPv6 clients with a router reload and did not see avalanche issues with ND nor DHCPv6.   Sorry, there was a typo in my email that said "not" when I should have said "lot".    I am interested to know why one client in a 100K client deployment has lot of IPv6 entries in the client Neighbor Cache?  How much is a lot?  10, 100, 1000 and how are these entries populated in the client?

Thanks,

Hemant

---

From: Ole Troan <[email-masked]>
Date: Fri, 21 Feb 2014 18:32:25 +0100

Hemant,

> The only means in ND to specify all destinations as off-link is to have the RA include no PIO and have the A-bit cleared.   The hosts use DHCPv6 for address assignment.   This is the NBMA ND link m…

RFC4861:
   Note, however, that a Prefix Information option
   with the on-link flag set to zero conveys no information concerning
   on-link determination and MUST NOT be interpreted to mean that
   addresses covered by the prefix are off-link.  The only way to cancel
   a previous on-link indication is to advertise that prefix with the
   L-bit set and the Lifetime set to zero.  The default behavior (see Section 5.2)
   when sending a packet to an address for which no
   information is known about the on-link status of the address is to
   forward the packet to a default router;

the effect of an unset L-flag is that all traffic is sent to the default router. redirects can be used to indicate that the destination is on-link. you are wrong with regards to the A-flag. and you are wrong with regards to addresses having prefixes.
that's an unfortunate short-hand we are using, but they are still different things. an address does not have a prefix length.

cheers,
Ole

---

From: Ole Troan <[email-masked]>
Date: Fri, 21 Feb 2014 18:34:58 +0100

and just to be clear:
me engaging in this debate with regards to ND register vs DHCP should in no way be interpreted that I agree that the solution to the problem (we're still working on defining), is a stateful ND table. ;-)

cheers,
Ole without hats.

---

From: "Eric Levy- Abegnoli (elevyabe)" <[email-masked]>
Date: Fri, 21 Feb 2014 17:36:02 +0000

Ole,

>> envisioned for ND address registration.

This is exactly what I did below :)
Adding link-local support in DHCP would require to change quite
substancially the flows to bootstrap the first address, when the host has
none yet.
Managing bindings requires to collect the information (as I said some bits
are missing today), maintain it, and serve it. That implies more changes.

>do we need to care about link-locals? are they part of the problem we

Yes we do. They are about half the problem, an't they? More if most
traffic is local to the link. Less otherwise.

>true, but an onlink DHCP server or snooping relay could do an address
>discovery request to populate the ND table.

I will not claim it cannot be done with DHCP. With changes. Like it could
be done with 10 others protocols storing and distributing addresses.
It just sound more difficult to do it there rather than in the place &
module where you already manage bindings.
Eric
 
>ND register also needs to solve how to deal with multiple routers. I
>believe -05 proposes that the host maintains the state in all on-link
>routers, but that has quite a few implications too.
>
>cheers,
>Ole

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 18:56:55 +0000

Ole,

-----Original Message-----
From: Ole Troan [mailto:[email-masked]] 
Sent: Friday, February 21, 2014 12:32 PM
To: Hemant Singh (shemant)
Cc: Erik Nordmark; 6man WG; Lorenzo Colitti; Andrew Yourtchenko (ayourtch)
Subject: Re: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

>   addresses covered by the prefix are off-link.  

The RA has the A-bit set and the L-bit cleared.  So as per the specification text above the L-bit is zero and the host does not get any on/off--link determination from the L-bit for the addresses covered by the prefix in the PIO.   The host configures its ipv6 address from the PIO in the RA using SLAAC.  Now the host has to send a packet out an interface and the packet destination in the host's Neighbor Cache.  The packet destination also matches the host's prefix to a /64.  So what does the host do?  

Issue 

(a) an address resolution NS (assuming the destination is on-link) due to SLAAC operation.

or

(b) send the packet to the default router(s).  The L-bit has conveyed no on- or off-link information.  So doesn't the host resort to (a) above.  If the host choses to go to operation (b) what signaled to the host that the destination is off-link?   If the host picked (a), why is that?

If you can't determine one concrete choice from the two above, we have an ambiguity. 

Hemant

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 19:24:36 +0000

Sigh and humble apologies.  One typo in my email below. 

Change 

Now the host has to send a packet out an interface and the packet destination in the host's Neighbor Cache

To

Now the host has to send a packet out an interface and the packet destination is NOT in the host's Neighbor Cache

Thanks,

Hemant

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Hemant Singh (shemant)
Sent: Friday, February 21, 2014 1:57 PM
To: Ole Troan
Cc: Erik Nordmark; Andrew Yourtchenko (ayourtch); 6man WG
Subject: RE: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

Ole,

-----Original Message-----
From: Ole Troan [mailto:[email-masked]] 
Sent: Friday, February 21, 2014 12:32 PM
To: Hemant Singh (shemant)
Cc: Erik Nordmark; 6man WG; Lorenzo Colitti; Andrew Yourtchenko (ayourtch)
Subject: Re: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

Issue 

or

Hemant

---

From: Ole Troan <[email-masked]>
Date: Fri, 21 Feb 2014 20:37:20 +0100

Hemant,

we are severely digressing here, but since it is Friday...

> If you can't determine one concrete choice from the two above, we have an ambiguity. 

there is no ambiguity. see the conceptual sending algorithm in section 5.2, RFC4861.
a prefix with the L flag = 0 is not added to the Prefix List. if the Prefix List is empty all packets will be sent to the default router, apart from packets destined to fe80::/10.

cheers,
Ole

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 20:06:58 +0000

Ole,

-----Original Message-----
From: Ole Troan [mailto:[email-masked]] 
Sent: Friday, February 21, 2014 2:37 PM
To: Hemant Singh (shemant)
Cc: Erik Nordmark; 6man WG; Lorenzo Colitti; Andrew Yourtchenko (ayourtch)
Subject: Re: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

>a prefix with the L flag = 0 is not added to the Prefix List. if the Prefix List is empty all packets will be sent to the default router, apart from packets destined to fe80::/10.

Thanks for this clarification - agreed.   Without any specific rules, if conceptually one thinks about ARP and ND and deems the default is off-link, then the decisions are very simple.  The algorithm for either ipv4 or ipv6 is deterministic in that if the algorithm cannot make a clear on-link determination, the algorithm defaults to off-link which implies sending traffic to the default router(s).  Only when clear indication of on-link exists, an address resolution NS is issued.

Hemant

---

From: Andrew Yourtchenko <[email-masked]>
Date: Fri, 21 Feb 2014 21:36:46 +0100

Hemant,

On Fri, 21 Feb 2014, Hemant Singh (shemant) wrote:

>>factors, so I don't have a good data to convince you. Equally I don't 
>>have a spare lab with >9000 hosts of various operating systems, moving 
>>around. I left it because it does seem like a conservatively prudent 
>>thing to do.
>
> It's not me.  Each time anyone has contacted the IETF mailers such as 
>the 6man or the v6ops WG for an avalanche problem with IPv6 ND or DHCPv6, 
>folks in the WG's have replied that both ND and the DHCPv6 specifications 
>have randomization and at least ND has an additional jitter that avoids 
>avalanches to cause any major problem in the ipv6 network.  I have 
>personally tested routers with over 40,000 IPv6 clients with a router

Were they 802.11 clients ? Wireless medium does have different properties.
Did you make any "flash failure" events ? Anyway, see below.

>reload and did not see avalanche issues with ND nor DHCPv6.   Sorry, 
>there was a typo in my email that said "not" when I should have said 
>"lot".    I am interested to know why one client in a 100K client 
>deployment has lot of IPv6 entries in the client Neighbor Cache?  How 
>much is a lot?  10, 100, 1000 and how are these entries populated in the 
>client?

Myself I did not like this portion of my reply with no quantifiable data 
(because operating on a basis of a belief is not a good engineering).

I happened to have a couple of hours of offline time, during which I tried 
to sketch the scenario and try to get as far as I can without building a 
lab.

It's a *very rough sketch*. If you think there can be parts that can be 
made better in it, tell me.

The initial assumptions for this thought experiment are as follows:

1) We have 10000 clients in a single /64.

2) There are multiple APs that bridge the traffic from wired onto
wireless medium, with the client count limited to 100 per AP.

3) there is 20x speed difference between unicast transmission and
multicast transmission.: the effective multicast speed is assumed to be
1 mbps, the effective unicast speed is assumed to be 20mbps data rate.

4) The APs are assumed to be "Naive" i.e. they do not perform any snooping
nor multicast->unicast conversions, but at the same time they are able to 
bridge the unicast traffic without flooding it between multiple access 
points. I.e. we assume a model where we have a single router (or an FHRP 
pair) and a set of 100 APs bridging the traffic.

(Corollary from the above: The effective unicast capacity is 100*20mbps, 
whereas the effective multicast capacity is 1*1Mbps, therefore the difference in 
throughput is 1000-fold).

Let's first consider the steady state. Suppose each host downloads
a file at 0.1 mbps.  Within each AP, therefore we have a 50% capacity 
utilization (0.1*100 = 10Mbps, we have 20 Mbps capacity).

It's easy to see this comfortably accomodates all the hosts. Obviuously
the unicast NUD in this traffic is fairly minimal, so I don't think it's 
even worth to count how much it is.

Now, lets look at a potential failure.

At the time of the NUD probe, it's enough to lose the 3 retries,
spaced 1 second apart.  The default reachable time is 30 seconds,
with the random jitter of 0.5 of that.

So, all that is needed to achieve a mass NUD failure is a ~30 second
outage during the period when all the hosts are sending the traffic.

A reboot of the majority of the networking gear takes several times longer 
than this.

Therefore, a crash of the default gateway during the peak hour is
a one guaranteed trigger for this to happen.

So, now we have a situation of 10000 hosts which have deleted their 
default gateway from the neighbor table, and send the multicast neighbor 
solicitations for it.

Assume the NS is 64 bytes, 10000 hosts sending such a packet means 64 
bytes * 8 bit * 10000 hosts = 5120000 bits/sec - or, 5 mbps.

Note, that this is only the shared bandwidth downstream back to the hosts 
- the airtime spent by the upstream traffic is 20x faster, so I am 
gratuitously discarding it.

Since the APs can not send the traffic at this rate, obviously, we will
need to drop some of it. Note, if the clients succeed with the ND to the 
default gateway, they will start streaming data again, so the effective 
multicast throughput will drop to the 0.5 mbps as soon as the noticeable 
portion of clients recovers.

Let's assume the best case of 20% of the clients managing to recover 
within the first second.

As we are approaching full recovery, the lesser number of clients will be 
able to get their multicast NS sent because the airtime is being taken by 
the payload traffic. Anyway, let's discard that and assume that every 
second 20% of the clients will recover.

This means that the recovery of the full set of the clients in this 
conditions will take an *absolute* minimum of 5 seconds.

Did I prove myself wrong ? Seems so. We can see that with some of 
the relaxed assumptions I took, the hosts seem to recover.

But, let's add to this a little bit of mDNS, and other multicast-loving 
protocols, which tend to generate a fair chunk of traffic when they detect 
the network was "restored".

This can shrink the available capacity several times. Add to this 
that the host does not necessarily stream at 100kbps, but might have 
higher data rates, and I think we can consider the available
multicast capacity at startup to be 1/10th of what it is in theory.

This is where the things may become interesting - 1/10th of the 
capacity means that it will take not 5, but 50 seconds for all the 
hosts to recover.

This means that the hosts which recovered in the very first second, will 
already be sending NUD traffic while the network is still under stress. If 
these packets are lost, the hosts might back into the pool of the 
"orphans" who are sending the multicast, because they delete their 
neighbor entry.

There are some other things to consider:

- I deliberately kept the scenario here with *one* ND entry.
Assuming your hosts are talking with 3-4 other hosts besides the default 
gateway. This increases the load and proportionally makes the dangerous 
state easier to achieve.

- another factor that I am omitting - that such a storm of ND 
onto the default gateway might cause rate-limiting of the control plane 
packets on the gateway. With some of the limits being as low as 1000pps, 
this might give a recovery time on the order of minutes even without the 
wireless multicast being the bottleneck, yet still resulting in a lot 
of multicast NS in the air still during the slow recovery.

This is about as precise of the construct I can build.

Is it perfect ? No, by no means. It also does assume that in the case of 
the default gateway the wireless performance will be limiting factor - I 
think it won't - so it's more of an appropriate scenario for a case of a 
p2p communications on the network. The default-gateway-only will be 
inherently much more stable, I think - because the multicast on the wired 
side is fast, so the drops on the wireless side will not matter.

It's probably worth it to change the text into "There is potential for the 
failing NUD to *contribute* to a longer recovery and possible creation of 
the locked situation in the case of flash failure - but the exact 
quantification of the impact in such an environment is a topic of further 
study".

And then maybe I could dump the above thought experiment into a separate 
draft, to see if the folks could contribute to the experiment / maybe 
someone could run it - and reference it in this item ?

It seems like an interesting area to dig a bit more in - creating a 
suitable model and playing with the parameters to see where it breaks 
seems like a useful exercise to understand how many hosts can there be in 
a single /64 on WiFi with a "naive" set of access-points.

Thoughts ?

--a

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 21:22:23 +0000

Andrew/Lorenzo,

Separate thought for section 4.7 since the L-bit cleared contention is resolved.  Section 4.7 has got to mention Redirect since the section has asked hosts to send traffic to the default router.   Since I got the Redirect thought, I went back to Erik Nordmark's review and he also mentions Redirect.   However, my point is different.   Section 4.7 needs to add text that "the router is configured to not to issue any Redirects for an IPv6 destination covered by the prefixes the router advertised in the RA with the L-bit cleared".   To keep the router configuration and internals simple, we can even go more restrictive in that text can say, "the router is configured to not issue any Redirects". 

Thanks,

Hemant

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 21:27:49 +0000

RFC 3315 came into being before RFC 4862.  RFC 4862 has a MUST clause for nodes to perform DAD for DHCPv6 addresses.  So shouldn't DHCPv6 clients comply with RFC 4862 for DAD behavior and ignore the SHOULD for performing DAD in RFC 3315?  This is the text from RFC 4862.

[Duplicate Address Detection MUST be performed on all unicast addresses prior to assigning them to an interface, regardless of whether they are obtained through stateless autoconfiguration, DHCPv6, or manual configuration,]

There is a set of exceptions to the above clause in RFC 4862 but those exceptions do not overrule what has been asked of DHCPv6 above for DAD. 

Hemant

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Fri, 21 Feb 2014 23:22:31 +0000

Andrew,

-----Original Message-----
From: Andrew Yourtchenko (ayourtch) 
Sent: Friday, February 21, 2014 3:37 PM
To: Hemant Singh (shemant)
Cc: Erik Nordmark; IETF IPv6; Lorenzo Colitti
Subject: RE: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

>Were they 802.11 clients ? 

No.  Sorry to not clarify that in my earlier email.  I have tested 40K broadband modems (some modem were from a virtual modem generator) on a service provider cable router in my lab. 

>Wireless medium does have different properties.

Agreed. 

Did you make any "flash failure" events ?

I thought a reload of a cable router and then getting modems online would be good to measure a failure though the failure may not be what you want and once I understand flash failure, I can reply further, if need be.    For service provider routers, I'd think a flash failure is the route processor card crashed but being a service provider router, the router moves over to the standby route processer card and thus this box won't simulate a flash failure if I understand flash failure correctly.  The switchover takes a few seconds for a service provider router.  

Wow, thanks for the detailed reply in rest of your email.  Once I get a chance to read that portion of your email, I will reply.

Thanks much,

Hemant

---

From: Ole Troan <[email-masked]>
Date: Sat, 22 Feb 2014 13:02:00 +0100

Eric,

> traffic is local to the link. Less otherwise.

I think that's an important point for the problem definition.
I do agree with you that if we need to register link-local addresses too, and change the model of link-local addresses always being on-link, then we might look for other solutions. that's not what efficient ND seems to do though. as far as I can see the ND efficient draft suggests ND for link-local traffic is done like today.

I did a couple of packet captures on my relatively fruity home network. I did not see a single data packet using link-local addresses. I saw ND and I saw lots of mDNS (someone needs to solve that problem too).
I don't claim my home network is representative, but I'd really like to see more data here.
I presume we're not going to justify this work based on the 10000 node dentist's office (1)?

any application depending on link-locals would also break in the routed home case.

and if we really wanted to solve it, what's the solution space?
 - redefine link-locals to have an L-flag?
 - physically or virtually restricting the link-local subnet size?
 - disable link-locals?

cheers,
Ole

(1) couldn't find a good reference to this use-case, anyone knows?
      I only got http://ietfreport.isoc.org/idref/draft-ietf-dhcp-ipv4-autoconfig/

---

From: Mark ZZZ Smith <[email-masked]>
Date: Sat, 22 Feb 2014 16:08:04 -0800 (PST)

----- Original Message -----
> From: Ole Troan <[email-masked]>
> To: Eric Levy- Abegnoli (elevyabe) <[email-masked]>
> Cc: Erik Nordmark <[email-masked]>; Andrew Yourtchenko <[email-masked]>; 6man WG <[email-masked]>
> Sent: Saturday, 22 February 2014 11:02 PM
> I do agree with you that if we need to register link-local addresses too, and 
> change the model of link-local addresses always being on-link, then we might 
> look for other solutions. that's not what efficient ND seems to do though. 
> as far as I can see the ND efficient draft suggests ND for link-local traffic is 
> done like today.
> 
> I did a couple of packet captures on my relatively fruity home network. I did 
> not see a single data packet using link-local addresses. I saw ND and I saw lots 
> of mDNS (someone needs to solve that problem too).

So I've been thinking about this problem off-and-on for a few months (specifically this, and more generally using hub-and-spoke link-layer forwarding in a multiaccess subnet, as I think it is a more general solution to the various RA/DHCPv6 spoofing etc problems, and also is more energy efficient), and I've started writing up a draft which suggests the following solution:

- Allow RA PIOs to announce the link-local prefix, to switch the on-link assumption for the link-local prefix off (the A bit in the PIO would be ignored for the LL prefix)

- Routers perform a limited form of ND proxying to facilitate hosts that don't support this LL RA PIO. ND proxying only occurs for Link-Local addresses.

The only issue I think that is left open after this link-local issue is host originated multicasts. The link-layer forwarding model would force them to only be received by the router(s) at the hub(s). The question is how do the multicasts reach the other hosts on the link?

Simplistically, the router could just flood the multicasts back to all hosts on the link (continuing to use the link-layer's broadcast/multicast capability retained for the devices at the hub(s)). I don't know if that is legal, but assuming it is, would the multicast origin host ignore its own multicasts? It could tell they were its own because the source address would be one of its own. I've had a bit of a look into the RFCs to see if this router and host multicast behaviour is defined, but haven't been able to find it. 

Alternatively, the router could selectively forward the multicasts back to all hosts on the link except the one it received it from. This could be achieved using link-layer unicasts and "MLDv2 Procedures for Link-Layer Unicast Delivery of Multicast IPv6" (http://tools.ietf.org/html/draft-smith-mldv2-link-unicast-00).

Regards,
Mark.

---

From: Erik Nordmark <[email-masked]>
Date: Mon, 24 Feb 2014 15:02:27 -0800

> DHCP also supports temporary addresses btw.
DHCP supports RFC 4941 temporary addresses, but it doesn't support 
others like CGA and stable-privacy-addresses. With IA_TA the DHCP server 
creates the temporary address, and that approach doesn't work with other 
addresses that need to be client-generated for security or privacy reasons.

One could envision changing DHCP to have a separate "please register 
this address" message and/or option, which would check against 
duplicates (of assigned and registered addresses) and then add it to the 
registered addresses in the server. But that would seem like a fair bit 
of new functionality in DHCP.

Currently RFC 3315 separates address assignment and DAD - the client 
SHOULD perform DAD on an address that is assigned by the DHCP server. 
This separation goes back to DHCPv4 where there was a similar check (I 
think using ICMP echo) which I believe is there for the reason that the 
DHCP servers list of assigned addresses could get out of sync with the 
hosts. (If DAD fails the host declines the address hence will get a 
different one.)

I don't see how we can do this without host changes even for the case of 
IA_NA and IA_TA DHCP assigned addresses; the hosts would need to remove 
the code which does the DAD check for DHCP. (And there would be 
additional DHCP protocol changes needed to support CGA and 
stable-privacy-addresses and any future address assignment scheme.)

    Erik

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 25 Feb 2014 07:39:32 +0000

And we need to remember that the assignment is just one component of the lifecycle of an address. 

When the client stops using an address, or moves the address somewhere else in the fabric, there is a need to update the network support that defends the address and proxies.

Cheers,
Pascal

> Le 25 févr. 2014 à 00:04, "Erik Nordmark" <[email-masked]> a écrit :
> 
> DHCP supports RFC 4941 temporary addresses, but it doesn't support others like CGA and stable-privacy-addresses. With IA_TA the DHCP server creates the temporary address, and that approach doesn't work with other addresses that need to be client-generated for security or privacy reasons.
> 
> One could envision changing DHCP to have a separate "please register this address" message and/or option, which would check against duplicates (of assigned and registered addresses) and then add it to the registered addresses in the server. But that would seem like a fair bit of new functionality in DHCP.
> 
> Currently RFC 3315 separates address assignment and DAD - the client SHOULD perform DAD on an address that is assigned by the DHCP server. This separation goes back to DHCPv4 where there was a similar check (I think using ICMP echo) which I believe is there for the reason that the DHCP servers list of assigned addresses could get out of sync with the hosts. (If DAD fails the host declines the address hence will get a different one.)
> 
> I don't see how we can do this without host changes even for the case of IA_NA and IA_TA DHCP assigned addresses; the hosts would need to remove the code which does the DAD check for DHCP. (And there would be additional DHCP protocol changes needed to support CGA and stable-privacy-addresses and any future address assignment scheme.)
> 
>   Erik
>

---

From: Mark ZZZ Smith <[email-masked]>
Date: Tue, 25 Feb 2014 01:12:23 -0800 (PST)

A 

----- Original Message -----
> From: Erik Nordmark <[email-masked]>
> To: Ole Troan <[email-masked]>; Erik Nordmark <[email-masked]>
> Cc: Andrew Yourtchenko <[email-masked]>; 6man WG <[email-masked]>
> Sent: Tuesday, 25 February 2014 10:02 AM
> might want to centralize the DAD checks doesn't imply that we want to remove 
> the ability for the host to pick its own privacy enhanced interface-IDs to form 
> its addresses.
> assignment, but don't think we want to require that for WiFi or other links 
> which have packet loss. (Packet loss occurs on wired networks as well, but the 
> drop distribution is different - might happen during spanning tree reconvergence 
> etc.)

<snip>

> stable-privacy-addresses and any future address assignment scheme.)

I think a DAD Proxy is what you're looking for -

RFC6957 - "Duplicate Address Detection Proxy."

More broadly though, I think a method that supports any forms of multicast from a host to the other hosts on the link, hair-pinned through the upstream router(s), would be better. It would support DAD operation, and would also support other multicasts from hosts, such as Multicast DNS for device (printer) discovery, and SSDP for DLNA server discovery. 

I've been thinking about this scenario for quite a while, and it seems to me that there are fundamentally two approaches to this problem. 

The first one involves minimal or no changes to the hosts (e.g., an optional RA PIO for link-local flagged as off-link and that is it, as a host local optimisation instead of ND proxy for link-local addresses), and emulate functions at the upstream router(s) that won't work because of host to host isolation within the link.

The other one involves wholesale changes to the hosts, and I think that would mean fully implementing an IPv6 NBMA subnet, perhaps with an RA option to announce to the hosts that the link is operating in NBMA mode, and the address(es) of the MARS cluster. 

The NBMA method would be convenient, as all these sorts of multicast/link-local etc. issues are already dealt with. However, I think wholesale changes to hosts to support it are probably infeasible.

Regards,
Mark.

---

From: Ole Troan <[email-masked]>
Date: Tue, 25 Feb 2014 10:28:35 +0100

Erik,

disclaimer:
let me first be clear about the context I'm discussing this in.
please don't interpret the fact that I have taken the position of 'defending' DHCP to mean that I'm proposing that as the solution. I just want to explore DHCP and understand the short-comings, and what unresolved problems there are. and I reserve the right to change my mind at any point. ;-)

<nochair>

> One could envision changing DHCP to have a separate "please register this address" message and/or option, which would check against duplicates (of assigned and registered addresses) and then add it…

RFC3315 says:
  The client MAY include addresses in the IAs as a hint to the server about addresses for
   which the client has a preference.

that's not too far off from the ND register semantics. in both cases the final say of which address the host can used is moved to the network.

> I don't see how we can do this without host changes even for the case of IA_NA and IA_TA DHCP assigned addresses; the hosts would need to remove the code which does the DAD check for DHCP. (And the…

are there two issues here?
1) DAD detecting a duplicate address
2) DAD wasting bandwidth and energy

on a WIFI link, DAD is sent L2 unicast to the AP. then the AP is expected to forward that packet as L2 multicast back onto the link. if we assume some smarts on the AP, then it doesn't have to send the packet back out on the wire, but only bridge it up to router(s) for example. (this isn't as far fetched as it seems, there are deployed products doing this already).

given all knowing state in the network, then collisions shouldn't happen. if the network suspected it had lost state, it could try to resolve the address on the link before assigning the address to the host.

leaving the host unchanged in this scheme only leaves us with the problem of the sleeping host waking up at ever beacon with a DTIM indication. since there wouldn't be a multicast message, it wouldn't have to stay awake actually waiting for a frame.

to conclude, if my understanding is correct (which it may very well not be, since I haven't really understood how 802.11 works before now), then DAD isn't a big issue. it can be solved (and is in deployed products) on the AP multicast handling.

we could of course also specify in a new IPv6 over foo document that DAD attempts = 0 on these links, or perhaps better add a signal in RAs.

cheers,
Ole

---

From: Ole Troan <[email-masked]>
Date: Tue, 25 Feb 2014 10:31:06 +0100

Pascal,

> When the client stops using an address, or moves the address somewhere else in the fabric, there is a need to update the network support that defends the address and proxies.

why isn't that: SOLICIT, CONFIRM, RENEW, REBIND, RELEASE?

cheers,
Ole without hat.

---

From: Ray Hunter <[email-masked]>
Date: Tue, 25 Feb 2014 12:04:42 +0100

Erik Nordmark wrote:
>>> we might want to centralize the DAD checks doesn't imply that we 
>>> want to remove the ability for the host to pick its own privacy 
>>> enhanced interface-IDs to form its addresses.

How would you do DAD for link-local addresses if the link is NBMA, and 
there's no multicast?

Don't you get into a chicken/egg problem with DHCPv6? [Section 16 of 
RFC3315 has a MUST for using a valid source address, either link-local 
-> multicast or an assigned address -> unicast]

>>> assignment, but don't think we want to require that for WiFi or 
>>> other links which have packet loss. (Packet loss occurs on wired 
>>> networks as well, but the drop distribution is different - might 
>>> happen during spanning tree reconvergence etc.)

Why not? DHCPv6 has retransmission (controlled by the client)
Why would ND registration be any different from the perspective of 
packet loss?

>>> Note that DHCPv6 (RFC 3315) has a SHOULD for doing DAD on the 
>>> addresses received from the DHCP server - needed since the server 
>>> could be confused.
>> I'd like to explore the difference between DHCP address assignment 
>> and ND address registration a little more.

I think that's a very good question.

>> there are multiple ways of dealing with DAD in this scenario, 
>> including solutions that don't require host changes.
> others like CGA and stable-privacy-addresses. With IA_TA the DHCP 
> server creates the temporary address, and that approach doesn't work 
> with other addresses that need to be client-generated for security or 
> privacy reasons.

DHCPv6 [3315] itself says very little about temporary addresses. I don't 
see why DHCPv6 couldn't be made to support 
draft-ietf-6man-stable-privacy-addresses etc., although it may undermine 
their raison d'etre if there isn't a trust relationship with the DHCPv6 
server. But then again you probably wouldn't trust DHCPv6 at all in that 
case.

> duplicates (of assigned and registered addresses) and then add it to 
> the registered addresses in the server. But that would seem like a 
> fair bit of new functionality in DHCP.
>
Why would new messages be required?

After discovering a unicast DHCPv6 server, wouldn't it be possible for a 
client to send a DHCPv6 request message containing an IA Address option 
in an IA-TA option?
Where the IA Address option contains the temporary IPv6 address 
requested to be registered? If I read RFC3315 18.2.1 that's standard 
behaviour isn't it?

> think using ICMP echo) which I believe is there for the reason that 
> the DHCP servers list of assigned addresses could get out of sync with 
> the hosts. (If DAD fails the host declines the address hence will get 
> a different one.)
>
Correct. There may be multiple DHCPv6 servers configured via relays, so 
you can also get into race conditions.

> I don't see how we can do this without host changes even for the case 
> of IA_NA and IA_TA DHCP assigned addresses; the hosts would need to 
> remove the code which does the DAD check for DHCP. (And there would be 

-- 
Regards,
RayH

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 03:10:47 -0800

> RFC6957 - "Duplicate Address Detection Proxy."
Mark,

RFC 6957 is useful for some types of deployment.

But it don't see how it relates to my observation (in response to Ole) 
that DHCPv6 doesn't help with different forms of address assignment and 
DAD. Even with DAD proxy, or some other form of DAD enhancement, we'd 
still do address assignment (SLAAC, DHCPv6, stable-private, etc) 
separately from DAD.
> More broadly though, I think a method that supports any forms of multicast from a host to the other hosts on the link, hair-pinned through the upstream router(s), would be better. It would support…
You mean instead of hair-pinning at L2 (through the AP) do the 
hair-pinning at L3 (through the router)?

Would the router decrement ttl as it forwards those packets?
(If not, then I think you are talking about hair-pinning at L2 but 
placing it in the router.)
Note that L2 hair-pinning typically (always?) does split horizon, hence 
you'd need to it in the devices along the path from the sender to the 
router (e.g., the AP) if you want to ensure that all receivers can get a 
copy of the packet.
Hair-pinning can potentially be avoided for L3 - if the router 
decrements ttl then it would be safer to send the multicast packet out 
the input port.

However, ttl decrements break the protocols that use the ttl=255 check 
(and ND isn't the only protocol which does this.)

Regards,
    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 03:59:34 -0800

On 2/20/14 8:56 AM, Andrew Yourtchenko wrote:
>
> Yeah, it's a very useful optimization to consider, we discussed it I 
> think alread. The reason I did not include it yet, was to go over the 
> 1) from the efficiency standpoint this approach is fantastic for the 
> wired and for the wireless (the implementations that do the 
> mcast->ucast conversion over the air the win is a bit less, but 
> nonetheless there is a win since we send only one packet on the wired 
> side instead of many and save the CPU on the gateway).
>
> 2) This adds an instantaneous behavior change into the network at the 
> peak load conditions - so if that code path has a problem, this 
> creates hard-to-debug situation.
>
> from other subnets onto the same AP) and that AP having a single 
> 802.11 group encryption key, which this behavior might make easier to 
I don't quite understand concern #3 (and #2). Since the routers doesn't 
know whether hosts rely on unsolicited multicast RAs, the multicast RAs 
must function. Thus adding the flash crowd optimization to multicast 
solicited RAs wouldn't be anything new or different - it would merely 
appear in one additional case.
>>
>> In section 4.3 there are two suggestions: MLD snooping and L2 unicast 
>> or L3 multicast packets. I think there are some operational concerns 
>> around MLD snooping - I hope others can fill in some information in 
>> that space since I don't know the issues.
>> The multicast-over-unicast refers to SAVI as a way to collect the 
>> state, but it doesn't specify where you see that state used. 
>> Presumably for DAD (and NS in general unless you also do section 
>> 4.7)? But SAVI doesn't claim to have all addresses since it is 
>> concerned with conflicts. Thus a host can exist and be silent - but 
>> DAD would still expect to reach it. Or a host could have moved to a 
>> different port and SAVI having stale state - yet DAD should work. My 
>> point is that the details of how the state is 1) maintained and 2) 
>> used to forward packets are key to analyzing how the neighbor 
>> discovery functionality and robustness would be affected by this idea.
>
> The hosts that are "quiet" are indeed a problem. However, I think it 
> might be very OS-specific - the today's smartphones are anything but 
> quiet :-)
I guess my first question was how this would work when there is no 
SAVI-like state for an address. (In that case you don't have a unicast 
address to which you can send the packet.) Would you multicast or drop?
> I think the smart-meter type of device would be in this category. But 
> to access those, the remote party will need to know their address... 
> so, if a device does some sort of service registration upon boot-up, 
> then it will not be a quiet device anymore.
But the service registration could have happened when the device was 
first installed - I don't think that is likely to happen each time the 
device wakes up.
> I haven't seen this failure in the "consumer portable devices" 
> networks that I ran - so this is purely a mental construct.
Yes, but IPv6 needs to work for other sleepy devices. We want whatever 
improvements we do to the standards in this space to be useful for the 
next 20 years or so. (Products, whether end devices or routers/switches 
can operate on shorter time scales and with more limited applicability 
than the core standards.)

>
> I do agree that there are limitations - but I think it is very 
> important to collect the data about the scope of these limitations, to 
> keep a balanced approach.
>
> The data I have as of today based on 5-6 networks with 10-20K hosts 
> says it is quite minimal. But I reserve the right to be wrong and thus 
> would be very interested to hear about other live networks where it is 
> different.
>
Using DHCPv6 doesn't reduce multicast (neither for address resolution 
nor for dad). One could change DHCP to record the link-layer addresses 
so that the routers would ask the DHCP servers for the link-layer 
addresses, but that seems like a lot of change.

Also, see my response to Ole on the other ways that the DHCPv6 we 
currently have doesn't help solve the problems at hand. [I see others 
brought up the same concerns around DHCPv6 on the list.]

>
>>
>> I section 4.4 the document refers to proxy but without specifying 
>> which of the different proxy approaches you have in mind.
>> There is the proxy ND RFC, and there is the DAD proxy internet-draft 
>> in 6man. Are you referring to one of those, or some slightly 
>> different form of proxy? (For example, ND proxy would respond with 
>> the LLA of the router/AP, but that might result in host movement 
>> looking like a duplicate address - again depends on the details.)
>
> and be compliant to the letter of it, yet allowed to perform the 
> process more optimally from the broader perspective.
>
Yes, because it depending on the details it might break SeND and other 
ND options, or it might only be robust and useful in very limited 
applicability for instance in specific topologies (such as a single 
wireless controller).

>> I don't have any issue with removing the somewhat arbitrary 9000 
>> second max AdvDefaultLifetime in section 5.1. However, the tradeoff 
>> for what default lifetime to use in section 4.5 needs to take into 
>> account one additional factor.
>> The default lifetime serves to garbage collect entries from the 
>> default router list should a router silently disappear. Thus for 
>> links that do not have a fixed (set of) link-local address(es) for 
>> the router(s), having a high default lifetime means that after a 
>> failure the hosts would have one entry in the default router list 
>> which us unreachable - until that high lifetime expires. I don't know 
>> if there has been a study on the performance impact of that would 
>> have on the hosts e.g., how often they would re-probe the default 
>> router.
>
> that would be removing the 9000 seconds max limit. Mind if sketch a 
> -00 on this matter and send you the possible text ?
That would be excellent.

>> Section 4.6 has the same concern. But 4.5 and 4.6  makes lots of 
>> sense e.g., in a VRRP deployment where the link-local address of the 
>> virtual default router would always be the same. Ditto for networks 
>> with a single point of failure single router at a fixed address 
>> (e.g., if the router is always at fe80::1 or some other fixed 
>> address.) Thus I think we should recommend 4.5 and 4.6 that within 
>> that applicability. Added benefit is that the routers control it, 
>> hence the operator of the network can set the values higher for VRRP 
>> or single router cases.
>
> different types of use cases - this could be very helpful for the 
> folks deploying them. We could turn this doc into a "large-scale 
> high-density WiFi (stadiums, exhibitions, campuses, etc.)" and then 
> have couple of others describing other types of deployments. Let's 
> discuss this in London ?
Yep.
>> In think it would be good to separate out the second half of section 
>> 4.7 (blocking link-locals) into a separate section, since it is quite 
>> implications since it changes the IP subnet model (RFC 4903 talks 
>> about this.) I not saying that we shouldn't consider this, but I do 
>> think it would fall in a very different category than the other ideas 
>> in this draft. Might even be best to have a separate draft on this 
>> radical idea so it can be explored fully.
>
5942 re-states the intent of 4861 with more detail. But doesn't change 
the above semantics. (It does remove these two bullets from on-link:
        *  a Neighbor Advertisement (NA) message is received for the
           (target) address, or
        *  any Neighbor Discovery message is received from the address.
)

The only case I know of where "blocking link-locals" is part of our 
standards is for multi-link subnets, for instance in 6lowpan. But in 
that case it was part of the architecture from the start. The issue we 
have with the general ND evolution is that we need to support the 
existing assumptions, and many of those assumptions come from IPv4. For 
instance, the use and semantics of an IPv4 link-local multicast has been 
carried to IPv6 link-locals.

Removing that will cause some breakage.
>
> My thought process was as follows: suppose we have 9000 people, each 
> with a smartphone, wandering inside a large building or on campus. We 
> want to limit the broadcast domains. So we will split these 9000 hosts 
> into 256
> subnets, each with 35-36 hosts. Those are small enough to be close to 
> what is a typical "small network" so everything will be fine, and the 
> protocols which do use link-local addresses will actually be able to 
> function.
>
> 1) "Lobby ambassador" problem: everyone arrives through the same 
> entrance, and turns on their portable device there. Therefore, the 
> assignment of a host to the subnet must happen there, not based on 
> physical location, but on some loadbalancing algorithm. I have 256 
> subnets, so e.g. for predictability we can use a first byte of a hash 
> of the host's mac address.
>
> 2) I need to span those 256 vlans across the entire venue, and ensure 
> I place each of the hosts into the correct VLAN on each AP. A typical 
> network of such a scale will have about 300 APs. I still need to send 
> RAs within each vlan. I need to ensure that the hosts do not receive 
> the "wrong" RAs. (and remember, there is not really a way in a naive 
> 802.11 imlementation to do "selective" multicast).
>
> advertisements, but what relation do they have to each other ? 
> Equality of the first byte of the hash of their mac address. Not very 
> useful.
>
> So I conclude, that while splitting the whole crowd into smaller 
> subnets somewhat takes care of the floods at the expense of additional 
> complexity, the resulting "working" link-local multicast protocols do 
> not really make any sense - because the partition of the network does 
> not have any other logic than pure loadbalancing.
I agree with all that.

For those types of deployments having /127 subnets (plus DHCPv6 PD if 
someone wants a personal area network behind their phone) makes a lot of 
sense.

But my issue is that the goals we are trying to achieve are more general 
than the stadium (or broadband subscriber) use case.

> Doing this in a network with 9000 devices belonging to different 
> people is the thing that made the most sense.
Agreed.
> guests. Thus, I think from the services discovery standpoint, the 
> products of the DNS-SD workgroup will be very beneficial here.
I don't think such an approach (first break it, then get folks to fix 
whatever broke whether that means fixing an IETF standard, some common 
implementation, or various to us unknown and proprietary approaches 
makes sense for a retrofit.

Only makes sense for a greenfield deployment where there are no 
assumptions that existing protocols and code continue to work.
>> links which have packet loss.
>
> This is more of a fallback scenario for those who want a 100% 
> guarantee of address uniqueness in the network - using the existing 
> mechanism.
But RFC 3315 doesn't guarantee uniqueness, which is why the host needs 
to do perform DAD in addition.
> I think it's worth modeling the real-world experience in the networks 
> with varying packet loss to see at which point it stops being usable, 
> and go from there.
>
> The classic IETF "Built-in hotel WiFi" topic: is it worth being 
> extra-sure that DAD works in that scenario ? Or is it worth fixing the 
> connectivity by bringing the loss down to acceptable level ? At how 
> many retransmits do we declare a failure ? Could we then explore a 
> similar approach to make the DAD more robust instead - keeping in mind 
> the probability of failure should be quite low ?
>
If we want a DAD probe mechanism that is more robust to failure then I 
think we should just reuse the techniques in ACD (RFC 5227) which 
performs ongoing conflict detection. However, that results in or 
broadcast/multicast messages!

If we want both less multicast packets *and* a more robust DAD 
(including efficiently handling DAD for sleeping nodes), then I don't 
see any approach other than making some devices on the link be able to 
speak more authoritatively about the addresses present on the link. 
Those devices can try to build that state implicitly (gleaning from 
packets on the link), or explicitly. (I'll try to capture the 
differences between those before next week.)
>
>> In don't understand 4.9. Should I read it as a host shutting things 
>> down if it goes to sleep even for a short time, and then waking up 
>> and multicasting a
>
The loss of reachability is expected.
But there is also a question about the efficiency when the device wakes 
up. Whether it needs to start from scratch (multicast RS, do DAD 
multicasts and wait for a second), or whether it can do DNA (unicast a 
NS to the router(s) to check it is on the same link) and avoid waiting 
for the lack of a response to a DAD probe.
>
> Sure it needs more whiteboarding. But if we assume one MAC address = 
> one stack (this is a fair assumption I think?), it should be possible, 
> here is a raw idea:
Need to bring a whiteboard to London ... ;-)
>
> By the destination of the RS being unicast, the router knows the host 
> is "resolicit-capable". The source address contains the MAC address, 
> so it is sufficient information to create a neighbor entry with a flag 
Or add a flag to the RS to say "I will re-solicit" ...
> Keep the running counters of number of "resolicit-capable" ND entries 
> and "legacy" ND entries.
I can see how you increment those counters (based on seeing a RS with a 
new MAC address.)
But when do you decrement them?

(I don't think NUD from the router can be used for this, because the 
host can be unreachable from the router for a few seconds due to radio 
issues, yet the host will still consider itself connected and not 
restart by sending an RS.)
> set the RA interval at 1/4 of the router lifetime. This way none of 
> the hosts ever reaches the half remaining time, resolicits will never 
> be sent, business as usual same as today.
>
> set the RA interval at 3/4 of the router lifetime. Then by this time 
> we will have seen the RS from all the capable nodes, so we can decide 
> whether
Presumably the RSs can be lost. One way to handle this is by agressive 
retransmissions (no RA after 1 second, then resend RS). Another way is 
to space the "retransmissions" inside the lifetime range e.g., by having 
the host send a RS when 1/3 of the router lifetime has passed, next when 
2/3 has passed. (Generalize to k/N for N transmissions before giving up.)
>
> This of course needs to be worked on in order to avoid the 
> synchronization issues (i.e. the host can not just blast an RS 
> straight at half lifetime). But they are solvable: Assuming even the 
> existing router lifetime limit of 2.5 hours, and RA interval of 1.5 
> hours, we have a space +- 0.5 hour of jitter.
>
Your suggestions made me realize the stuff I put in section 8.9 in 
efficient-nd-05 is more complex than it needs be. No need to worry about 
prefix and other lifetimes - sufficient to look at the default router 
lifetime and make sure (by sending RSs) that the host hears from all the 
routers. That's good.
>
> NB: this is a very first napkin sketch, so it is fairly simplistic. 
> But I think if we were to tinker a bit more with the timers, this can 
> be made to work.
>
> There is also a question - what happen if the router has to clean up 
> the legacy host ND entry due to resource constraints - this can be 
> taken care of by temporarily going into "legacy only" mode for a 
> period of a couple of RA lifetimes.
The router is free to just discard the NCEs - unless we change DAD to 
use them in some proxy approach.
Even if we change DAD that way, the router can just do the SAVI thing 
(send a unicast NS to the host and see if it still there) - no need to 
multicast RAs to clean up.
However, the unicast NS cleanup (or any other cleanup driven by the 
router expecting the host to respond) has issues with sleeping hosts. (I 
just realize their might be different forms of sleep - completely off 
and will take up based on a timer, or being woken up by arriving 
(unicast) packets. I don't know if there is confusion around that and 
whether we need different terms to make it more clear.)
> But with a lifted router lifetime limit, it will be every 9 hours.
Assuming DAD is handled without involving the hosts.
> http://www.ti.com/lit/ds/symlink/cc3000.pdf and suppose they were one 
> day to implement IPv6 with the similar power consumption.
>
> I count the 802.11g, because of more efficient spectrum usage. This 
> gives us maximum power consumption of 207ma, and a shutdown current of 
> 5ua.
>
> which has standby of 0.1ua, and 12ma of active consumption (graph at 
> page 173 of this pdf)
>
That leaves plenty of time to do the current DAD.

However, Margaret had some numbers a while back with lots more frequent 
wakeups, but with very short runtime. I don't know if that was captured 
in RFC 6574 or in Margarets slides from the workshop. In any case, one 
issue I remember is that for Ipv4 the runtime was a lot less than one 
second, but with IPv6 there was an additional 1 second to wait for DAD 
to complete which blew the power budget.

    Erik
>
> Now, assuming we power it off the typical NiMh 1.2V elements with 
> 1800mAh capacity, the continous running time off it will be 
> 1800*3600/220 = 29454 seconds.
>
> Let's assume for simplicity that we wake up every 8 hours, so this 
> gives 60 seconds per day. This gives us the active life span of 490 days.
>
> have retain rate of 70%-85% within a year, probably this is a 
> reasonable lifecycle (also for this reason I am not accounting for the 
> standby current, it's comparable to self-discharge).
>
> approach would take into account the 802.11 maintenance stuff - and, 
> also it's really the "extreme" case which does not listen to the 
> packets while asleep - so the lifetime would certainly vary - but I 
> think this shows that this is not a totally unreasonable path.
>
> --a
>
>>
>> Regards,
>>    Erik
>>
>>
>>
>

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 04:22:10 -0800

On 2/25/14 3:04 AM, Ray Hunter wrote:
> there's no multicast? '
You can't.
(I'm not the one advocating turning an Ethernet into an NBMA network.)

But we do have an open issue if we want to make link-local addresses 
efficient, since 4861 assumes that they are always on-link hence 
multicasting NSs etc.
> -> multicast or an assigned address -> unicast]
[I'm not the one saying that DHCPv6 would help her - in fact I think it 
is a distraction to conflate address assignment with DAD.]

DHCPv6 assumes that a link-local address can be formed and used (hence 
do DAD on the link-local) before the first DHCPv6 packet can be sent. 
Yet another reason why DHCPv6 doesn't help to reduce the impact of DAD.

> Why would ND registration be any different from the perspective of 
I think there is some context missing.

DHCPv6 can not be used for CGA, stable-privacy, or any other addresses 
that a host can form. For that reason I don't think we should say that 
if you want better reliability you should use DHCPv6 (and loose the 
ability to use those forms of addresses.) [But see below.]

ARO is retransmitted as specified in efficient-nd-05.
> DHCPv6 [3315] itself says very little about temporary addresses. I 
> don't see why DHCPv6 couldn't be made to support 
> draft-ietf-6man-stable-privacy-addresses etc., although it may 
> undermine their raison d'etre if there isn't a trust relationship with 
> the DHCPv6 server. But then again you probably wouldn't trust DHCPv6 
> at all in that case.
RFC 3315 has explicit support for 6574 - the address is formed on the 
dhcpv6 server.
That approach implies that any other address assignment scheme such a 
stable-privacy needs to be done on the server. Implies giving the key to 
the server etc. Removes their raison d'etre AFAICT.

One could of course define new DHCP options and behaviors to take a 
different approach to support stable-privacy, but that would be a fair 
bit of DHCP changes.

But you raise questions about RFC 3315 vs. DHCPv6 server behavior when 
addresses are included in the IA_TA option. I can't see anything about 
this in 3315. Makes sense to double-check.

> After discovering a unicast DHCPv6 server, wouldn't it be possible for 
> a client to send a DHCPv6 request message containing an IA Address 
> option in an IA-TA option?
I don't see a sentence in 18.2.1 which says that the address in an IA is 
registered (after checked that it hasn't been assigned to anybody else).

The number of IAs (IA_NA and IA_TA) are used to determine how many 
address to assign to the host.

Section 12 says "Clients ask for temporary addresses and servers assign 
them." Nothing about checking whether they are already assigned to 
somebody else in that section. Ditto in section 22.5 - "The server will 
generate new temporary addresses and return them to the client."

(But I haven't checked DHCPv6 server implementations.)

    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 05:21:58 -0800

On 2/21/14 6:03 AM, Hemant Singh (shemant) wrote:
> ErikN has commented on section 4.7.   I have additional comments.   ND as defined in RFC 4861 has no means to specify a prefix as off-link.  Thus the L-bit equals zero is not a valid means to signal an off-link prefix to SLAAC hosts.
It is correct that L=0 doesn't mean anything. For instance, if a prefix 
P1 has been advertised in a PIO with L=1 and later the host receives a 
PIO with P1 and L=0, that doesn't change anything. (But section 6.2.7 in 
RFC 4861 would flag it if routers advertise the same prefix that way - 
Oops - 6.2.7 doesn't explicitly list the PIO flags which I think is an 
omission.)

However, ND does indicate that prefixes are off link. That is done by 
never advertising them, or only advertising them with L=0. That can be 
overridden by having individual IPv6 addresses be considered on-link 
through a redirection where target==destination.
> The only means in ND to specify all destinations as off-link is to have the RA include no PIO and have the A-bit cleared.   The hosts use DHCPv6 for address assignment.   This is the NBMA ND link m…
That isn't correct. PIO with L=0 works. SLAAC can be used even if L=0 - 
the case when L=0 and A=1.

Regards,
    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 05:39:09 -0800

On 2/21/14 12:36 PM, Andrew Yourtchenko wrote:
>
> Myself I did not like this portion of my reply with no quantifiable 
> data (because operating on a basis of a belief is not a good 
> engineering).
>
> I happened to have a couple of hours of offline time, during which I 
> tried to sketch the scenario and try to get as far as I can without 
> building a lab.
>
> It's a *very rough sketch*. If you think there can be parts that can 
> be made better in it, tell me.
Andrew,
I like this sketch. A few comments below.
> nor multicast->unicast conversions, but at the same time they are able 
> to bridge the unicast traffic without flooding it between multiple 
> access points. I.e. we assume a model where we have a single router 
> (or an FHRP pair) and a set of 100 APs bridging the traffic.
>
> (Corollary from the above: The effective unicast capacity is 
> 100*20mbps, whereas the effective multicast capacity is 1*1Mbps, 
> therefore the difference in throughput is 1000-fold).
>
> the unicast NUD in this traffic is fairly minimal, so I don't think 
> it's even worth to count how much it is.
>
> A reboot of the majority of the networking gear takes several times 
> longer than this.
>
> default gateway from the neighbor table, and send the multicast 
> neighbor solicitations for it.
>
Just to make it clear, each NS retransmissions is 5 Mbits and the 
retransmission time is 1 second. Hence with 4861 we get 5 mbps for 3 
seconds. However, they hosts aren't that likely to all discover the NUD 
failure in the same 3 second window. With ReachableTime=30 seconds it 
depends on when there last successful NUD probe or higher-level 
reachability advise can in. Hmm - with TCP the reachability advise might 
come with every left window edge advancement i.e. they would synchronize 
quite closely.

>
> Note, that this is only the shared bandwidth downstream back to the 
> hosts - the airtime spent by the upstream traffic is 20x faster, so I 
> am gratuitously discarding it.
>
> need to drop some of it. Note, if the clients succeed with the ND to 
> the default gateway, they will start streaming data again, so the 
> effective multicast throughput will drop to the 0.5 mbps as soon as 
> the noticeable portion of clients recovers.
While the multicast NS will see downstream drops, what matters for the 
resolution is that the router receives it and responds. Thus why 
wouldn't that part work?
The NA from the router will be unicast so if that unicast isn't dropped 
due to the multicasts then it should resolve quickly. I don't know if 
unicast is prioritized higher than multicast or not.

(We'll see downstream overload due to the multicasts, but that might 
just last for 1 second if the NS/NA to/from the router gets through.)

> As we are approaching full recovery, the lesser number of clients will 
> be able to get their multicast NS sent because the airtime is being 
> taken by the payload traffic. Anyway, let's discard that and assume 
> that every second 20% of the clients will recover.
>
> Did I prove myself wrong ? Seems so. We can see that with some of the 
> relaxed assumptions I took, the hosts seem to recover.
>
> But, let's add to this a little bit of mDNS, and other 
> multicast-loving protocols, which tend to generate a fair chunk of 
> traffic when they detect the network was "restored".
>
> This can shrink the available capacity several times. Add to this that 
> the host does not necessarily stream at 100kbps, but might have higher 
> data rates, and I think we can consider the available
Yes, in general TCP will fill the pipe thus at each AP all of the 
downstream bandwidth will be used.

But if the streaming comes via the default router (which crashed or was 
unreachable for a few seconds), then the stream would also slow down due 
to TCP, in which case there is less of a multicast overload issue. There 
might be cases where this TCP slowdown doesn't happen though.

   Erik

> This means that the hosts which recovered in the very first second, 
> will already be sending NUD traffic while the network is still under 
> stress. If these packets are lost, the hosts might back into the pool 
> of the "orphans" who are sending the multicast, because they delete 
> their neighbor entry.
>
> Assuming your hosts are talking with 3-4 other hosts besides the 
> default gateway. This increases the load and proportionally makes the 
> dangerous state easier to achieve.
>
> - another factor that I am omitting - that such a storm of ND onto the 
> default gateway might cause rate-limiting of the control plane packets 
> on the gateway. With some of the limits being as low as 1000pps, this 
> might give a recovery time on the order of minutes even without the 
> Is it perfect ? No, by no means. It also does assume that in the case 
> of the default gateway the wireless performance will be limiting 
> factor - I think it won't - so it's more of an appropriate scenario 
> for a case of a p2p communications on the network. The 
> default-gateway-only will be inherently much more stable, I think - 
> because the multicast on the wired side is fast, so the drops on the 
> wireless side will not matter.
>
> It's probably worth it to change the text into "There is potential for 
> the failing NUD to *contribute* to a longer recovery and possible 
> creation of the locked situation in the case of flash failure - but 
> the exact quantification of the impact in such an environment is a 
> topic of further study".
>
> And then maybe I could dump the above thought experiment into a 
> separate draft, to see if the folks could contribute to the experiment 
> / maybe someone could run it - and reference it in this item ?
>
> seems like a useful exercise to understand how many hosts can there be 
> in a single /64 on WiFi with a "naive" set of access-points.
>
> Thoughts ?
>
> --a
>

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 05:57:27 -0800

> I think that's an important point for the problem definition.
Ole,
It is at least a question we should ask in the problem definition.
> I do agree with you that if we need to register link-local addresses too, and change the model of link-local addresses always being on-link, then we might look for other solutions. that's not what…
See section 18 "Open Issues".
(6lowpan-nd has a limited-applicability approach to handle link-local 
addresses which we can't use in general.)
> any application depending on link-locals would also break in the routed home case.
It wouldn't necessarily break - it all depends on the semantics that 
folks expect.
For instance, in 6lowpan a link-local which matches the radio topology 
makes a lot of sense - is useful for routing and other adjacency 
protocols (such as trickle distribution).

A (large) home could be intentionally set up so that the subnet in the 
north wing gives you the mDNS etc resources for the north wing.
But a more likely case is that the routing happens between different 
network topologies (Ethernet, IEEE 802.15.4, etc) where there is no 
particular user intent to have different resource discovery scopes tied 
to the different links.
>   - physically or virtually restricting the link-local subnet size?
I don't think we can remove link-local from the whole link. For 
instance, the routers might use VRRP and VRRP uses IPv6 link-locals 
(with a ttl=255 check.) And in some topologies the routers might have 
OSPFv3 or other adjacencies over the link, and the routing protocols use 
IPv6 link-locals.

At best we could have the hosts not send packets using link-locals for 
applications.

Telling the host to send link-local packets to the routers wouldn't help 
with protocols with do the ttl=255 check, and I don't know if this 
applies to mDNS or other protocols which use link-locals.
However, it would seem to be useful to avoid the hosts trying to send 
such link-locals, since they would just use resources for no real use. 
Thus a "send link-locals to router" indication in an RA seems useful if 
we want to solve this. That enables some compatibility for hosts (which 
interpret that flag but do not change their application protocols) since 
the router could proxy mDNS etc (handwaving a bit).

    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 06:10:31 -0800

Mark,

I understand the potential benefits of doing this for multicast.
Are you also looking at requiring it for unicast?
(Direct host-host unicast would seem more efficient - but in some use 
cases with filtering requirements it would make sense to send those via 
the router as well.)
> - Routers perform a limited form of ND proxying to facilitate hosts that don't support this LL RA PIO. ND proxying only occurs for Link-Local addresses.
*If* we need to solve this I think that is a reasonable design.

We do need to look at ttl=255 in a bit more detail - I assume you'd 
decrement ttl when the routers forward link-locals back out on the link.
> Simplistically, the router could just flood the multicasts back to all hosts on the link (continuing to use the link-layer's broadcast/multicast capability retained for the devices at the hub(s)).…
DAD as currently defined would break, since DAD can't cope with hearing 
its own packets.
But if we do that differently (enhanced-dad draft with nonce, or address 
registration) that wouldn't be an issue for hosts that implement that 
new approach. Still need to handle legacy hosts doing current DAD though 
... need to think about that one.

Once DAD is taken care of and we assume unique link-layer addresses, 
then having the host ignore multicast packets with its own IP source 
address will ensure that other protocols/applications don't see any change.

> Alternatively, the router could selectively forward the multicasts back to all hosts on the link except the one it received it from. This could be achieved using link-layer unicasts and "MLDv2 Proc…
That would be hard to implement in existing hardware. The hardware which 
performs multicast replication assumes the same packet is being 
replicated (with some handling of ttl decrement vs. not in order to 
handle the bridge-route-bridge paradigm for devices that are both 
routers and bridges i.e. where some ports see a L2 forwarded multicast 
packet and other ports get the same packet after a ttl decrement by the 
L3 forwarding function.)

Regards,
    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 06:45:49 -0800

Ole,

Thanks for finding that text.

So a potential approach would be to have the host form its address using 
the SLAAC method it prefers (such as stable-privacy) and then go through 
the DHCP (discovery plus request/response having the IA_NA/TA with the 
formed address). I don't know if we can use the 2 packet DHCP exchange 
for this or need the 4 packet one.  The host would get lifetimes back 
from the DHCP server so it would know when it needs to renew.

If all the hosts on the link were to do this, and the DHCP service is 
100% reliable, then this would ensure no duplicates.
But it has issues with incremental deployment where some hosts do this 
but not the legacy hosts, since the legacy hosts would do current DAD 
probes and not find out about the addresses registered with the DHCP 
server. If the DHCP server is on-link it could proxy respond (and also 
implicitly learn about registrations from those DAD probes - with the 
limitations for implicit registrations which I'm trying to write down), 
but what about the case of DHCP relay(s) on the link and an off-link 
DHCP server?

In that case it seems like the DHCP relays need to relay all DAD packets 
to the DHCP server, so that it can respond (duplicate or ok), and build 
it state.

And should the DHCP service loose some or all of its state (e.g., a cold 
standby is started after a hardware failure) then it doesn't have a way 
to rebuild its state from the network until the lifetimes cause the new 
hosts to renew. And for legacy hosts the DHCP server/relay would need to 
glean on data packets to reliably find out about addresses that are in 
use since the legacy host do not "renew" with an additional DAD probe in 
a predictable time.

Thus it seems like in order to solve DAD using DHCPv6 we need to
1. solve how do reliably do implicit registrations from DAD (and data 
packets?) for legacy hosts in DHCP relay/server
2. new interaction between DHCP relay and server to handle legacy hosts 
in normal operation
3. ditto in the case of DHCP server state loss

Note that this doesn't remove the need to multicast Neighbor Solutions 
for address resolutions, since the DHCP server or relay is not required 
to be implemented in the router (it might be in some deployments and 
products). Even when it implemented in the router, there would need to 
be a mechanism to share the state of IP->MAC address between the routers 
on the link.
Thus to solve that we need
4. DHCP server track MAC address to get IP->MAC binding
5. Sharing of DHCP server state with all the routers on the link

With the above we still haven't addressed the unsolicited multicast RAs, 
thus we'd also need
6. Method to avoid multicast RAs - host driven unicast RS to refresh?

Note that if we solve 1, 5', and 6 we have a complete solution i.e.
1. solve how do reliably do implicit registrations from DAD (and data 
packets) in routers
5': sharing of that registration state (explicitly or implicitly) 
between the routers on the link
6. Method to avoid multicast RAs - host driven unicast RS to refresh?

Thus I don't see how "reusing DHCP" makes for less work - it seems to 
create a lot more  work by changing not only hosts and routers but also 
DHCP servers and DHCP relays plus defining additional need for state 
sharing which doesn't exist today.

[And as I've indicated, I think that #1 is actually hard even if there 
are products that do that with constrained applicability today. More 
about that later since it doesn't affect the difference between "reuse 
DHCP" and other parts of the solution space AFAICT.]
> 2) DAD wasting bandwidth and energy
Yes, we can think about those separate.
I'd call #1 something like "More robust/reliable DAD" in the presense of 
packet loss, sleeping hosts, etc.
> on a WIFI link, DAD is sent L2 unicast to the AP. then the AP is expected to forward that packet as L2 multicast back onto the link. if we assume some smarts on the AP, then it doesn't have to send…
Presumably the AP would do that for a large set of packets - e.g., all 
solicited-node MC addresses.
> given all knowing state in the network, then collisions shouldn't happen. if the network suspected it had lost state, it could try to resolve the address on the link before assigning the address to…
Unless the host that has the address is in deep sleep at that point in time.

> leaving the host unchanged in this scheme only leaves us with the problem of the sleeping host waking up at ever beacon with a DTIM indication. since there wouldn't be a multicast message, it would…
If the network lost the state about the address, how can it remember to 
which MAC address it can unicast it?
I think it needs to multicast the frame. And the host needs to be in 
shallow sleep so the multicast will wake it up.

(Philosophically having completely separate behavior and code to deal 
with infrequent events like this is a bad idea; that behavior/code will 
not be exercised hence it is very likely to have bugs.)
> to conclude, if my understanding is correct (which it may very well not be, since I haven't really understood how 802.11 works before now), then DAD isn't a big issue. it can be solved (and is in d…
I don't see the details of how this would work. Hence I can't declare it 
to be solved.
> we could of course also specify in a new IPv6 over foo document that DAD attempts = 0 on these links, or perhaps better add a signal in RAs.
DAD attempts = 0 might make sense for EUI-64 derived addresses. But 
since we are moving away from those I'm not sure that is a good idea.

     Erik

---

From: Ole Troan <[email-masked]>
Date: Tue, 25 Feb 2014 16:45:33 +0100

Erik,

> So a potential approach would be to have the host form its address using the SLAAC method it prefers (such as stable-privacy) and then go through the DHCP (discovery plus request/response having the IA_NA/TA with the formed address). I don't know if we can use the 2 packet DHCP exchange for this or need the 4 packet one.  The host would get lifetimes back from the DHCP server so it would know when it needs to renew.

you could only use 2 message exchange if there was a single DHCP server on the link.

> If all the hosts on the link were to do this, and the DHCP service is 100% reliable, then this would ensure no duplicates.
> But it has issues with incremental deployment where some hosts do this but not the legacy hosts, since the legacy hosts would do current DAD probes and not find out about the addresses registered with the DHCP server. If the DHCP server is on-link it could proxy respond (and also implicitly learn about registrations from those DAD probes - with the limitations for implicit registrations which I'm trying to write down), but what about the case of DHCP relay(s) on the link and an off-link DHCP server?

DHCP always has an on-link presence, be it the relay or the server. in theory (which is very far from practice), relays are stateless. for this discussion I suggest we assume DHCP must be deployed so that there is a DHCP server on-link and on every router.

with regards to your legacy point, with DHCP we can "declare" that there is no legacy. if you want to connect to networks like these, then DHCP is the only option for address assignment. just like there are networks deployed like that today.

> In that case it seems like the DHCP relays need to relay all DAD packets to the DHCP server, so that it can respond (duplicate or ok), and build it state.

any scheme we can find that has no or few legacy implications would be good.
in the above scheme with the deployment restrictions outlined, DAD messages could be silently dropped at the AP/router.

> And should the DHCP service loose some or all of its state (e.g., a cold standby is started after a hardware failure) then it doesn't have a way to rebuild its state from the network until the lifetimes cause the new hosts to renew. And for legacy hosts the DHCP server/relay would need to glean on data packets to reliably find out about addresses that are in use since the legacy host do not "renew" with an additional DAD probe in a predictable time.

I agree that DHCP performs poorly with regards to "server initiated" messages.
after a state-loss event you could do one of (again assuming no host changes):
 - DHCP bulk lease query
 - fall back to normal ND behaviour
 - low timers
 - learn DHCP database from neighbouring routers

> 1. solve how do reliably do implicit registrations from DAD (and data packets?) for legacy hosts in DHCP relay/server
> 2. new interaction between DHCP relay and server to handle legacy hosts in normal operation

I don't think we should go there, and if that's where we want to go, then I'm much more open at looking at new approaches.

> Note that this doesn't remove the need to multicast Neighbor Solutions for address resolutions, since the DHCP server or relay is not required to be implemented in the router (it might be in some deployments and products). Even when it implemented in the router, there would need to be a mechanism to share the state of IP->MAC address between the routers on the link.

yes. so I have never heard about a case where a DHCP relay wasn't on the forwarding path.
relays or servers must in any case be on-link. I suggested above that we simplify the scenario (and discussion) by assuming the DHCP server was on-link and running on a router.

with regards to the IP -> MAC binding.
 - DHCP server gleans that information from ND/DHCP packets (e.g. RS) (ugly score of 3)
 - Hosts implement RFC6939 (ugly score of 2)
 - DHCP server uses normal address resolution to learn the L3/L2 mapping

> 5. Sharing of DHCP server state with all the routers on the link

yes, that's the trickiest part of this scheme.
this is a challenge whatever solution you choose here. even with ND register, where the proposal is for hosts to register with all on-link routers, you may get into a state where the databases aren't in sync. and also it forces hosts to quickly detect the appearance of a new router on the link.

a fallback to normal ND is the only approach I can see off hand. for any solution we propose here.

> With the above we still haven't addressed the unsolicited multicast RAs, thus we'd also need

yes, that's independent of this I think. but as I think Andrew outlines in his document you can get reasonably far with that without host changes.

> 1. solve how do reliably do implicit registrations from DAD (and data packets) in routers
> 5': sharing of that registration state (explicitly or implicitly) between the routers on the link
> Thus I don't see how "reusing DHCP" makes for less work - it seems to create a lot more  work by changing not only hosts and routers but also DHCP servers and DHCP relays plus defining additional need for state sharing which doesn't exist today.

if we need to put the state in the network, then I don't see that the maintaining network state for the L3/L2 mapping is different regardless of how you wrap the data.

> [And as I've indicated, I think that #1 is actually hard even if there are products that do that with constrained applicability today. More about that later since it doesn't affect the difference between "reuse DHCP" and other parts of the solution space AFAICT.]

I think DHCP is largely a deployment choice. and that unless we agree on the assumptions, e.g. no legacy (which means we don't need 1) then I agree with you. DHCP as a solution only makes sense within certain constraints.

again I don't see how 6 isn't orthogonal to this. ah, sorry double negative. ;-)

4 and 5 are the biggest issues with a DHCP based solution.
no host changes, but definitely router/DHCP changes, and possibly AP changes.

> I'd call #1 something like "More robust/reliable DAD" in the presense of packet loss, sleeping hosts, etc.
>> 
> Presumably the AP would do that for a large set of packets - e.g., all solicited-node MC addresses.
>> 
> If the network lost the state about the address, how can it remember to which MAC address it can unicast it?
> I think it needs to multicast the frame. And the host needs to be in shallow sleep so the multicast will wake it up.
> 
> (Philosophically having completely separate behavior and code to deal with infrequent events like this is a bad idea; that behavior/code will not be exercised hence it is very likely to have bugs.)

agree.

> I don't see the details of how this would work. Hence I can't declare it to be solved.
>> 
> DAD attempts = 0 might make sense for EUI-64 derived addresses. But since we are moving away from those I'm not sure that is a good idea.

DAD was invented largely to detect manual misconfiguration.
as long as the network is assigning the address instead of the host, DAD isn't required.
the assigning entity could do the duplicate detection itself, and that it can do from its own valid source address.

depending on network entities detecting a DAD message and using that to inject state in the network is a poor choice I think. DAD is unreliable, default DAD attempts is zero. although it is sent as L2 unicast it will still have a typical failure rate of 1-5% on WIFI.

if we want network state for link-local addresses, and duplicate address detection, and also allow applications to use link-local addresses, then we need to 'figure' something out. I agree that without host changes that's tricky. while the network might detect and install state for the link-local address on e.g. RS/DHCP SOLICIT, the host might not be receptible to changing it after DAD has passed.

lots of good points here, do you collect them for a later draft?
 
cheers,
Ole

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 25 Feb 2014 18:03:18 +0000

Well, 

Release is of course important, but what´s really missing is NUD.

It is quite useful for the proxy to be able to revalidate the presence of the device when the lifetime is long and traffic very occasional.

Our implementation has a "reachable" timer that, when elapsed, will force to go back to the device before answering to a lookup.

Cheers,  

Pascal

> Le 25 févr. 2014 à 10:31, "Ole Troan" <[email-masked]> a écrit :
> 
> Pascal,
>

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 25 Feb 2014 13:53:06 -0800

On 2/25/14 7:45 AM, Ole Troan wrote:

I guess the host would need to do the discover once, but then can do
each address registration with just 2 messages.

You can't have it on every router unless there is a server-server
protocol to coordinate the address assignments/registrations between
the routers that are on the same link. There is no IETF standard for
this (it was discussed and abandoned in the DHC WG a long time
back).

I fear you are equating DHCPv6 as implemented by hosts today, and
the extensions on the hosts necessary to use DHCP for address
registrations. If you say that you will only support hosts that
implement DHCP address registrations (for all addresses SLAAC -
4941, stable-privacy, microsoft privacy addresses, etc) then you
will support no hosts at all on those links.

That sounds like a non starter to me.

Who would respond to that? There is no relay in your picture as I
understand it.

When the DHCP server or router crashes it typically starts clean,
hence it doesn't know whether it lost state, or is a newly
installed/configured node. Thus it wouldn't be "fall back" AFAICT,
but actually a requirement to always start in normal ND behavior and
then somehow move towards DHCP address registrations.

You probably want to walk through all the details and write down the
proposal - more efficient than me trying to guess and second guess
how you think it might work and poke holes at my own guesses.

- low timers

I don't see how you can support legacy hosts without that. We can't
assume that legacy hosts do DHCP address registrations for RFC 4941
temporary addresses when the A flag is set. Clearing the A flag
means that some legacy hosts will not work on those links.

The last choice implies a multicast NS from the DHCP server, right?

The idea was to avoid modifying hosts (by reusing DHCP) and reduce
multicast - that seems to be hard.

efficient-nd (that you like to call nd register) is robust against
database inconsistencies - eventual consistency.

If a new router appears and there are no use of multicast NS then
the router needs to get the NCE database in any approach.

That could be using a yet-to-be-invented-and-standardized db sync
protocol, vendor proprietary protocol, using unicast NS/NA between
the routers, or involving the hosts as in ARO.

In all of those cases the new router might be asked to forward
packets before it has a complete db. Might result in packets being
dropped on the floor, or fallback to multicast NS to the hosts.

I want to verify that I understand the context. Are you saying that
any time there are two or more routers on the link we can not use
the new scheme and can only use 4861? That doesn't make sense - we
must aim higher.

Sorry, I don't understand your comment nor how it relates to the
part of my text that you quoted.

Please explain.

Explain how a host which uses RFC 4941 address does not need
changes.

No, it was actually a lot more about EUI-64 derived IIDs and the
fact that some vendors had issued NICs which all had the same MAC
addresses. Those rare situations were a pain to track down. Hence at
or around the time of the 1st Stockholm IETF we came up with this
single NUD probe.

Since then we have temporary addresses, CGA, stable-privacy etc
which depend on DAD being able to detect duplicates.

But manual misconfig never entered in the picture AFAIR.

Yes, but that doesn't work with the security and privacy model for
stable-privacy and CGA and their likes. Hence a non-starter AFAICT.

depending on network entities detecting a DAD message and using that to inject state in the network is a poor choice I think. DAD is unreliable, default DAD attempts is zero.

Default DAD attempts is one - not zero.

But it isn't reliable in the case of random packet loss, spanning
tree reconvergence triggering packet loss, sleeping hosts, etc, etc.

although it is sent as L2 unicast it will still have a typical failure rate of 1-5% on WIFI.

Yes, I'm trying to capture the design tradeoffs.

Counting on you all to keep me honest ;-)

Erik

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 26 Feb 2014 08:07:50 +0000

I can not find a reference to TOS bits or priority in 4861...

Certainly NUD should be higher priority than traffic?

Pascal

> Le 25 févr. 2014 à 14:39, "Erik Nordmark" <[email-masked]> a écrit :
> 
>> Myself I did not like this portion of my reply with no quantifiable data (because operating on a basis of a belief is not a good engineering).
>> 
>> I happened to have a couple of hours of offline time, during which I tried to sketch the scenario and try to get as far as I can without building a lab.
>> 
>> It's a *very rough sketch*. If you think there can be parts that can be made better in it, tell me.
> Andrew,
>> nor multicast->unicast conversions, but at the same time they are able to bridge the unicast traffic without flooding it between multiple access points. I.e. we assume a model where we have a single router (or an FHRP pair) and a set of 100 APs bridging the traffic.
>> 
>> (Corollary from the above: The effective unicast capacity is 100*20mbps, whereas the effective multicast capacity is 1*1Mbps, therefore the difference in throughput is 1000-fold).
>> 
>> a file at 0.1 mbps.  Within each AP, therefore we have a 50% capacity utilization (0.1*100 = 10Mbps, we have 20 Mbps capacity).
>> 
>> the unicast NUD in this traffic is fairly minimal, so I don't think it's even worth to count how much it is.
>> 
>> A reboot of the majority of the networking gear takes several times longer than this.
>> 
>> So, now we have a situation of 10000 hosts which have deleted their default gateway from the neighbor table, and send the multicast neighbor solicitations for it.
>> 
>> Assume the NS is 64 bytes, 10000 hosts sending such a packet means 64 bytes * 8 bit * 10000 hosts = 5120000 bits/sec - or, 5 mbps.
> Just to make it clear, each NS retransmissions is 5 Mbits and the retransmission time is 1 second. Hence with 4861 we get 5 mbps for 3 seconds. However, they hosts aren't that likely to all discover the NUD failure in the same 3 second window. With ReachableTime=30 seconds it depends on when there last successful NUD probe or higher-level reachability advise can in. Hmm - with TCP the reachability advise might come with every left window edge advancement i.e. they would synchronize quite closely.
> 
>> 
>> Note, that this is only the shared bandwidth downstream back to the hosts - the airtime spent by the upstream traffic is 20x faster, so I am gratuitously discarding it.
>> 
>> need to drop some of it. Note, if the clients succeed with the ND to the default gateway, they will start streaming data again, so the effective multicast throughput will drop to the 0.5 mbps as soon as the noticeable portion of clients recovers.
> While the multicast NS will see downstream drops, what matters for the resolution is that the router receives it and responds. Thus why wouldn't that part work?
> The NA from the router will be unicast so if that unicast isn't dropped due to the multicasts then it should resolve quickly. I don't know if unicast is prioritized higher than multicast or not.
> 
> (We'll see downstream overload due to the multicasts, but that might just last for 1 second if the NS/NA to/from the router gets through.)
> 
>> 
>> Let's assume the best case of 20% of the clients managing to recover within the first second.
>> 
>> As we are approaching full recovery, the lesser number of clients will be able to get their multicast NS sent because the airtime is being taken by the payload traffic. Anyway, let's discard that and assume that every second 20% of the clients will recover.
>> 
>> This means that the recovery of the full set of the clients in this conditions will take an *absolute* minimum of 5 seconds.
>> 
>> Did I prove myself wrong ? Seems so. We can see that with some of the relaxed assumptions I took, the hosts seem to recover.
>> 
>> But, let's add to this a little bit of mDNS, and other multicast-loving protocols, which tend to generate a fair chunk of traffic when they detect the network was "restored".
>> 
>> This can shrink the available capacity several times. Add to this that the host does not necessarily stream at 100kbps, but might have higher data rates, and I think we can consider the available
> Yes, in general TCP will fill the pipe thus at each AP all of the downstream bandwidth will be used.
> 
> But if the streaming comes via the default router (which crashed or was unreachable for a few seconds), then the stream would also slow down due to TCP, in which case there is less of a multicast overload issue. There might be cases where this TCP slowdown doesn't happen though.
> 
>  Erik
> 
>> 
>> This is where the things may become interesting - 1/10th of the capacity means that it will take not 5, but 50 seconds for all the hosts to recover.
>> 
>> This means that the hosts which recovered in the very first second, will already be sending NUD traffic while the network is still under stress. If these packets are lost, the hosts might back into the pool of the "orphans" who are sending the multicast, because they delete their neighbor entry.
>> 
>> Assuming your hosts are talking with 3-4 other hosts besides the default gateway. This increases the load and proportionally makes the dangerous state easier to achieve.
>> 
>> - another factor that I am omitting - that such a storm of ND onto the default gateway might cause rate-limiting of the control plane packets on the gateway. With some of the limits being as low as 1000pps, this might give a recovery time on the order of minutes even without the wireless multicast being the bottleneck, yet still resulting in a lot of multicast NS in the air still during the slow recovery.
>> 
>> Is it perfect ? No, by no means. It also does assume that in the case of the default gateway the wireless performance will be limiting factor - I think it won't - so it's more of an appropriate scenario for a case of a p2p communications on the network. The default-gateway-only will be inherently much more stable, I think - because the multicast on the wired side is fast, so the drops on the wireless side will not matter.
>> 
>> It's probably worth it to change the text into "There is potential for the failing NUD to *contribute* to a longer recovery and possible creation of the locked situation in the case of flash failure - but the exact quantification of the impact in such an environment is a topic of further study".
>> 
>> And then maybe I could dump the above thought experiment into a separate draft, to see if the folks could contribute to the experiment / maybe someone could run it - and reference it in this item ?
>> 
>> It seems like an interesting area to dig a bit more in - creating a suitable model and playing with the parameters to see where it breaks seems like a useful exercise to understand how many hosts can there be in a single /64 on WiFi with a "naive" set of access-points.
>> 
>> Thoughts ?
>> 
>> --a
>

---

From: Ole Troan <[email-masked]>
Date: Wed, 26 Feb 2014 11:31:13 +0000

Erik,

> I guess the host would need to do the discover once, but then can do each address registration with just 2 messages.

after discovery the other protocol exchanges are 2 message.
that should be the same as efficient ND. first you discover the routers, then you exchange messages with chosen routers. 

> You can't have it on every router unless there is a server-server protocol to coordinate the address assignments/registrations between the routers that are on the same link. There is no IETF standard for this (it was discussed and abandoned in the DHC WG a long time back).

there is work apparently on failover:
RFC7031
draft-ietf-dhc-dhcpv6-failover-design

by moving the state into the network, maintaining that state on all first-hop router is going to be a challenge. that is the same problem, if you use DHCP or if you use ND.

> I fear you are equating DHCPv6 as implemented by hosts today, and the extensions on the hosts necessary to use DHCP for address registrations. If you say that you will only support hosts that implement DHCP address registrations (for all addresses SLAAC - 4941, stable-privacy, microsoft privacy addresses, etc) then you will support no hosts at all on those links.
> 

you are quick to jump to conclusions. ;-)
I think we need to separate what _can_ be done within the protocol and how the protocol is deployed today.

in DHCP as deployed today, you get address assignment. what addresses are assigned to hosts is defined in the address assignment policy on the DHCP server. just like on any other network that uses DHCP for address assignment.

is it really a requirement that hosts must generate addresses? if so why? what problem does that solve in this context?

> Who would respond to that? There is no relay in your picture as I understand it.
> When the DHCP server or router crashes it typically starts clean, hence it doesn't know whether it lost state, or is a newly installed/configured node. Thus it wouldn't be "fall back" AFAICT, but actually a requirement to always start in normal ND behavior and then somehow move towards DHCP address registrations.
> 
> You probably want to walk through all the details and write down the proposal - more efficient than me trying to guess and second guess how you think it might work and poke holes at my own guesses.

right, maintaining this state is the crux of the problem. which is also the main reason why I'm skeptical about efficient ND.

> I don't see how you can support legacy hosts without that. We can't assume that legacy hosts do DHCP address registrations for RFC 4941 temporary addresses when the A flag is set. Clearing the A flag means that some legacy hosts will not work on those links.

as above. legacy hosts get address assignment, hosts that care about self generating interface identifiers will get address registration (if allowed by network policy).

> The idea was to avoid modifying hosts (by reusing DHCP) and reduce multicast - that seems to be hard.

yes, but it would only be on initial connect.
I think Andrew would argue that you can achieve the same optimisation by increasing the reachable timers.
full circle. ;-)

> efficient-nd (that you like to call nd register) is robust against database inconsistencies - eventual consistency. 
> 
> If a new router appears and there are no use of multicast NS then the router needs to get the NCE database in any approach. 
> That could be using a yet-to-be-invented-and-standardized db sync protocol, vendor proprietary protocol, using unicast NS/NA between the routers, or involving the hosts as in ARO.
> 
> In all of those cases the new router might be asked to forward packets before it has a complete db. Might result in packets being dropped on the floor, or fallback to multicast NS to the hosts.

yes, it is the yet-to-be-invented that scares me.

> I want to verify that I understand the context. Are you saying that any time there are two or more routers on the link we can not use the new scheme and can only use 4861? That doesn't make sense - we must aim higher.

write it down.
might be my Scandinavian pessimism (which I assume you share), I'm not too hopeful on simple solutions here.
if you can assume the routers are wired, then you could run a routing protocol to advertise the bindings between themselves. sounds like we have a bar bof, napkin, pen and beer required for this part of the discussion. ;-)

> Sorry, I don't understand your comment nor how it relates to the part of my text that you quoted.
> Please explain.

same as above. maintaining synchronized state across all first-hop routes for sleepy nodes, needs some thought.
regardless of the wrapping (ND or DHCP) of the bits on the wire between host and router.

> Explain how a host which uses RFC 4941 address does not need changes.

it uses IA_TA. no change.

> No, it was actually a lot more about EUI-64 derived IIDs and the fact that some vendors had issued NICs which all had the same MAC addresses. Those rare situations were a pain to track down. Hence at or around the time of the 1st Stockholm IETF we came up with this single NUD probe.
> 
> Since then we have temporary addresses, CGA, stable-privacy etc  which depend on DAD being able to detect duplicates.

OK, that I got from Steve, and is probably the assignment type with the biggest probability of collision.

> Yes, but that doesn't work with the security and privacy model for stable-privacy and CGA and their likes. Hence a non-starter AFAICT.

privacy addresses are supported in DHCP.
I don't understand the requirement for supporting CGA, I think there have been proposals for CGA and DHCP.
we need to go back and understand the requirements. with the DHCP hint mechanism the hosts can in any case suggest a CGA address to use. for both efficient ND and DHCP the "network" is in control if which addresses the hosts are allowed to use.

> Default DAD attempts is one - not zero.

yes, of course. "might as well have been zero". ;-)

> But it isn't reliable in the case of random packet loss, spanning tree reconvergence triggering packet loss, sleeping hosts, etc, etc.

cheers,
Ole

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Wed, 26 Feb 2014 12:04:23 +0000

One option is to use new ethertype for ND packets and have the access-point provide downstream qos to this ethertype.   The hosts also filter ND traffic using the new ethertype.  

Hemant

-----Original Message-----
From: Pascal Thubert (pthubert) 
Sent: Wednesday, February 26, 2014 3:08 AM
To: Erik Nordmark
Cc: Andrew Yourtchenko (ayourtch); Hemant Singh (shemant); IETF IPv6
Subject: Re: Comments on draft-yourtchenko-colitti-nd-reduce-multicast

Pascal

---

From: Ralph Droms <[email-masked]>
Date: Wed, 26 Feb 2014 08:44:30 -0500

> On Feb 26, 2014, at 6:31 AM, Ole Troan <[email-masked]> wrote:
> 
> Erik,
> 

Or, the ND messages could be relayed to the DHCP server...

The relay could even snoop the assigned addresses (which is already done now [?] for other purposes.

> in DHCP as deployed today, you get address assignment. what addresses are assigned to hosts is defined in the address assignment policy on the DHCP server. just like on any other network that uses DHCP for address assignment
> 

So, legacy hosts can be assigned addresses w/o hints while registering hosts send in the hint to indicate address registration.

> right, maintaining this state is the crux of the problem. which is also the main reason why I'm skeptical about efficient ND.

...and DHCP already has tackled the problem of restoring state through leasequery.

>> I don't see how you can support legacy hosts without that. We can't assume that legacy hosts do DHCP address registrations for RFC 4941 temporary addresses when the A flag is set. Clearing the A f…

Hosts with no DHCP implementation at all?

> as above. legacy hosts get address assignment, hosts that care about self generating interface identifiers will get address registration (if allowed by network policy).

I agree...

> yes, it is the yet-to-be-invented that scares me.

There may be something to learn about the sync protocol from work on DHCP sync.  We thought it was pretty straightforward until we got to the problem of simultaneous updates between different host/server pairs.

> if you can assume the routers are wired, then you could run a routing protocol to advertise the bindings between themselves. sounds like we have a bar bof, napkin, pen and beer required for this pa…

Where and when?

> regardless of the wrapping (ND or DHCP) of the bits on the wire between host and router.

Yup.

>> Counting on you all to keep me honest ;-)

- Ralph

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 26 Feb 2014 08:10:41 -0800

> draft-ietf-dhc-dhcpv6-failover-design
Good to know.
> by moving the state into the network, maintaining that state on all first-hop router is going to be a challenge. that is the same problem, if you use DHCP or if you use ND.

The difference being that for ARO there is a draft which specifies all 
the details for how to do it.

With DHCP there is at best a requirements and design document. And those 
do not address state sharing between DHCP relay, which we need if we 
want a complete solution (re)using DHCP.

> you are quick to jump to conclusions. ;-)
Not conclusions - but assumed requirements that incremental deployment 
is critical otherwise protocols don't get deployed.

You said "declare that there is no legacy" which sounds like an approach 
where only green-field deployment makes sense.
That is a non-starter to me  and not worth while.
> I think we need to separate what _can_ be done within the protocol and how the protocol is deployed today.
There are definitely separable considerations around reuse - one is 
reusing existing protocols without extensions. Another is to avoid 
modifying existing implementations of said protocols. A third is not 
modifying existing deployment practices.
> is it really a requirement that hosts must generate addresses? if so why? what problem does that solve in this context?
Privacy. (which is why we did RFC 3041 and stable-privacy-addr).

But we've only concluded in this thread that there is no addition needed 
to the DHCP standard to register host generated addresses using IA_TA.

There is a host implementation issue AFAICT - at least in ISC DHC it is 
not possible to feed a host generated address into the DHC client - it 
can only request that the server to *create* and assign temporary 
addresses (by using the '-T' option). I don't have source access to 
other DHC client code to check.

But the hosts would presumably need to change for other reasons if we go 
the DHCP path. There is no need to multicast DAD probes in that case, 
and getting rid of those multicasts would be helpful.
> right, maintaining this state is the crux of the problem. which is also the main reason why I'm skeptical about efficient ND.
There is an internet draft for that.

I'm waiting for the similarely detailed draft for how to do this with 
DHCP ;-)

More seriously, I have spent the last day sketching on how this can be 
done reusing DHCP but it seems like either we need to invent  of new 
protocol mechanisms here and there, or we end up restricting things 
severely. The restrictions seem to be:
     - no support for DHCP relays - MUST have a DHCP server on every link
     - no support for redundant DHCP servers (until some standard DHC 
failover protocol has been specified and implemented)
     - can only support unmodified host implementations that have DHCP 
on the links that enable "efficient-dhcp"
     - can not support temporary addresses in unmodified hosts

I think we at least need to do better in terms of legacy host support.

> yes, it is the yet-to-be-invented that scares me.
I don't know if you intended that comment as a critizism of efficient-nd 
or a more general comment. Efficient-nd is fully specified and ready for 
review. Much of it has already been implemented and tested in the form 
of RFC 6775.

The DHCP state sharing is yet-to-be-invented.

>> I want to verify that I understand the context. Are you saying that any time there are two or more routers on the link we can not use the new scheme and can only use 4861? That doesn't make sense…
Huh? The above refers to your proposal in email about reusing DHCP. 
Seems like you should write down how you intended it to work so we can 
stop the guesswork.

> might be my Scandinavian pessimism (which I assume you share), I'm not too hopeful on simple solutions here.
If I ever had such pessimism I must have left it behind a long time ago.
It is not the decision to move forward to solve an important problem 
which is the issue - it is to work out the technical details that takes 
work - need to ensure that the protocol is robust, incrementally 
deployable,  introducing minimal complexity, etc.
> if you can assume the routers are wired, then you could run a routing protocol to advertise the bindings between themselves. sounds like we have a bar bof, napkin, pen and beer required for this pa…
See above - the devil is in the details.

> it uses IA_TA. no change.
Correct - no change to the DHC options, nor server. But the DHC client 
code and behavior needs to change. Needs to detect a duplicate (which 
the client can only tell by comparing the IA_TA it got from the server 
with what it sent in the IA_TA.) We probably also want to clarify that a 
server should honor the suggested IA_TA addresses unless it is a duplicate.

   Erik

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 26 Feb 2014 07:32:53 -0800

> Certainly NUD should be higher priority than traffic?
I don't think the question has come up in the past.
No harm in using a different TClass for Neighbor Discovery and/or 
different L2 priority (e.g., 802.1p).

But it might not make much of a difference if the main point where the 
priority matters is on the receiving host or router when it passes the 
packet to the CPU. For that the local implementation-specific mechanisms 
can presumably determine that it is an ND packet and queue/process it 
appropriately.

RFC 6583 talks about even finer grain prioritization for ND traffic - 
without any use of TClass or 802.1p.

Regards,
    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 26 Feb 2014 12:02:12 -0800

> Or, the ND messages could be relayed to the DHCP server...
Ralph,

To support DHCP relays with off-link servers and handle legacy hosts 
doing DAD, then we need a relay-to-server extension. One way to do that 
would be to relay the ND messages. But we might need some additional 
info with those. At least we'd want a response with "OK" vs. "duplicate" 
from the server, which doesn't exist in today's ND. And we'd want to 
have DUIDs for the legacy hosts to fit in with what the server does - 
might make sense creating the MAC address-based DUIDs on the relay and 
passing them along to the server.

Supporting relays is solvable - but it requires some new protocol 
mechanisms.
> The relay could even snoop the assigned addresses (which is already done now [?] for other purposes.
The relay would be required to build "registered NCEs" based on the 
assigned/verified addresses from the server, and use those plus the 
associated MAC addresses in the forwarding plane. (And somehow ensure 
that other routers on the link have the same info.)
> So, legacy hosts can be assigned addresses w/o hints while registering hosts send in the hint to indicate address registration.
Yep - just requires code change on the hosts.
> ...and DHCP already has tackled the problem of restoring state through leasequery.
But here the state needs to to be distributed differently. The servers 
(plural) and all the routers (which might be all the DHCP relays) need 
to have consistent state. The leasequery mechanism allows a relay to 
rebuild its gleaned state from a server.

Would we be comfortable mandating that all the routers on a link also be 
DHCP relays, so they can use leasequery?
> Hosts with no DHCP implementation at all?
Even if they host implementation contains DHCP, the DHC client code 
(like ISC's) doesn't have a way to request specific addresses (but has 
the ability to renew IA_TA addresses already part of a lease).
Unless other DHC client code has very different support, I don't see how 
deployed host code can handle RFC 4941 addresses using DHCP.

> There may be something to learn about the sync protocol from work on DHCP sync.  We thought it was pretty straightforward until we got to the problem of simultaneous updates between different host/…

For address registration it would be better to look at software like 
Zookeeper which does quorum election. The would outside of the IETF 
isn't about simple active/standby or active/active network elements.

Regards,
    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 26 Feb 2014 11:31:58 -0800

> draft-ietf-dhc-dhcpv6-failover-design

Looking at those docs, they design seems to focus on address assignment 
using a pair of servers. For instance,

    A subset of available resources (addresses or prefixes) is reserved
    for secondary server use.  This is required for handling a case where
    both servers are able to communicate with clients, but unable to
    communicate with each other.

That is insufficient for address registration, since AR assumes it 
should work unless somebody else has registered the address. Thus this 
would typically require a quorum (2 out of 3, 3 out of 5, etc) so ensure 
that there can be no duplicate registrations when some of the servers 
are down or unreachable.

Thus address registration is subtle different than address assignment, 
and the design center for DHCP is the latter.
(But still investigating what can be done using DHCP.)

    Erik

---

From: Mark ZZZ Smith <[email-masked]>
Date: Wed, 26 Feb 2014 12:52:25 -0800 (PST)

Hi Erik,

----- Original Message -----
> To: Mark ZZZ Smith <[email-masked]>; Ole Troan <[email-masked]>
> Sent: Tuesday, 25 February 2014 10:10 PM
> of

I agree. The DAD proxy is how the DAD issue is dealt with in broadband N:1 / hub-and-spoke etc. link-layer forwarding models when hosts/CPE are layer 2 isolated from each other, and can only send traffic to other on-link destinations via the default router (performing the DAD proxy function)

>> 
>>  More broadly though, I think a method that supports any forms of multicast 
> from a host to the other hosts on the link, hair-pinned through the upstream 
> router(s), would be better. It would support DAD operation, and would also 
> support other multicasts from hosts, such as Multicast DNS for device (printer) 
> discovery, and SSDP for DLNA server discovery.

Yes. 

> router (e.g., the AP) if you want to ensure that all receivers can get a 

Yes, layer 2 split horizon is preserved. It is the default router that forwards traffic to all destinations, even those that fall within the prefixes from which the origin host has its addresses.

> decrements ttl then it would be safer to send the multicast packet out 

The router would perform standard IPv6 forwarding, including TTL decrement. Although Link-Locals currently won't use their default gateway to reach other Link-Local destinations, if that was changed, I think it is quite legal for a router to forward a link-local packet back onto the same link (with a TTL decrement) - RFC4291 only precludes them being forwarded onto a different link. 

> (and ND isn't the only protocol which does this.)

That is a good point, one I hadn't considered. I'd be interested to know which non-ND protocols use the TTL=255 trick if you know them off of the top of your head. We might be lucky, the other ones which do use the TTL=255 trick might be ones that for security purposes we wouldn't want the default router to forward between hosts attached to the same link anyway. 

The use of the TTL=255 trick for ND would actually be useful for naturally preventing rogue RAs from the hosts - if the default router forwarded them to other hosts on the link, decrementing the TTL to 254, the receiving hosts should then ignore them.

Regards,
Mark.

> Regards,
> 
>     Erik
> 
>>  I've been thinking about this scenario for quite a while, and it seems 
> to me that there are fundamentally two approaches to this problem.
>> 
>>  The first one involves minimal or no changes to the hosts (e.g., an 
> optional RA PIO for link-local flagged as off-link and that is it, as a host 
> local optimisation instead of ND proxy for link-local addresses), and emulate 
> functions at the upstream router(s) that won't work because of host to host 
> isolation within the link.
>> 
>>  The other one involves wholesale changes to the hosts, and I think that 
> would mean fully implementing an IPv6 NBMA subnet, perhaps with an RA option to 
> announce to the hosts that the link is operating in NBMA mode, and the 
> address(es) of the MARS cluster.
>> 
>>  The NBMA method would be convenient, as all these sorts of 
> multicast/link-local etc. issues are already dealt with. However, I think 
> wholesale changes to hosts to support it are probably infeasible.
>> 
>> 
>>  Regards,
>>  Mark.
>> 
>> 
>> 
>> 
>>>       Erik
>>> 
>>>

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 27 Feb 2014 20:12:51 +0100

On 2/26/14 9:52 PM, Mark ZZZ Smith wrote:
> The router would perform standard IPv6 forwarding, including TTL 
> decrement. Although Link-Locals currently won't use their default 
> gateway to reach other Link-Local destinations, if that was changed, I 
> think it is quite legal for a router to forward a link-local packet 
> back onto the same link (with a TTL decrement) - RFC4291 only 
> precludes them being forwarded onto a different link. 
That is correct - link locals can be forwarded out the same link.
I don't know if that is always a good idea for multicast packets - but 
if the router received the L3 multicast packet as a L2 unicast then it 
would make lots of sense to forward it back out on the link.
> That is a good point, one I hadn't considered. I'd be interested to know which non-ND protocols use the TTL=255 trick if you know them off of the top of your head. We might be lucky, the other ones…
I haven't tried to compile a list.
I know that routing protocols and VRRP does this (which is a reason why 
a "send link-locals to default router" should not be applied to the 
routers themselves.)

It doesn't look like mDNS uses it according to the RFC. It would be good 
to check this though.

> The use of the TTL=255 trick for ND would actually be useful for naturally preventing rogue RAs from the hosts - if the default router forwarded them to other hosts on the link, decrementing the TT…
Yes, for RAs it would help.
But if the hosts still send multicast NSs (e.g., for link-local 
targets), then the ttl decrement ensures that they will never get a 
response.

    Erik

---

From: Andrew Yourtchenko <[email-masked]>
Date: Fri, 28 Feb 2014 16:25:51 +0100

On Tue, 25 Feb 2014, Erik Nordmark wrote:

>> alread. The reason I did not include it yet, was to go over the tradeoffs 
>> without the pressure of the submission deadline being a few hours away :)
>> 
>> 3) There are a few trickier scenarios with L3 roaming (hosts arriving from 
>> other subnets onto the same AP) and that AP having a single 802.11 group 
>> encryption key, which this behavior might make easier to accidentally have 
>> broken.
> I don't quite understand concern #3 (and #2). Since the routers doesn't know 
> whether hosts rely on unsolicited multicast RAs, the multicast RAs must 
> function. Thus adding the flash crowd optimization to multicast solicited RAs 
> wouldn't be anything new or different - it would merely appear in one 
> additional case.

My worry is more about the humans than the hosts :-)

char *foo = NULL;
if(unlikely(event)) {
   // Almost never happens
   *foo = 'x';
} else {
   // Almost always happens
}

Is a strawman bug which will be harder to catch just because it is 
triggered in rare circumstances.

That said, since the router/wireless infra will have to ratelimit ingress 
RSs anyway, they might as well send a multicast RA at that point.

>>> (and NS in general unless you also do section 4.7)? But SAVI doesn't claim 
>>> to have all addresses since it is concerned with conflicts. Thus a host 
>>> can exist and be silent - but DAD would still expect to reach it. Or a 
>>> host could have moved to a different port and SAVI having stale state - 
>>> yet DAD should work. My point is that the details of how the state is 1) 
>>> maintained and 2) used to forward packets are key to analyzing how the 
>>> neighbor discovery functionality and robustness would be affected by this 
>>> idea.
>> 

> I guess my first question was how this would work when there is no SAVI-like 
> state for an address. (In that case you don't have a unicast address to which 
> you can send the packet.) Would you multicast or drop?

My instinct is to say "multicast" since that will provide the 
backwards-compatible recovery.

But I think it can be more nuanced. Piggybacking on the resolicit idea 
from this thread below, if the gateway keeps track of the counts of 
"vanilla" and "resolicit-capable" hosts over time, it might be able to 
make an informed decision. If it saw a legacy host within a certain 
timeframe in the past (I am assuming that it would always see the router 
solicitation), then it would err on the side of multicasting, else if 
there are no vanilla hosts and the router did not clean up any of the 
neighbors within a router lifetime, there's no point to multicast.

>> access those, the remote party will need to know their address... so, if a 
>> device does some sort of service registration upon boot-up, then it will 
>> not be a quiet device anymore.
> But the service registration could have happened when the device was first 
> installed - I don't think that is likely to happen each time the device wakes 
> up.

True, it actually should *not* happen.

So the failure scenario is:

1) the host does behave according to today's specs
2) the host is providing a service for the remote hosts
3) the router has deleted the neighbor entry for that host
4) the host does not call home to recreate this entry
5) the router has decided not to multicast the NS to resolve an address this host

It seems like a robust implementation can fairly easily avoid the (3) and 
(4) - whereas a router can try to be more conservative about (5).

> improvements we do to the standards in this space to be useful for the next 
> 20 years or so. (Products, whether end devices or routers/switches can 
> operate on shorter time scales and with more limited applicability than the 
> core standards.)

Indeed. Another aspect is to try to make improvements in such a way that 
they are not required on two sides at once to be useful - this should 
help with their adoption.

>
>> 
>> I do agree that there are limitations - but I think it is very important to 
>> collect the data about the scope of these limitations, to keep a balanced 
>> approach.
>> 
>> The data I have as of today based on 5-6 networks with 10-20K hosts says it 
>> is quite minimal. But I reserve the right to be wrong and thus would be 
>> (side note: if these sleepy hosts are quiet and are concerned that their 
>> address be known, they might consider to use DHCPv6 as well).

> Using DHCPv6 doesn't reduce multicast (neither for address resolution nor for 
> dad). One could change DHCP to record the link-layer addresses so that the 
> routers would ask the DHCP servers for the link-layer addresses, but that 
> seems like a lot of change.
>
> Also, see my response to Ole on the other ways that the DHCPv6 we currently 
> have doesn't help solve the problems at hand. [I see others brought up the 
> same concerns around DHCPv6 on the list.]

yeah. I won't fork the discussion and will see if I can contribute 
anything within that thread :-)

>>> router/AP, but that might result in host movement looking like a duplicate 
>>> address - again depends on the details.)
>> 
>> attached clients A and B during the ND saw the packet sequences that were 
>> close enough to the ND spec to achieve identical practical result and be 
>> compliant to the letter of it, yet allowed to perform the process more 
>> optimally from the broader perspective.
>> 

> Yes, because it depending on the details it might break SeND and other ND
> options, or it might only be robust and useful in very limited applicability 
> for instance in specific topologies (such as a single wireless controller).

That'd be a bug for that implementation to fix.

Arguably having a spec in this particular case might do more harm that 
good - if there is a bug in the spec which an implementation follows, then 
first a spec has to be fixed, then the code has to be fixed.

>>> have a fixed (set of) link-local address(es) for the router(s), having a 
>>> high default lifetime means that after a failure the hosts would have one 
>>> entry in the default router list which us unreachable - until that high 
>>> lifetime expires. I don't know if there has been a study on the 
>>> performance impact of that would have on the hosts e.g., how often they 
>>> would re-probe the default router.
>> 
>> This is indeed a useful consideration that is worth noting in a doc that 
>> would be removing the 9000 seconds max limit. Mind if sketch a -00 on this 
>> matter and send you the possible text ?
> That would be excellent.

Ok, I'll sketch it today/on weekend and send unicast.

>>> default router would always be the same. Ditto for networks with a single 
>>> point of failure single router at a fixed address (e.g., if the router is 
>>> always at fe80::1 or some other fixed address.) Thus I think we should 
>>> recommend 4.5 and 4.6 that within that applicability. Added benefit is 
>>> that the routers control it, hence the operator of the network can set the 
>>> values higher for VRRP or single router cases.
>> 
>> I totally agree! We probably might have a series of documents for different 
>> types of use cases - this could be very helpful for the folks deploying 
>> them. We could turn this doc into a "large-scale high-density WiFi 
>> (stadiums, exhibitions, campuses, etc.)" and then have couple of others 
>> describing other types of deployments. Let's discuss this in London ?
> Yep.

>>> draft. Might even be best to have a separate draft on this radical idea so 
>>> it can be explored fully.
>> 
> 5942 re-states the intent of 4861 with more detail. But doesn't change the 
> above semantics. (It does remove these two bullets from on-link:
> The only case I know of where "blocking link-locals" is part of our standards 
> is for multi-link subnets, for instance in 6lowpan. But in that case it was 
> part of the architecture from the start. The issue we have with the general 
> ND evolution is that we need to support the existing assumptions, and many of 
> those assumptions come from IPv4. For instance, the use and semantics of an 
> IPv4 link-local multicast has been carried to IPv6 link-locals.
>

Forward-referencing to [*] below.

>> 
>> My thought process was as follows: suppose we have 9000 people, each with a 
>> smartphone, wandering inside a large building or on campus. We want to 
>> predictability we can use a first byte of a hash of the host's mac address.
>> 
>> place each of the hosts into the correct VLAN on each AP. A typical network 
>> of such a scale will have about 300 APs. I still need to send RAs within 
>> each vlan. I need to ensure that the hosts do not receive the "wrong" RAs. 
>> (and remember, there is not really a way in a naive 802.11 imlementation to 
>> do "selective" multicast).
>> 
> For those types of deployments having /127 subnets (plus DHCPv6 PD if someone 
> wants a personal area network behind their phone) makes a lot of sense.
>
> But my issue is that the goals we are trying to achieve are more general than 
> the stadium (or broadband subscriber) use case.
>

Agreed - though I think it might make sense to more explictly document the 
use case and see what applies still ?

e.g.: say we have sensors on a factory floor. Those sensibly will sit on a 
different SSID compared to all the chatty smartphones worn by the workers; 
and given they will be much more "well-behaved", we may not need to be as 
aggressive in suppressing the traffic between the hosts.

Given we are concerned about their energy consumption, most probably the 
direct access to them will be restricted (else one can just ping -f and 
drain the battery) - so that traffic will be quite controlled.

This is a wildly different use case from a high-density BYOD where we 
essentially have a swarm of almost hostile devices outside of our 
control right in the middle of the network.

>> Of course, I need to disclaim: doing this in the home network with one AP 
>> and < 100 devices all belonging to a single person is not useful at all.
>> 
>> Now that we've cut off all the services, we need to figure out how to bring 
>> them back... And I think the further mental construct here that would be 
>> useful is to consider this as a giant homenet with a lot of guests. Thus, I 
>> think from the services discovery standpoint, the products of the DNS-SD 
>> workgroup will be very beneficial here.

> implementation, or various to us unknown and proprietary approaches makes 
> sense for a retrofit.

[*]

Talking specifically mDNS, it breaks itself as we increase the # of 
hosts - whether we keep the multicast on or not - we just get to pick the 
way it breaks:

1) we keep the multicast - so every phone gets the 10000s of 
advertisements, can not make any use of them anyway, only drains the 
battery. (from my experiments, the # of services usable within the UI is 
limited)

2) we get rid of the multicast - and break the discovery upfront in a 
predictable way, declaring it within the AUP for the network.

(Aside: I did not mention it in the doc because I wanted to keep it 
focused on the ND, but in the high-density environments I specifically 
block udp/5353 multicast in its entirety. This does not have to be this 
way, and grouping the hosts into "broadcast domains" based on some 
criterion might be a different approach, but it's nothing implements 
this, to the best of my knowledge).

>
> Only makes sense for a greenfield deployment where there are no assumptions 
> that existing protocols and code continue to work.

Above, we are discussing the use cases different from HD WiFi that do not 
exist today. I count those as green field.

>>> From a deployment perspective DHCPv6 is available for address assignment, 
>>> but don't think we want to require that for WiFi or other links which have 
>>> packet loss.
>> 
>> This is more of a fallback scenario for those who want a 100% guarantee of 
>> address uniqueness in the network - using the existing mechanism.

> But RFC 3315 doesn't guarantee uniqueness, which is why the host needs to do 
> perform DAD in addition.

I need to correct myself. "Those who want a bigger guarantee".

No mechanism gives a 100% guarantee. We asymptotically approach 100% to 
varying degrees.

>> that DAD works in that scenario ? Or is it worth fixing the connectivity by 
>> bringing the loss down to acceptable level ? At how many retransmits do we 
>> declare a failure ? Could we then explore a similar approach to make the 
>> DAD more robust instead - keeping in mind the probability of failure should 
>> be quite low ?
>> 

> If we want a DAD probe mechanism that is more robust to failure then I think 
> we should just reuse the techniques in ACD (RFC 5227) which performs ongoing 
> conflict detection. However, that results in or broadcast/multicast messages!

For the 802.11 case we can have a reasonable assurance that there is no 
duplication of the MAC address (else a bunch of L2 stuff would have been 
broken). So then the question of duplicates reduces to detecting the cases 
where the existing MAC<->IP mapping does not hold. If both of the hosts 
involved into collision are active, the router should be able to notice 
this just from the unicast traffic ?

>
> If we want both less multicast packets *and* a more robust DAD (including 
> efficiently handling DAD for sleeping nodes), then I don't see any approach 
> other than making some devices on the link be able to speak more 
> authoritatively about the addresses present on the link. Those devices can 
> try to build that state implicitly (gleaning from packets on the link), or 
> explicitly. (I'll try to capture the differences between those before next 
> week.)

The DAD also has the explicit function of detecting the duplicate MAC 
addresses. If we omit this function, I think the process can be simplified 
by quite a bit ?

Duplicate MAC addresses are an interesting corner case - but forcing 
everyone to pay the tax for this corner case might be something worth 
reevaluating ?

>>> multicasting a
>> But I certainly know my gadgets stop being reachable if I do not touch them 
>> for a day"
> But there is also a question about the efficiency when the device wakes up. 
> Whether it needs to start from scratch (multicast RS, do DAD multicasts and 
> wait for a second), or whether it can do DNA (unicast a NS to the router(s) 
> to check it is on the same link) and avoid waiting for the lack of a response 
> to a DAD probe.

If the DNA succeeds - therefore we're on the same link, then we have some 
memory of it ?

Some more on this at [**] below.

Also, the DNA with a unicast NS makes me wonder - can the host achieve 
a similar function by sending a unicast RS *if* it remembers that the 
router earlier was sending the solicited RAs unicast ?

(Treat this as a thought experiment, I realize that I'm probably trying
to rewrite a dozen specs with one sentence ;-)

> Need to bring a whiteboard to London ... ;-)

I'll get a pack of napkins! :-)

> Or add a flag to the RS to say "I will re-solicit" ...

Yes. I tried to avoid having an explicit field - since the overall system 
should converge within 1 router lifetime anyway.

> I can see how you increment those counters (based on seeing a RS with a new 
> MAC address.)

Assuming no explicit field: you increment the "legacy" counter upon the 
multicast RS from a host you do not have a neighbor entry for. Then you 
decrement the legacy counter and increment the "resolicit-capable" counter 
upon seeing a unicast RS.

> (I don't think NUD from the router can be used for this, because the host can 
> be unreachable from the router for a few seconds due to radio issues, yet the
> host will still consider itself connected and not restart by sending an RS.)

Yes, I was counting this particular case as a "NUD is impatient" type of 
situation which should not happen :-).

But I think this is fixable relatively easily by having a single timer of 
"legacy quiet hosts sleepy interval": if you lost all legacy hosts, you wind up a timer 
and behave in a legacy fashion until it expires. If the legacy hosts 
reappear before this interval expires, you reset this timer.

I think the very same logic could govern the decision "to multicast or to 
drop?" above, as well... - if no legacy hosts and timer expired - drop, 
else multicast.

The value for the timer is probably something that would need to be set 
administratively. The bonus is that the default value can be "infinity" - 
that is, always send multicast RAs => today's behavior.

> retransmissions (no RA after 1 second, then resend RS). Another way is to 
> space the "retransmissions" inside the lifetime range e.g., by having the 
> host send a RS when 1/3 of the router lifetime has passed, next when 2/3 has 
> passed. (Generalize to k/N for N transmissions before giving up.)

Yes, indeed. I chose the "start at time T with retranstmits + exponential 
backoff" because this seems to take care better of a wider range of the 
possible loss on the medium: i.e. if you just space 3 retransmissions 
within the lifetime, as the loss increases, the chances to get the RA 
decrease. The exponentially decayed retransmits do not seem to have this 
problem - they autotune. They might create a denser burst in case of the 
loss though.

>> But they are solvable: Assuming even the existing router lifetime limit of 
>> 2.5 hours, and RA interval of 1.5 hours, we have a space +- 0.5 hour of 
>> jitter.
>> 
>> of by temporarily going into "legacy only" mode for a period of a couple of 
>> RA lifetimes.

> The router is free to just discard the NCEs - unless we change DAD to use 
> them in some proxy approach.
> Even if we change DAD that way, the router can just do the SAVI thing (send a 
> unicast NS to the host and see if it still there) - no need to multicast RAs 
> to clean up.
> However, the unicast NS cleanup (or any other cleanup driven by the router 
> expecting the host to respond) has issues with sleeping hosts. (I just 
> realize their might be different forms of sleep - completely off and will 
> take up based on a timer, or being woken up by arriving (unicast) packets. I 
> don't know if there is confusion around that and whether we need different 
> terms to make it more clear.)

I think the router might handle them in a FIFO fashion: keep them while 
it has enough space, and start purging them if there is a growing 
contention.

I think the timer idea I wrote about above, should take care of this case.

>> 
>> Of course, it requires the sleepy hosts to wake up every now and then. But 
>> with a lifted router lifetime limit, it will be every 9 hours.

yup.

> wakeups, but with very short runtime. I don't know if that was captured in 
> RFC 6574 or in Margarets slides from the workshop. In any case, one issue I 
> remember is that for Ipv4 the runtime was a lot less than one second, but 
> with IPv6 there was an additional 1 second to wait for DAD to complete which 
> blew the power budget.

This is interesting, and completely un-discussed so far aspect - 
battery-life impacting which does not have much to do with multicast ! So, 
essentially we have to wait 1 extra second before we can use the address.

Assuming it is indeed a problem, notice that this is a problem for the 
router-less segment as well => therefore should be addressed in the hosts 
themselves independently off the network. (i.e.: assuming a host 
self-assigns an IPv6 link-local address, probably the devices are 
*already* paying the tax on it, even in IPv4-only networks!)

I think we should square it into a separate corner and attack there :-)
(probably another draft?)

Couple of approaches for further discussion:

1) the host does not have to *wait wide awake* to receive the packet - 
since the reply will awake it equally. So it is a matter of more careful 
host programming to send a DAD NS, go back to a nap, and then awake either 
when NA is received or when 1 second expires.

2) investigate a different approach to a "1 second" timeout, treating it 
not as a fixed value, but as an envelope: at zero your chance 
of performing successful DAD is zero; then somewhere at around the 
inter-host RTT+sigma the usefulness of it is maximized, then further 
increase of the interval does not increase its usefulness much. [**] So, 
supposedly, the hosts that wake up and detect that they are on the same 
link, might use the memory about the network characteristics to shorten 
this interval.

--a

>> Let's assume for simplicity that we wake up every 8 hours, so this gives 60 
>> seconds per day. This gives us the active life span of 490 days.
>> 
>> asleep - so the lifetime would certainly vary - but I think this shows that 
>> this is not a totally unreasonable path.
>> 
>> --a
>> 
>>> 
>>> Regards,
>>>    Erik
>>> 
>>> 
>>> 
>> 
>

---

From: Andrew Yourtchenko <[email-masked]>
Date: Fri, 28 Feb 2014 16:26:35 +0100

Erik,

On Tue, 25 Feb 2014, Erik Nordmark wrote:

>> whereas the effective multicast capacity is 1*1Mbps, therefore the 
>> difference in throughput is 1000-fold).
>> 
>> So, now we have a situation of 10000 hosts which have deleted their default 
>> gateway from the neighbor table, and send the multicast neighbor 
>> solicitations for it.
>> 
>> Assume the NS is 64 bytes, 10000 hosts sending such a packet means 64 bytes 
>> * 8 bit * 10000 hosts = 5120000 bits/sec - or, 5 mbps.

> retransmission time is 1 second. Hence with 4861 we get 5 mbps for 3 seconds. 
> However, they hosts aren't that likely to all discover the NUD failure in the 
> same 3 second window. With ReachableTime=30 seconds it depends on when there 
> last successful NUD probe or higher-level reachability advise can in. Hmm - 
> with TCP the reachability advise might come with every left window edge 
> advancement i.e. they would synchronize quite closely.

Yes, it's a bit of a fringe assumption here of all the hosts being active 
and then losing the connectivity, and somewhat uniform implementations of 
TCP (or, stuff like name resolutions). In reality I think they will be 
spaced more.

>
>> 
>> Note, that this is only the shared bandwidth downstream back to the hosts - 
>> the airtime spent by the upstream traffic is 20x faster, so I am 
> resolution is that the router receives it and responds. Thus why wouldn't 
> that part work?
> The NA from the router will be unicast so if that unicast isn't dropped due 
> to the multicasts then it should resolve quickly. I don't know if unicast is 
> prioritized higher than multicast or not.
>
> (We'll see downstream overload due to the multicasts, but that might just 
> last for 1 second if the NS/NA to/from the router gets through.)

Yes, since the router is on the wired, the 
bandwidth degradation due to a "dummy" multicast retransmission is the 
biggest problem. The NS will go unicast over the air, then go to the wired 
+ multicast over the air, and the unicast reply from the default 
gateway on the wired will go over unicast.

>> This can shrink the available capacity several times. Add to this that the 
>> host does not necessarily stream at 100kbps, but might have higher data 
>> rates, and I think we can consider the available
> unreachable for a few seconds), then the stream would also slow down due to 
> TCP, in which case there is less of a multicast overload issue. There might 
> be cases where this TCP slowdown doesn't happen though.

Yes, indeed. Also, given the high utilisation of the media at that point, 
I wonder if the cross-channel interference will play a bigger role, 
therefore affecting the results.

This seems like an extremely tricky beast to model with any reasonable 
accuracy!

--a

>
>  Erik
>
>> 
>> This is where the things may become interesting - 1/10th of the capacity 
>> means that it will take not 5, but 50 seconds for all the hosts to recover.
>> 
>> these packets are lost, the hosts might back into the pool of the "orphans" 
>> who are sending the multicast, because they delete their neighbor entry.
>> 
>> default gateway might cause rate-limiting of the control plane packets on 
>> the gateway. With some of the limits being as low as 1000pps, this might 
>> give a recovery time on the order of minutes even without the wireless 
>> multicast being the bottleneck, yet still resulting in a lot of multicast 
>> NS in the air still during the slow recovery.
>> 
>> suitable model and playing with the parameters to see where it breaks seems 
>> like a useful exercise to understand how many hosts can there be in a 
>> single /64 on WiFi with a "naive" set of access-points.
>> 
>> Thoughts ?
>> 
>> --a
>> 
>