Subject: A common problem with SLAAC in "renumbering" scenarios

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 08:00:21 -0300

Folks,

We have posted a new I-D discussing a problem that can arise in typical
deployment scenarios where the CPE obtains a prefix via DHCPv6-PD and
advertises a prefix on the LAN side.

Our draft is available at:
https://tools.ietf.org/html/draft-gont-6man-slaac-renum

The Abstract is:
   A very common IPv6 deployment scenario is that in which a CPE employs
   DHCPv6 Prefix Delegation to obtain an IPv6 prefix, and at least one
   prefix from within the leased prefix is advertised on a local network
   via SLAAC.  In scenarios where e.g. the CPE crashes and reboots,
   nodes on the local network continue using outdated prefixes which
   result in connectivity problems.  This document analyzes this problem
   scenario, and proposes workarounds.

Any comments will be welcome.

Thanks!

Cheers,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 31 Jan 2019 13:40:51 +0200

Isn't that handled by L-13 of RFC7084?

L-13:  If the delegated prefix changes, i.e., the current prefix is
          replaced with a new prefix without any overlapping time
          period, then the IPv6 CE router MUST immediately advertise the
          old prefix with a Preferred Lifetime of zero and a Valid
          Lifetime of either a) zero or b) the lower of the current
          Valid Lifetime and two hours (which must be decremented in
          real time) in a Router Advertisement message as described in
          Section 5.5.3, (e) of [RFC4862].

--
Tassos

---

From: "Bernie Volz (volz)" <[email-masked]>
Date: Thu, 31 Jan 2019 11:44:01 +0000

Perhaps it is a non issue, but what happens if multiple RAs are sent as there is too much in one to send as a single RA (given MTU and avoiding fragmentation)? And while perhaps for the target market this isn’t likely, as hosts may need to operate in more complex networks ...

- Bernie

> On Jan 31, 2019, at 6:01 AM, Fernando Gont <[email-masked]> wrote:
> 
> Folks,
> 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 31 Jan 2019 12:47:38 +0100 (CET)

> Any comments will be welcome.

"   o  A CPE MUST record, on stable storage, the list of prefixes being
       advertised on each LAN segment."

This is not always doable as some devices have very small flash with no 
write-leveling, so doing this generally might ruin the flash cells on some 
devices. This above requirement will drive cost, especially in the low end 
market of HGWs.

       *  Any prefixes that were previously advertised via SLAAC, but
          that are not currently intended for address configuration, MUST
          be advertised with a PIO option with the "A" bit set to 1 and
          the "Valid Lifetime" and a "Preferred Lifetime" set to 0.

Doesn't RFC7084 already say this? L-13.

Another comment, typically the lifetime of the LAN PIO is capped at the 
DHCPv6-PD WAN lease time. I couldn't find this requirement in RFC7084, but 
I only searched for "lifetime". Anyhow, Openwrt does this and might be 
good to mention.

When the new prefix is received it'll most likely have a higher preferred 
and valid lifetime, so hosts should use the new prefix just by means of 
them preferring the higher preferred lifetime of that PIO. So the problem 
is a bit less than you write in the draft.

So while I am generally sympathetic to this draft and what it tries to 
achieve (especially the part where it ties the router announcing something 
to what it's later announcing, and lack of something means it's implicitly 
zero preferred time for that), we need to figure out a few more things 
first. RFC7084 was informational which made it fairly easy to get through, 
now you're proposing a standards track document so we need to make sure 
everything works. Changing all hosts is a big thing.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Thu, 31 Jan 2019 12:47:39 +0100

The problem is that if the router has been rebooted (example, power outage), then it doesn’t know what was the previous or new prefix (if it has changed).

So, even if the router follows RFC7084, it will not be able to “recognize” the change, unless the prefixes are “stored” in flash or other stable storage.

I’ve seen only one vendor doing that.

Regards,

Jordi

De: ipv6 <[email-masked]> en nombre de Tassos Chatzithomaoglou <[email-masked]>
Organización: Forthnet
Fecha: jueves, 31 de enero de 2019, 12:41
Para: Fernando Gont <[email-masked]>, "[email-masked]" <[email-masked]>, IPv6 Operations <[email-masked]>
Asunto: Re: [v6ops] A common problem with SLAAC in "renumbering" scenarios

 

The Abstract is:

Thanks!
 
Cheers,

-------------------------------------------------------------------- IETF IPv6 working group mailing list [email-masked] Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6 -------------------------------------------------------------------- 

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Thu, 31 Jan 2019 12:57:06 +0100

Exactly, this is the reason why 99% of the CEs don't save this data in the flash, because it is prone to creating troubles and bringing the CE to a "brick" state.

I've seen that several times, having for example statistics written in the internal router flash, easily will break the router, or at least needed a factory reset or something similar. Alternatively using a "good quality" USB stick to save this kind of data, avoids the problem (most of the time, because from time to time I've got also good USBs being damaged after some months of usage).

I think for example, RIPE Atlas get the same problem from time to time, so the USBs get wrong. Sometimes reformatting is sufficient, but not always.

Mandating that in low cost CEs will mean CEs are no longer "low cost", unless new "safe" opensource ways to keep the flash are available and can be included in new firmware releases.

Regards,
Jordi

---

From: Sander Steffann <[email-masked]>
Date: Thu, 31 Jan 2019 13:04:16 +0100

> This is not always doable as some devices have very small flash with no write-leveling, so doing this generally might ruin the flash cells on some devices. This above requirement will drive cost, especially in the low end market of HGWs.

A flash cell is at least capable of 1000 to 3000 erase/write cycles. Even if the prefix would change every single day that gives a lifetime of 3 to 9 years. And prefixes shouldn't change that often anyway. When looking at more realistic estimates line a change on average once a week you'l get to 20 years. Seems more than enough for low end extra cheap devices. I don't see this driving cost up, except for spending a few hours on properly engineering this.

Cheers,
Sander

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 09:11:06 -0300

On 31/1/19 08:47, Mikael Abrahamsson wrote:
> write-leveling, so doing this generally might ruin the flash cells on
> some devices. This above requirement will drive cost, especially in the
> low end market of HGWs.

I agree 100% percent, for exact reason you point out. In fact, we
discussed whether making this a SHOULD, MUST, or not having any
normative language about this one at all. Our reasoning was that CPEs
not implementing this would simply not claim compliance with this spec.

Me, I agree with you that CPEs wouldn't do this in practice, and also
that the place to solve this is on the host side -- for robustness reasons.

Any proposal on how to address your comment? -- Include the reasoning
above? s/MUST/SHOULD/? Remove any normative language for this one?

> Doesn't RFC7084 already say this? L-13.

Yes, and we missed this (thanks!). -- that said, we added this bullet
for completeness sake. The case we care about in this doc is the reboot
scenario.

> DHCPv6-PD WAN lease time. I couldn't find this requirement in RFC7084,
> but I only searched for "lifetime". Anyhow, Openwrt does this and might
> be good to mention.

Very good point. Will do in the next rev.

> When the new prefix is received it'll most likely have a higher
> preferred and valid lifetime, so hosts should use the new prefix just by
> means of them preferring the higher preferred lifetime of that PIO. So
> the problem is a bit less than you write in the draft.

Nothing in the spec says preference is related to the "Preferred
Lifetime" timer. i.e., an address is preferred, or it is not...

FWIW, this problem came to my radar as a result of an ISP asking for
help on how to address this.

> achieve (especially the part where it ties the router announcing
> something to what it's later announcing, and lack of something means
> it's implicitly zero preferred time for that), we need to figure out a
> few more things first. RFC7084 was informational which made it fairly
> easy to get through, now you're proposing a standards track document so
> we need to make sure everything works. Changing all hosts is a big thing.

FWIW, I personally think the way to go is the host-side fix. You do want
hosts to get smarter about this, and obviously cannot rely on the CPE to
solve this for you.

While one might get away with an Informational document that specifies
this -- v6ops has done this for RA-related stuff, I think the best and
proper way to do it is with a std track document.

But, of course, the point of posting the I-D is to see what other folks
think....

Thanks!

Cheers,

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 31 Jan 2019 13:12:31 +0100 (CET)

On Thu, 31 Jan 2019, Sander Steffann wrote:

> A flash cell is at least capable of 1000 to 3000 erase/write cycles. 
> Even if the prefix would change every single day that gives a lifetime 
> of 3 to 9 years. And prefixes shouldn't change that often anyway. When 
> looking at more realistic estimates line a change on average once a week 
> you'l get to 20 years. Seems more than enough for low end extra cheap 
> devices. I don't see this driving cost up, except for spending a few 
> hours on properly engineering this.

There are places where they flap your WAN (PPPoE session for instance) 
every day to renumber you. I have personally experienced this.

So it might be possible to do what you're saying but only writing partial 
information of the prefix, so skip writing lifetimes, and just keep the 
actual prefix. This is not what a typical DHCP server/client does today, 
it re-writes the file every time the lease is renewed so it knows about 
it. Some kind of compromise might be where a "hint" file is written and 
the only time this is re-written is when the prefix is changed.

OpenWrt today stores most volatile things in /tmp which is just /tmpfs. 
This has the added disadvantage that after a reboot the LAN lease file is 
empty.

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 09:13:13 -0300

>           Section 5.5.3, (e) of [RFC4862].

Yes, we missed this REQ, and will update our doc accordingly. HOwever,
this bullet was included for completeness sake. The case we care and are
trying to address here is that in which the CPE reboots.

Thanks!

Cheers,
Fernando

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 09:18:26 -0300

On 31/1/19 08:57, JORDI PALET MARTINEZ wrote:
> I've seen that several times, having for example statistics written in the internal router flash, easily will break the router, 

Yes, this has been my point while discussing these issues while working
on the I-D.

> or at least needed a factory reset or something similar. Alternatively using a "good quality" USB stick to save this kind of data, avoids the problem 
>(most of the time, because from time to time I've got also good USBs
being damaged after some months of usage).

Same here. I run a lot of Raspberry Pis for measurement purposes, and
quite often the SD cards get damaged.

> Mandating that in low cost CEs will mean CEs are no longer "low cost", unless new "safe" opensource ways to keep the flash are available and can be included in new firmware releases.

FWIW, i don't think mandating this is the way to go. We were not sure
whether to make this a MUST (and then CPEs might claim whether they
comply with this spec or not), a SHOULD, or have no normative language.

I guess we could elaborate a bit more about the CPEs, discuss why it may
not be feasible to do this, and s/MUST/SHOULD/. What do you think?

Thanks!

Cheers,

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 09:21:32 -0300

> A flash cell is at least capable of 1000 to 3000 erase/write cycles. Even if the prefix would change every single day that gives a lifetime of 3 to 9 years. And prefixes shouldn't change that often…

You don't really know where these boxes are going to be deployed. I know
of several places where you do experience electric outages several times
a week.

---

From: Sander Steffann <[email-masked]>
Date: Thu, 31 Jan 2019 13:53:59 +0100

Hi Fernando,

> of several places where you do experience electric outages several times a week.

That doesn't require a write until the connection comes back with a different prefix. The "bit of engineering" I mentioned does have to include a bit of code to only write the new prefix once it has been replaced by something else.

Cheers,
Sander

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Thu, 31 Jan 2019 14:15:49 +0100

Several times a day in many places!

Regards,
Jordi

-----Mensaje original-----
De: v6ops <[email-masked]> en nombre de Fernando Gont <[email-masked]>
Fecha: jueves, 31 de enero de 2019, 13:22
Para: Sander Steffann <[email-masked]>, Mikael Abrahamsson <[email-masked]>
CC: IPv6 Operations <[email-masked]>, "[email-masked]" <[email-masked]>
Asunto: Re: [v6ops] A common problem with SLAAC in "renumbering" scenarios

    

    
    _______________________________________________
    v6ops mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/v6ops

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Thu, 31 Jan 2019 14:24:55 +0100

If the problem is no really about increasing the hardware cost, but proper engineering, as Sander suggested, to manage how to write info in the flash, and a CE, even if it has several power outages per day, can last for more than 5 years, I think is fine to go for MUST.

So, in the worst case a CE that get a power outage or whatever else, average 2 times a day, give, for a flash with 3.000 cycles, between 4-5 years CE lifetime, which I think in most of the case will be sufficient.

I'm sure that this can be improved not just with better flash, but with techniques to better manage the flash, file system, etc.

My concern is why many routers get corrupted when "saving" data in the flash if there is a power outage in the middle of the operation. I'm convinced it is again about proper engineering.

I guess this is the same problem for IoT devices when being update OTA, etc., so sure it is solved, but it is not being applied to Linux filesystems in routers?

Regards,
Jordi

-----Mensaje original-----
De: Fernando Gont <[email-masked]>
Fecha: jueves, 31 de enero de 2019, 13:18
Para: JORDI PALET MARTINEZ <[email-masked]>, Mikael Abrahamsson <[email-masked]>
CC: IPv6 Operations <[email-masked]>, "[email-masked]" <[email-masked]>
Asunto: Re: A common problem with SLAAC in "renumbering" scenarios

    

    
    
    Thanks!
    
    Cheers,

---

From: Philip Homburg <[email-masked]>
Date: Thu, 31 Jan 2019 14:43:37 +0100

>everything works. Changing all hosts is a big thing.

I think changing host behavior is good in this case if we can make hosts more
robust. I prefer the network to do the right thing, but reality is that that
will not always happen, so it is good to have robust hosts.

That said, I don't like playing games with the valid lifetime. Changes to 
preferred lifetime are relatively benign, setting the valid lifetime of a
prefix to zero may have more serious consequences.

I think the algorithm can be simplified to:
- When a RA is received that contains a PIO with the A bit set, then set the
  preferred lifetime to zero for all addresses derived from prefixes that were
  announced by the same router and that are not covered by the current RA.

In extreme cases this may cause the preferred lifetime to ping-pong. But that
just causes source address selection to become more random, which should be
fine.

---

From: Richard Patterson <[email-masked]>
Date: Thu, 31 Jan 2019 13:51:31 +0000

> trying to address here is that in which the CPE reboots.

I think this requirement already resolves the CPE-side of things, and
the rest I believe, is solved server-side?
If the CPE is hard rebooted, it hasn't sent a Release, so I'd expect
the lease-state to be kept server-side, and re-offered if the DUID+MAC
binding is the same when the CPE reconnects.

Although, during soft reboots, I have often seen CPEs sending a
Release, telling the servers to relinquish the lease.  Perhaps we
should discuss the benefits/detriments of doing this?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 31 Jan 2019 15:23:00 +0100 (CET)

> should discuss the benefits/detriments of doing this?

This is a good point. On our HGWs we disabled ever sending release just to 
try to always give the same prefix to the customer.

---

From: Richard Patterson <[email-masked]>
Date: Thu, 31 Jan 2019 17:49:40 +0000

> try to always give the same prefix to the customer.

Likewise for soft reboots, but we still send a Release if the user
specifically clicks the "Disconnect" button in the UI, just in case
they do wish to change prefixes.

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 16:15:19 -0300

> If the problem is no really about increasing the hardware cost, but proper engineering, as Sander suggested, to manage how to write info in the flash, and a CE, even if it has several power outages…

I'm fine either way. That said, I still use some CPEs older than 5
years, and would find quite disappointing if I had to replace any of
them because of *this*.

> My concern is why many routers get corrupted when "saving" data in the flash if there is a power outage in the middle of the operation. I'm convinced it is again about proper engineering.

In this case, I bet it is. Essentially you'd have to have an alternative
means to bootstrap the device, if you screw the main one.

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 16:18:41 -0300

> binding is the same when the CPE reconnects.

Well, if you want to play safe, you cant expect anything. Also, in some
cases it may be the actual *goal* to get a *different* prefix. i.e., the
so-called "dynamic" prefixes, which seem to be the recommended default
there.

> should discuss the benefits/detriments of doing this?

I will add some text on the topic. However, if we elaborate more on the
CPE behaviour we may want to end up splitting the document: one part
that focuses on the host-behaviour and that is aimed at 6man, and the
CPE part aimed at v6ops?

Thanks!

Cheers,

---

From: Fernando Gont <[email-masked]>
Date: Thu, 31 Jan 2019 17:02:43 -0300

> prefix to zero may have more serious consequences.

FWIW, our first initial version only changed the "Preferred Lifetime",
and never modified the "Valid Lifetime". Then we realized it was easy to
update the "Valid Lifetime" with the same "mechanism" and incorporated
that. THe most important bit is, indeed, updating the "Preferred Lifetime".

Question:
How about making the update to the "Valid Lifetime" a SHOULD or MAY,
instead?  Or, maybe, at least, set it to the current "Valid Lifetime"?
Otherwise, with the default values for the "Valid Lifetime", the
addresses might lie around for 1 month....

> just causes source address selection to become more random, which should be

The mechanism in the document tries to accommodate the case where the
information conveyed in an RA is split among two RA messages -- so it's
the second RA with the PIO that causes the update, rather than the first
one. In all current practical cases, all info is normally conveyed into
a single RA. However, the mechanism that's currently in the I-D allows
for cases where it can be split in two.

Based on whether we want to allow such case or not, we may or may not
simplify the mechanism as you suggest.

What do other folks think?

Thanks!

Cheers,

---

From: Mark Smith <[email-masked]>
Date: Fri, 1 Feb 2019 08:26:10 +1100

Hi,

So I thought about this problem back when I was doing some work with
IPv6 CPE vendors and working on a production IPv6 residential
deployment in 2010. As the CPE vendors were commonly using 'radvd', I
thought it would be easier for me to write some code for that to
handle this situation and push it directly to the 'radvd' people
rather than try to go through the vendors and have them up stream it.

Here's the descriptions of the two options that were added for a
prefix (from radvd.conf(5))

       DeprecatePrefix on|off

              Upon  shutdown,  this  option  will cause radvd to deprecate the
              prefix by announcing it in the radvd shutdown  RA  with  a  zero
              preferred  lifetime and a valid lifetime slightly greater than 2
              hours. This will encourage end-nodes using this prefix to depre‐
              cate any associated addresses immediately. Note that this option
              should only be used when only one router is announcing the  pre‐
              fix onto the link, otherwise end-nodes will deprecate associated
              addresses despite the prefix still  being  valid  for  preferred
              use.

              See  RFC4862, section 5.5.3., "Router Advertisement Processing",
              part (e).

              Default: off

       DecrementLifetimes on|off

              This option causes radvd to decrement the  values  of  the  pre‐
              ferred  and  valid lifetimes for the prefix over time. The life‐
              times are decremented by the number of seconds  since  the  last
              RA. If radvd receives a SIGUSR1 signal, it will reset the values
              of the preferred and valid lifetimes back to the initial  values
              used by radvd when it started. If radvd never receives a SIGUSR1
              signal, it will continue to decrement the  lifetimes  until  the
              preferred  lifetime  reaches  zero. After a final RA with a zero
              value preferred lifetime, radvd will cease to announce the  pre‐
              fix.  If a SIGUSR1 signal then causes the lifetimes to be reset,
              the prefix will then re-appear in the RAs.

              This option is intended to be used in conjunction with a  DHCPv6
              client that is using the Identity Association for Prefix Delega‐
              tion (IA_PD) option to acquire a prefix from a Delegating Router
              for use by a Requesting Router. In this scenario, the prefix(es)
              from within the delegated prefix that  are  announced  by  radvd
              would age in parallel with and at the same rate as the delegated
              prefix, and expire at approximately the same time, if the  dele‐
              gated prefix's life isn't extended.

              See RFC3633, "IPv6 Prefix Options for Dynamic Host Configuration
              Protocol (DHCP) version 6".

              Default: off

As mentioned in the first option description, to deprecate an
address/prefix, a RA PIO valid lifetime of zero is invalid, and must
be greater than 7200 seconds / 2 hours. This is to prevent a DoS by a
rogue RA announcing a 0 valid lifetime. RFC4862, section 5.5.3.,
"Router Advertisement Processing", part (e) discusses this. I chose
the value 7203 seconds.

The second option is to ensure synchronised ageing of addresses on
hosts in parallel with the age of the DHCPv6-PD prefix. In case of a
CPE reboot, if the CPE gets a different new prefix via DHCPv6-PD, the
new prefix's lifetimes will probably be greater than those of the old
DHCPv6-PD prefix, and if I recall correctly, I was seeing hosts choose
to use the highest preferred lifetime addresses. So the addresses from
the newer PD prefix should be used by hosts, and the addresses from
the old and now invalid one should eventually age out and disappear.
I've recently thought there was something in the source address
selection RFC about choosing highest value lifetimes in source address
selection, however I can't seem to find it, so I may be
misremembering.

This second option isn't perfect though, as it won't work if the newer
PD prefix has lower initial lifetimes than the current lifetimes of
the old prefix at the time CPE reboots. That could happen if an ISP
has decided to roll out lower PD prefix lifetimes.

More broadly, a router reboot is or should be a transient packet loss
event. Hosts are expected to tolerate transient packet loss, which is
why TCP didn't originally have any keepalives at all, and the
keepalives it commonly has now has a large initial timeout value of 2
hours. Devices (well, routers) in the Internet aren't intended to have
knowledge of transport layer connections, so that if they reboot they
don't have to reacquire or relearn transport layer connection state
before being able to forward hosts' packets. IPv4/IPv6 forwarding is
purposely stateless.

So if a CPE reboots and reattaches to the network/Internet at the same
point, it should be given back the same DHCPv6-PD prefix i.e. the
stable prefix scenario. If the CPE reattaches to the network at a
different point (for some definition of different attachment point
e.g. different access link, different PPPoE username), then giving it
a different PD prefix is reasonable and probably should be expected.

I sympathise with the privacy issue of the stable prefix. I think it
would be better to have the end-user initiate a PD prefix change via a
web portal or similar, rather than rebooting their CPE, because it
isn't possible for an ISP to distinguish whether a CPE reboot was
caused by a bug or the end-user was actively doing it for the purposes
of changing the PD prefix, so it isn't possible to absolutely know if
the same or a different PD prefix should be given back to the CPE.

If an ISP wants to or needs to initiate the prefix changes, they could
have a mechanism to cycle customers through old and new DHCPv6-PD
prefixes, phasing them in and out over time using IPv6 address ageing.
I don't think that would be a very simple mechanism to build, however
there is the cost of providing prefix privacy over time. (One issue
might be that some customers have connections they want to have up for
long periods of time e.g. a CCTV video feed, longer than the ISP
chosen default prefix lifetime for privacy. Those customers should
probably given a option somewhere to opt out of the changing prefixes,
or to be able to set their own privacy prefix change interval.)

Note that the ISP I worked for in 2010 provided (an still provides)
stable PD prefixes (and dynamic SLAAC ones on the PPPoE/PPP session),
so the ability to cope with changing PD prefixes upon CPE reboot
wasn't a capability we required. I mainly spent my own time on the
problem and the 'radvd' code.

I think that's all of what I thought about this at the time, I'll see
if I can remember anything else.

Regards,
Mark.