Subject: Operational Renumbering Procedure

From: [email-masked]
Date: Fri, 18 Apr 2003 09:35:57 -0700 (PDT)

I'd be interested in your comments on
http://www.ietf.org/internet-drafts/draft-baker-ipv6-renumber-procedure-00.txt.

This document grew out of a brief exchange on the IETF list, and an
internal exchange with some operations folks at Cisco. What I'm trying to
explore in the document is why some people say "renumbering is easy" and
others say "renumbering is next to impossible" - the latter often with
expletives added for color. We get a whine from various and sundry saying
"please make renumbering easy to do", but we aren't especially told what to
make easy.

It turns out that operationally, the best thing we can do to make
renumbering easy is make manual configuration really hard. It is the cases
of manual configuration - things that are not automated in the first place -
that are hard to automate.

I'll take direction on what to do with the draft. I have had one network
operator (who has to renumber an IPv4 network) tell me that the procedure is
of value to him a it gives him a step in defining things. So it may be of
value as an informational document. But for me, the real value lies in
helping the IPv6 and the operational communities talk with each other.
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Pekka Savola <[email-masked]>
Date: Fri, 25 Apr 2003 22:40:16 +0300 (EEST)

> helping the IPv6 and the operational communities talk with each other.

I think this is certainly something we need to do in the multi6 and even
ipv6 wg context.

There are two related processes, in my mind:

 1) 
  a) documenting best practices to have your network in a state you can 
renumber it if needed
  b) practices to use if/when you need to renumber

and:

 2) developing mechanisms which make the renumbering even a bit more 
easier.  It's difficult, and we shouldn't expect too much of it, but I'm 
pretty sure there may be ways to at least eliminate the worst quirks.

In the "optimal" case these would be tied together in a feedback loop :-)

I've had some thoughts on both, but as there has not been a sufficient
community interest, I haven't bothered to go deeper into it.

...

As to your draft -- I didn't read it properly, but a few comments:

 - a bit much of too pretty language (e.g. the first paragraph), have to 
concentrate too much to understand what it means

 - the claim, originally made in RFC2072 I suppose:

   RFC 2072 [6] describes the implications of renumbering in an IPv4
   network. A number of issues are raised, not the least of which is
   that while IPv4 in no sense precludes the configuration of more than
   one prefix on an interface in an equal sense, IPv4 equipment usually
   does not provide this capability.

   .. IMO appears to be wrong.  The IPv4 equipment I use certainly allows 
all of this.  Typically only those which are restricted anyway (like 
printers configured from an LCD panel or such!) may only allow one, but 
this doesn't seem to be an architectural limitation.

 - you're having (perhaps unintentionally) an ISP-driven perspective or 
some unstated assumtions; that is:

2.3 Stable routing both prefixes

   Once the network has been configured with the new prefix and has had
   sufficient time to stabilize, it becomes a stable platform with two
   addresses configured on each and every infrastructure component
   interface (apart from serial interfaces that use only the link local
   address), and two addresses available for the use of any end system,
   one in the old prefix and one in the new. However, due to DNS [3][4]
   advertisement and history, sessions are opened with the addresses in
   the old prefix; the new is idle. This is a stable configuration.

... in practice there is no such stable state with IPv6, in the generic 
case.  Reason is that if the ISP's use ingress filtering for the site, 
using both prefixes will result in about 50% of packets getting dropped 
unless you have something like source-based routing.  This problem has 
been noted earlier in host-centric multihoming, for example.

The other explantions why this was overlooked may have been:
 - if you renumber an ISP or a major customer, there are typically no 
ingress filtering issues or at least you're in a position to negotiate 
about them
 - with IPv4, some get around these issues by using NAT in the border 
routers

 - I'd suggest removing the references and assumptions of RFC2894 -- or at
least sweeping them up to a corner of a draft, as it seems to be
practically useless & unimplemented approach.  Nit: "presumes the use of 
IPv6 Autoconfiguration as described in RFC 2894" seems odd -- the main 
body of that RFC is *not* about autoconfiguration as we typically 
understand it (in the RFC2461/2462 sense).

 - section 3 should more clearly separate (both are mentioned to some
extent, but after the mention, you go straight to the first) the cases of
"manually configured but local"  (in theory, you can keep a better track
of these using some methods!) and "manually configured but remote" (e.g.
someone else's access lists etc.).  These issues have entirely different
implications I think.

 - s/liklihood/likelihood/

 - in acknowledgements, s/Michel P/Michel Py p/ ?

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Fred Baker <[email-masked]>
Date: Fri, 25 Apr 2003 14:00:29 -0700

>In the "optimal" case these would be tied together in a feedback loop :-)

um, yes. In mentioning 2894, I was hoping that (2) was covered. Perhaps I 
should refer to 2461/2462.

> concentrate too much to understand what it means

OK.

>all of this.

Hmm. The equipment I am referring to is Cisco Routers, *nix machines, 
Microsoft Windows, and so on. To put two IPv4 prefixes on a Cisco Router 
interface, one needs to configure two sub-interfaces rather than one 
interface, and until relatively recently this was not necessarily 
supported. Sun Workstations and Linux machines have a very clear sense of 
what single IP prefix and address is used on an interface, as does Windows; 
I'm not at all certain how I would go about putting a second address on a 
Windows interface at all, and on a Sun, it would be a "secondary address" 
or an entry in the host file (such as localhost has). A secondary address 
is in no sense the same thing as having two co-equal prefixes on an 
interface. Cisco routers have secondary address as a capability, but it is 
not at all the same thing as having two addresses; if you remove the 
primary, the other doesn't simply "become" the primary...

>been noted earlier in host-centric multihoming, for example.

um, I'm assuming that if someone is using a given prefix via an ISP, the 
ISP is not filtering it out. Either the ISP is routing that prefix to them 
and therefore it meets the RPF check or whatever at the ISP, or the user 
has provided the address to the ISP (in BGP or by contract) and asked them 
to please not filter it out.

A scenario in which an enterprise or tertiary ISP intends to use multiple 
prefixes via the same transit ISP and doesn't advise their ISP to tune the 
ingress filtering seems a trifle daft, for the exact reason you point out.

>  - with IPv4, some get around these issues by using NAT in the border

yes to both, except that in this case I'm looking at IPv6. The same 
procedure can be used with IPv4, with some adaptations. The acronym "DHCP" 
becomes important.

>understand it (in the RFC2461/2462 sense).

no, it is not about what one does to auto-renumber systems. It is how to 
use that facility at the operational level.

Let me give you an IPv4 example of what I have visions of. My Cable ISP 
gives the router at my home an address using DHCP; the lease is two weeks 
(!). Imagine that someone at the ISP decided to renumber their network by 
going to each DNS server and replacing the A records with new ones in the 
new prefix, and simply changing each router interface, which has a 
configuration like

         interface <whatever>
           ip address 12.107.10.1 255.255.255.128

and typing perhaps

         interface <whatever>
           ip address 13.107.10.1 255.255.255.128

Now, the network would pretty quickly sort out routing, updating LSAs and 
constructing routes to the various parts of it. Anyone who had gotten a DNS 
translation of a name within the DNS time-to-live would continue to use the 
old (now unroutable) address for the server until the DNS translation 
expired, and only then get the new translation. Well, yes, that is a worst 
case, but a possible one. The router at my home wouldn't ask about its 
address and prefix until the DHCP lease expired, which might be two weeks. 
Basically, we would have a pretty serious outage.

What I'm trying to propose here is not a new way to automatically renumber, 
but an operational procedure for using the existing automated facility. It 
says "when you beat your head against the wall, it hurts. Solution: don't 
beat your head against the wall. Configure both prefixes, route both 
prefixes, and let all those timers expire and people switch over to the new 
prefix. *Then* discard the old."

The next question is "so what about all those things that aren't using the 
automated facility and as a result don't get the message?" Here, as some 
say, there be dragons. The *other* thought in the draft is "don't do that" 
and "if you're forced to do it anyway, here's where you get to run around 
and re-do all your static configurations." It turns out that there are some 
places where you're forced to use static configuration: ACLs and Route Maps 
come to mind pretty quickly. Those are examples of what has to be addressed 
with appropriate configuration.

>someone else's access lists etc.).  These issues have entirely different

OK

>  - in acknowledgements, s/Michel P/Michel Py p/ ?

thanks.

---

From: Bill Fenner <[email-masked]>
Date: Fri, 25 Apr 2003 16:28:03 -0700

>Hmm. The equipment I am referring to is ... *nix machines, ...

BSD 4.4-based operationg systems allow multiple IPv4 prefixes on an
interface in an equal sense.  Configuring the second and further prefixes
requires an additional argument to the ifconfig command simply to signify
that it's an additional address instead of a replacement; it does not
make the address "primary" or "secondary" or different in any way.
Deleting the "primary" address just means that there's another address
that's "primary", since it's the arbitrary distinction of being first
on the list of address/prefixes assigned to the interface.

Source address selection is done based on the routing table; e.g. in the
following configuration:

fenestro% ifconfig de1
de1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        inet 172.24.37.94 netmask 0xffffffe0 broadcast 172.24.37.95
        inet 10.0.1.242 netmask 0xffffff00 broadcast 10.0.1.255

if the nexthop of the route is on 10.0.1.0/24, the source address is
10.0.1.242 .  If the nexthop is on 172.24.37.64/27, the source address
is 172.24.37.94 .

e.g.
Destination        Gateway            Flags    Refs      Use  Netif Expire
10.0.2/24          10.0.1.1           UGSc        0        6    de1

fenestro% ping 10.0.2.1

results in

16:20:02.815564 10.0.1.242 > 10.0.2.1: icmp: echo request

and deleting the 172.24.37.94 address from the interface affects this
not one whit:

fenestro% sudo ifconfig de1 172.24.37.94 delete
fenestro% ifconfig de1
de1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        inet 10.0.1.242 netmask 0xffffff00 broadcast 10.0.1.255

fenestro% ping 10.0.2.1

still results in

16:26:11.722139 10.0.1.242 > 10.0.2.1: icmp: echo request

  Bill

---

From: Pekka Savola <[email-masked]>
Date: Sat, 26 Apr 2003 08:58:32 +0300 (EEST)

On Fri, 25 Apr 2003, Fred Baker wrote:
> supported. 

Really?  All the software versions in 3+ years I've used are supported 
multiple addresses on the same interface using "secondary" keyword. (But 
see below.)

> Sun Workstations and Linux machines have a very clear sense of 

Works in Linux just fine.  Often, additional addresses are added as 
"aliased" or "sub" interfaces, but that's not necessary, you can put all 
of them in the main interface too.

> or an entry in the host file (such as localhost has). 

I'm not aware of these systems, but some windows boxes at least allow 
aliased IP addresses and more than one default gateway.  I don't know what 
happens if you remove the ones you added first.

> A secondary address 

I'd say this is an implementation problem (or rather, a shortcut to make
address selection etc. easier).  For example in Linux, you can add/remove
addresses freely, as with BSD.
 
> ISP is not filtering it out.

Depending on what you meant with "via" there are two interpretations:

1) ISP will allow the prefix that particular ISP gives you, that's without 
doubt.

2) You *cannot* control whether a prefix is used via an ISP, when it's 
selected as source address but the route goes to the destination.  (For 
more, see explanation below.)

An alternative would be pushing the whole routing table to the hosts.  
That has been proposed too, though..

> Either the ISP is routing that prefix to them 
> and therefore it meets the RPF check or whatever at the ISP, 

OK.

> or the user 

I'm not completely sure which prefix you refer to.

> ingress filtering seems a trifle daft, for the exact reason you point out.

You may be missing my original point.  Let me try to elaborate even more.

Let's simplify the network to a very minimal configuration:

2001:1::/32  2001:2::/32
    ISP1      ISP2         
      \       /
        router  2001:1:1::/48 (statically routed from ISP1)
          |     2001:2:2::/48 (        ---"""---      ISP2)
          |
         host   2001:1:1::1, 2001:2:2::2

Assume the host has both 2001:1:1::1 and 2001:2:2::2 in DNS under 
"www.example.com".

Some hosts in the Internet use these addresses.  Client A uses the latter
and client B uses the former (due to round-robin algorithm, these can be
in either order).  They're connected like:

 Client A         Client B
      \            /
      ISP A ---- ISP B
         \        /
        ISP1 -- ISP2
            \   /
            router
               |
              host  

(or whatever -- what's important that Client A/B are not direct customers 
using ISP1/ISP2's address space, respectively).

Assume that the best return path from host to Client A goes through 
ISP1/ISPA and to Client B, through ISP2/ISPB.

Now, packets from Client A to 2001:2:2::2 go through ISPA then to *ISP2*
by some fashion, and from there to the router.  However, the reply packets 
use 2001:2:2::2 as the source address, but are routed towards Client A 
through ISP1 and ISPA.

Similar with Client B.

The situation remains even if there is only one active default route at 
the site's router, there are two routers, or whatever.

ISP1 allows through 2001:1:1::/48 but not 2001:2:2::/48.  ISP2 allows 
through 2001:2:2::/48 but not 2001:1:1::/48.  Therefore the packets get 
dropped.

But you may be assuming that either:
 1) the site is using provider-independent addresses, and this is not a 
problem (but then, renumbering is rarely a problem either).  Such 
addresses do not exist for endsites at the moment though, which is good.
 2) ISP1 would accept any BGP announcement from the site (like a prefix 
derived from ISP2).  With provider-based addressing, that's pretty much 
unacceptable, or
 3) the site could negotiate exceptions to the ingress filtering.  
However, these would have to be made everywhere, and there may be upstream 
ISP's also ingress-filtering.  Doesn't seem like a realistic option,
 4) there is some source-based routing implemented at the site router -- 
this would "fix" the problem.

> procedure can be used with IPv4, with some adaptations. The acronym "DHCP" 

Of course -- I realize you talk about IPv6 -- but these were the reasons
for the omission I thought possible in this case.

> use that facility at the operational level.

I still think that gives a wrong impression as the reference introduces a 
protocol -- that's what's typically meant when the reference is flashed 
out.
 
> Basically, we would have a pretty serious outage.

I think it's pretty much established that renumbering with a time span 
longer than lease/TTL times is just not realistic.

In this case, they should lower the lease time, wait two weeks to make it 
kick in (unless you can "push" the new lease times), and see whether they 
could do the renumbering procedure with a shorter lease times (e.g. an 
hour).
 
> prefix. *Then* discard the old."

I symphatize: this is the procedure I'd like too, but it has certain 
problems still.
 
> automated facility and as a result don't get the message?" 

What exactly are you referring to with "the message"?  Not a message, 
protocol-wise?

> Here, as some 

Yep.

---

From: Hans Kruse <[email-masked]>
Date: Mon, 28 Apr 2003 10:22:15 -0400

Just to complete the list, Windows NT and up (2000, XP, I have not yet 
tried in 2003) all accept multiple IPv4s per interface.  I believe there is 
a certain arbitrariness about what is "primary" depending on the 
application, but deleting the "first in the list" address does not take the 
machine off the network.

--On Friday, April 25, 2003 14:00 -0700 Fred Baker <[email-masked]> wrote:

> what single IP prefix and address is used on an interface, as does
> Windows; I'm not at all certain how I would go about putting a second
> address on a Windows interface at all, and on a Sun, it would be a
> "secondary address" or an entry in the host file (such as localhost has).
> A secondary address is in no sense the same thing as having two co-equal
> prefixes on an interface. Cisco routers have secondary address as a
> capability, but it is not at all the same thing as having two addresses;
> if you remove the primary, the other doesn't simply "become" the
> primary...

Hans Kruse, Associate Professor
J. Warren McClure School of Communication Systems Management
Adjunct Associate Professor of Electrical Engineering and Computer Science
Ohio University, Athens, OH, 45701
740-593-4891 voice, 740-593-4889 fax
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------