Subject: [rrg] [Fwd: I-D Action:draft-carpenter-renum-needs-work-00.txt]

From: Brian E Carpenter <[email-masked]>
Date: Fri, 24 Oct 2008 09:41:38 +1300

Hi,

Ran and I drafted this in reaction to the earlier discussion here
about the impracticality of renumbering. We'd like comments and, above
all, contributions. For now, this list is suggested for any discussion,
but I will also mention the draft on the intarea list.

    Brian

-------- Original Message --------
Subject: I-D Action:draft-carpenter-renum-needs-work-00.txt
Date: Thu, 23 Oct 2008 13:30:02 -0700 (PDT)
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Renumbering still needs work
	Author(s)       : B. Carpenter, R. Atkinson
	Filename        : draft-carpenter-renum-needs-work-00.txt
	Pages           : 18
	Date            : 2008-10-23

This document reviews the existing mechanisms for site renumbering
for both IPv4 and IPv6, and identifies operational issues with those
mechanisms.  It also summarises current technical proposals for
additional mechanisms.  Finally there is a gap analysis.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-carpenter-renum-needs-work-00.txt

_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Robin Whittle <[email-masked]>
Date: Mon, 27 Oct 2008 16:20:42 +1100

Short version:   Renumbering of end-user networks can never be
                 sufficiently testable, reliable or secure to
                 be in any sense "routine".

                 Therefore, we cannot expect to have a routing
                 scaling solution which depends on such
                 "routine" renumbering be accepted by the vast
                 majority of end-user network administrators -
                 which is a requirement, since we need to get
                 almost all such networks to adopt it.

Hi Brian and Ran,

Regarding:

  http://tools.ietf.org/html/draft-carpenter-renum-needs-work-00

I don't think it will ever be practical (while retain existing
Internet protocols) to solve the routing scalability problem in a
way which requires end-user networks renumber in order to use
another ISP.

Even if we ignore the problem of keeping sessions alive during a
renumbering operation, I think there are fundamental problems with
your intention (I assume) to find a way of making either IPv4 or
IPv6 renumbering "a relatively routine event", as your I-D mentions.
 	
Even if you ignored the problems of implementing the requisite
automated renumbering functions in existing equipment (primarily
hosts, including printers and in routers, access points and other
networking gear) I think there are fundamental problems with what
needs to be done beyond this.

To be practical, something with the extreme impact of renumbering
needs to be extremely reliable.  In order for this to be the case, I
think it needs to be:

1 - Highly or completely automated, so it can all be done under
    central control.

2 - Secure - against hackers or accidental activation.

3 - Fully testable.  There must be a sham approach to implementing a
    renumbering operation in order to prove to the administrators
    that all required numbers are changed, in the right time, to
    the correct new numbers.  But as I discuss below, this is
    nowhere near sufficient to prove the whole renumbering
    operation would in fact succeed.

IP addresses turn up in DNS zone files and ACLs all over the place -
often well beyond the physical network in question and often in
places which are no suitable for secure, automated control by the
administrators of the end-user network.

Let's say the end-user network is a hosting company.  Each host in
that network has a single IP address.  Each such host handles the
websites sites of 20 companies:  www.fu1.com, www.bar2.com etc.
Each such company has an IP address for one of these hosts in its
own DNS.

So in order to securely, automatically, renumber the hosting
company's network, that company would need to already have installed
secure, automated, testable, IP address changing mechanisms into all
the master nameservers of all its customer companies.  This is
clearly impractical.  Furthermore, those customer companies
generally wouldn't run their own DNS, so each such company fu1 and
bar2 etc. would need to negotiate and authorise their DNS company to
enable some hosting company (unknown to the DNS company) to be able
to automatically update a particular IP address, and only that
address, within the customer company's DNS records.

What about caching times?

If you are going to make IP addresses unstable, compared to the
current situation, then everywhere you have an IP address, in an
ACL, or whatever, and you haven't already got a secure, automated,
testable, renumbering system, you need to replace it with some
construct which is either remote controlled, as just suggested, or
by a DNS lookup.  Then some software process needs to check that DNS
entry frequently - but how frequently?  Not only must you substitute
an IP address with a DNS entry, but you need to specify a caching
time.  Ideally, those times should be long, to avoid overly
burdening the DNS system.  However, if you want to renumber, it
would be good to make the times short instead, just before the
renumbering operation.  There's no obvious way a "DNS company"
(whoever runs the DNS of fu1.com) is going to be able to let fu1's
hosting company control this.  The administration for this would be
costly and error prone, even if it was technically easy to do.
(Maybe it is technically possible with RFC 3007.)

Overall, I can't imagine any administrator of a substantial network
would come to accept that renumbering was a safe, non-disruptive,
thing to do to their network.  Unless all the world's hosts and
applications use new protocols which assume dynamically changing IP
addresses, there's no way an administrator is going to be able to
assure the owners of the network that renumbering will work as expected.

It is bad enough that renumbering completely changes the IP
addresses of every device in the network to some other one or more
values.  The whole network has to be commanded to do this, securely,
quickly - and yet as soon as the process starts, everything is in
flux.  How are these security arrangements meant to work when hosts
may have one or the other or both IP addresses?

Networks are already fragile enough, without having them depend on
another one or more layers of supposedly secure stuff which is
supposed to change their whole addressing structure without anything
going wrong.

How could the administrators know their renumbering management
system would work as intended?  To add some extra layer to simulate
the renumbering process would be helpful, but how could anyone be
sure this was adequate?   The only way to test a renumbering system
is to run it - to actually renumber the entire network.

If this could somehow be done in a few seconds, it might be workable
- renumber, test and renumber back to the original addresses, all in
about 10 or 20 seconds.

However, this will never be practical.  DHCP servers need new
marching orders and it would take minutes or hours or whatever for
the various changes to propagate to hosts.  Likewise, there are DNS
caching times to consider.

The only real test would be when all hosts have their new addresses
and all the cached, old, DNS values have been flushed from all hosts
around the world which are using them.  Likewise, ACLs or any other
settings in hosts outside the network need to be reconfigured before
it is possible to establish whether the renumbering operation was
completely successful.

This would be inordinately disruptive, not least because it will
probably require automated or manual effort on the part of systems
outside the physical network.  Any such automated effort requires
manual effort to set up the appropriate software, configure it
properly, set up security arrangements etc.  Then, these automated
systems need to be individually tested, which is a manual process.

Since networks are generally changing, how could the administrators
be sure their renumbering system would really work for 100% of the
network as it is from day-to-day?  I don't see how they could,
except by doing a test renumbering, and then somehow finding out
what parts, if any, were not renumbered as expected.

If this sort of "routine renumbering" requires new features in
hosts, then there is a major problem developing the protocols and a
many year delay - such as 5 to 10 years - before such protocols are
likely to be found in many or most printers, NAS devices, access
points, mobile devices etc. which are commonplace in many networks.
 (The same problem of lack of IPv6 support in printers etc. no-doubt
holds back many networks from switching to IPv6.)

In short, I think there profound and unchangeable reasons (assuming
we are still relying on TCP, UDP, existing applications etc.) why
administrators of any network with more than a handful of IP
addresses will never accept that renumbering their entire network
should be a routine event.

Imagine giving the command, and watching as the whole network
attempt to reorganise itself by the individual efforts of each
device - as the TCP session through which the command was given dies
due to the renumbering process itself!  It is bad enough commanding
a server in another country to "reboot" - but it sounds like a
nightmare telling an entire network to renumber itself, in the hope
that it will still be manageable and actually function enough as a
network, even if not fully operational, after the process is completed.

If the renumbering involves the DNS servers used by internal hosts
and/or the outside world, how is the administrator going to be
confident that internal hosts will behave as expected, when they are
asynchronously changing their IP addresses, while even their local
DNS server is changing its IP address too?  Only the root DNS server
addresses remain valid, so each host would need to frequently
rediscover the IP address of local DNS servers, since it could not
rely on any actual IP address in a config file, or in a DHCP lease.
 I suppose the saving grace is being able to define multiple DNS
server IP addresses - with only one needing to be operational.

I believe a requirement that end-user network renumbering be secure,
reliable and sufficiently low in cost to be "routine" cannot be an
essential part of any routing scaling solution.

Fortunately, there are promising, potentially practical, approaches
to solving the routing problem without routine renumbering of
end-user networks and without any changes to host operating systems
or applications:   LISP, APT, Ivip and TRRP.  For IPv6, maybe
Six/One Router will be practical.

An architecturally cleaner approach would be to scrap existing
protocols and therefore operating system stacks and applications.
Then we could develop new protocols for which IP addresses could be
fluidly changing without impact on applications.  If that could be
done with a new approach to routing, or perhaps the existing BGP
system plus SHIM6 etc. to scalably provide multihoming, with
portability between ISPs being handled through "routine renumbering"
then this *might* be a scalable solution which is reasonably
elegant.  But that would only be for IPv6, since renumbering from
one IP address to another, without having both during a changeover
period sounds unworkable to me.

However, this is never going to happen.  We need to work with
existing hosts and protocols.

We need to make something which is highly attractive to the very
first adopter, in five or so years time.  So it can't rely on host
changes etc. and must provide immediate benefits over costs,
irrespective of how many or how few other networks have adopted the
system.

I intend Ivip to meet these requirements.  LISP with PTRs is in
principle capable of meeting these requirements too, but I think
Ivip is a better approach.  APT has a backwards compatible approach
for handling packets sent from non-upgraded networks, so perhaps
that could be attractive to end-users too, without there needing to
be many other APT adoptors.  However, the granularity of the EIDs
for this support would not be finer than /24 unless there was a
single "APT Island".

I think your I-D is useful, since it identifies many of the barriers
to making the renumbering of end-user networks a "routine" process.

I hadn't thought about some things you mention, such as IP addresses
being stored in cookies, or perhaps in proxies.

 - Robin

---

From: Eliot Lear <[email-masked]>
Date: Mon, 27 Oct 2008 00:47:24 -0700

Robin,
>                   almost all such networks to adopt it.

I regret that I do not have time these days to read through more than 
your short version, but I generally agree with your design assumption.  
This is not to say that I don't hope for better.  DHCP prefix delegation 
is a good step in the right direction, particularly for SMB & home 
networks.  I've consistently said that our scaling problem is NOT with 
enterprise networks, but with home and personal networks.  In those 
latter two cases I hold out more hope.  If enterprise networks were 
merely a 2^6 exception, we would be in good shape.

Eliot

---

From: Victor Grishchenko <[email-masked]>
Date: Mon, 27 Oct 2008 09:43:19 +0100

IPv4 world was built on an assumption of static IP addresses, so
it is clearly impossible to introduce routine renumbering as an
incremental feature because of the reasons you mentioned and
other reasons.
"Fluid" addresses are possible in the world of prefix bunches,
multi-path TCP etc. Changing an underlying assumption affects
all the stuff piled on top of it.

---

From: Robin Whittle <[email-masked]>
Date: Mon, 27 Oct 2008 19:55:52 +1100

Short version:   Does anyone agree with Eliot's view?

                   "... our scaling problem is NOT with enterprise
                    networks, but with home and personal networks."

                 I don't.  I think that we should aim for a single
                 solution to be attractive to end-user networks
                 from universities and corporations to DSL-connected
                 SOHO users.

		 While renumbering for such small networks is a lot
                 easier than for corporate networks, unless there
                 are no other alternatives, I don't support the
                 scaling problem being solved just for "small"
                 networks, however defined, or differently for
                 "small" and "large" networks.

Hi Eliot,

You wrote:

> your short version, 

That's fine.  The title said it all.  I still haven't read your paper:

http://tprcweb.com/files/Lehr%20Vest%20Lear%20Internet%20Address%20TPRC%208_15_08.pdf

and I have a substantial message from Ran to respond to.

> but I generally agree with your design assumption.

OK.  My "design assumption" is that since we can't move to a
completely new set of protocols which would free us from the
requirement that IP addresses be generally stable, and that since
renumbering is a major upheaval for the network, and cannot be
reliably, securely automated that we cannot base a scalable routing
solution on "routine renumbering" of end-user networks.

My belief, argument, assumption or whatever is based on things which
I think can never change as long as we use the current protocols -
which is itself something I think we cannot change in any time frame
relevant to the routing scalability problem.

> This is not to say that I don't hope for better.  

OK.  I understand that you hope that things will get better - the
same things I am sure are completely unchangeable.  Hope is one
thing, but we would need to be pretty confident that such major
changes could be achieved soon if we were to base a routing
scalability solution on them.

> DHCP prefix delegation is a good step in the right direction, 
> particularly for SMB & home networks.

I am not sure what "SMB" means in this context.

> I've consistently said that our scaling problem is NOT with
> enterprise networks, but with home and personal networks.  

I don't recall you or anyone else arguing this - and I don't agree
with it.

I recognise that in the future, the numeric growth in multihomed
end-user networks will numerically be in many small networks which
exist now - which currently are not multihomed, due to the very high
costs of doing so with current BGP techniques.

Now we are discussing scalable routing again, I understand from this
that you think there could be two different solutions for "large"
and "small" networks, however defined - or perhaps that no solution
is needed for "large" networks.

> In those latter two cases I hold out more hope.  If enterprise
networks
> were merely a 2^6 exception, we would be in good shape.

I don't understand your last sentence.

While I wouldn't completely rule out different scaling solutions for
radically different sizes and types of networks, I have argued that
we should try to find a single solution which works equally well,
and is highly attractive, for networks of all sizes, from university
and corporate networks to single homes and offices currently
dangling from a DSL line.

  Map-encap space only for small end-users? PI space prices
  http://psg.com/lists/rrg/2008/msg00454.html  2008-02-22

If there were two solutions, or no solution for "larger" networks,
then those smaller organisations which aim to be large will tend to
adopt the "large" solution (if there is one) from the outset.

While I agree that the task of renumbering a network with a single
public address via a DSL service is easy compared to renumbering a
university network, I think we should aim for a single solution for
all such end-user networks.  If it is attractive to all types of
end-user networks, then there will be one less question to debate
about whether or not to adopt it.

 - Robin

---

From: Tim Chown <[email-masked]>
Date: Mon, 27 Oct 2008 10:06:32 +0000

> all the stuff piled on top of it.

There are always corner cases that either can't be covered or that have
some notable cost to work around.

I know of someone who recently asked me about a problem they had... their
sensor network had a hard-coded IP address for the server to send data to.
Unfortunately the server was hosted in a network that was being renumbered
due to a change in ISP.

The snag as you've probably already guessed was that these sensors were 
buried some way underground and not reconfigurable.

In every case there are things you can do to mitigate the cost of a 
renumbering event, but there are most likely tradeoffs in doing so.

-- 
Tim

---

From: "Teco Boot" <[email-masked]>
Date: Mon, 27 Oct 2008 11:12:26 +0100

I wouldn't say renumbering is always testable, is reliable and is secure.
But I think there are a large number of use cases that this is not a
problem.

Moreover, IMHO there are use cases where server farms needs very high
availability and this could be provided by using many PA addresses,
multi-homing and automatic renumbering with DNS Dynamic Update.

Teco.

> -----Oorspronkelijk bericht-----
> Van: [email-masked] [mailto:[email-masked]] Namens Robin
> Whittle
> Verzonden: maandag 27 oktober 2008 6:21
> Aan: RRG
> CC: Randall Atkinson
> Onderwerp: [rrg] Renumbering can never be secure, testable, reliable or
> routine - draft-carpenter-renum-needs-work
> 
<skip>

---

From: Stig Venaas <[email-masked]>
Date: Mon, 27 Oct 2008 11:46:00 +0100

> renumbering event, but there are most likely tradeoffs in doing so.

I think there are lots of things that could be done with design of
protocols, applications and management systems to ease renumbering.
Protocol design is maybe the least important IMO. The problem is that
renumbering is generally not considered when designing such things.

Stig

---

From: Eliot Lear <[email-masked]>
Date: Mon, 27 Oct 2008 14:24:36 +0100

Hi Robin,

>                   from universities and corporations to DSL-connected

Let's not conflate the problem with the solution.  By problem I mean 
that whatever number of enterprises are multihomed can and should be 
dwarfed by the number of home users and small/medium businesses (SMBs) 
in the timeframes we're discussing.  Whether the solution fits both is a 
separate discussion.  Ideally you want one, but sometimes the best 
approach is a short menu.

> solution on "routine renumbering" of end-user networks.

And I agree.  I do not see this happening in even the medium term.

> relevant to the routing scalability problem.

There are many reasons why I buy into the above design assumption.  As 
someone else wrote, many of the problems are not with protocols but both 
in implementation and also with the trust model of DNS.

> I don't understand your last sentence.

What I am saying is that if we had 2^6 enterprise networks announcing, 
using BGP, we could survive and survive well.  We can debate about the 
2^6 #, of course.
> adopt the "large" solution (if there is one) from the outset.

This very much depends on the upgrade paths available from one to 
another.  If it's easy to go from small to large solutions, or if the 
cost of going to a large solution is very high no matter what you're 
using, then I would think small organizations would start small.

Eliot

---

From: Robin Whittle <[email-masked]>
Date: Tue, 28 Oct 2008 00:52:48 +1100

Hi Eliot,

I think we broadly agree, except that you see more promise in
potentially two different scalability solutions to suit small and
large end-user networks (or no solution for big networks), whereas I
prefer one solution for all sizes of end-user networks.

Also, I think you meant 10^6 rather than "2^6", meaning a million
enterprise networks:

> What I am saying is that if we had 2^6 enterprise networks
> announcing, using BGP, we could survive and survive well.
> We can debate about the 2^6 #, of course.

I guess from this that you can imagine the DFZ being OK at a million
plus prefixes.

Maybe it can (I tend to think not), but doesn't the average
enterprise network have lots of physical sites, each of which it
would want to multihome via two or more local ISPs?  From your
message, I got the impression you thought a million enterprise
networks could be accommodated without any new system, using current
BGP techniques.

What assumptions are you making about the total number of prefixes a
million "enterprise" networks would advertise?  Or did you mean a
million prefixes for all enterprise networks, which I think is a
very much smaller number of enterprise networks.

  - Robin

---

From: Robin Whittle <[email-masked]>
Date: Tue, 28 Oct 2008 01:02:09 +1100

Hi Teco,

> But I think there are a large number of use cases that this is not a

OK - but are you suggesting that we should pursue any scalable
routing solution which is based on the notion that administrators
will accept renumbering of entire end-user networks as "routine"?

I can't imagine there's anything we could do - other than waving a
wand which changes the world's applications host stacks and routers
- which would convince administrators that renumbering was
sufficiently testable, secure or reliable for them to be happy doing
it as part of ordinary operations, for instance when selecting one
or more new ISPs.

The only scalable approach to multihoming without the major new
developments of a core-edge-separation system - such as map-encap -
is for IPv6 only: SHIM6.  For SHIM6, each end-user network needs at
least two ISPs, and each host gets at least two IP addresses for
SHIM6 to work with.  SHIM6 fails the "Will it be adopted widely and
quickly, even when initially hardly anyone uses it?" test of
providing substantial benefits to all early adoptors, since it only
works when the host at the other end of the communication also runs
SHIM6.  Also, SHIM6 only provides multihoming and TE without
centralised control.  It doesn't provide portability without
renumbering, which the core-edge separation systems can.

> multi-homing and automatic renumbering with DNS Dynamic Update.

Maybe so - but what if the servers were for websites of hundreds or
thousands of customers, and the customers' DNSes were the ones which
needed to be updated?  I argued that technically and
administratively this would be prohibitively costly, fragile,
insecure etc.

I wasn't suggesting that there be no efforts to make it easier to
renumber networks.  Just that any such developments are not
something we can expect to make administrators regard renumbering as
"routine" in the 3 to 5 years we have (my guess) to devise a
scalable routing solution.

In his most recent message, Eliot seems to agree that within the
time frame we have for solving the scalable routing problem we can't
move everyone to a whole new set of protocols.

I think he agrees that we can't assume that renumbering could be
"routine" for larger networks - but I think he is hopeful about it
being "routine" for smaller networks.

  - Robin

---

From: "William Herrin" <[email-masked]>
Date: Mon, 27 Oct 2008 10:03:04 -0400

> http://www.ietf.org/internet-drafts/draft-carpenter-renum-needs-work-00.txt

Brian,

You missed the more fundamental DNS problem:

95% of the software which interacts with the DNS does so via the
gethostbyname call. Gethostbyname does not return the TTL for the
information, thus the applications which use it do not conceive of a
hostname whose address changes during the course of the application's
operation.

Even if you were to deprecate and eventually eliminate the
gethostbyname call, painless renumbering is prevented by the rest of
the API which expects the layer-7 application to interact with the
layer-3 address. Because the application works right 99.9% of the time
(when IP addresses aren't changing) few small-time applications have
or will implement the TTL logic.

Some very large systems get it wrong too. One obvious example: Windows
XP and Windows Vista cache DNS query results for 15 minutes regardless
of the TTL reported by the DNS resolver. Windows 98 cached DNS query
results for 24 hours or until the next reboot.
(http://support.microsoft.com/default.aspx?scid=KB;en-us;263558).
Before you think to blame  all our woes on Microsoft: In Solaris 8 and
earlier, nscd also ignores the DNS TTL, passing already-stale records
back to apps which call gethostbyname.

Anecdotal reports from web server administrators suggest that a
hostname-based requests for web sites continue to trickle in at the
old IP address for months after the server address has changed in the
DNS.

Bottom line: because the APIs call for the applications to interact
with the layer-3 addresses directly instead of interacting with only
hostname targets, painless renumbering is an unattainable goal.

You also need a heading: SMTP issues.

Nearly all of the trust-management systems (aka anti-spam systems) for
email derive from the server IP address. It tends to go haywire when
the address of any large originator of email changes, already a source
of much pain and consternation for system administrators.

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Eliot Lear <[email-masked]>
Date: Mon, 27 Oct 2008 16:02:04 +0100

Robin,
> Also, I think you meant 10^6 rather than "2^6", meaning a million

Uch.  Indeed.  Thanks for the catch.

> I guess from this that you can imagine the DFZ being OK at a million

I don't know about the "plus", but we really aren't all that far from a 
million now (I'm thinking orders of magnitudes).

> would want to multihome via two or more local ISPs?

Of course we always get into trouble when we talk about "average".  I 
think what often happens is that an enterprise hands a contract off to a 
service provider for both internal and external connectivity.  Discount 
structures being what they are, I suspect this will tend to be the case 
into the future.  There are other models too that would scale more 
poorly than the existing the routing system, but the degenerative case 
is certainly the walking human network of devices.  And that may be a 
case we choose not to handle within the routing system.

>  From your

Yes.

> million "enterprise" networks would advertise?

O(10^6) routing entries (yes, I have that right, thanks).

>    Or did you mean a

And of course I would have to say that any system that escapes these 
engineering boundaries is preferable to one that does not, other things 
remaining equal.

Eliot

---

From: "William Herrin" <[email-masked]>
Date: Mon, 27 Oct 2008 11:33:54 -0400

On Mon, Oct 27, 2008 at 4:55 AM, Robin Whittle <[email-masked]> wrote:
>  I don't support the scaling problem being solved just for "small" networks, however defined,

Agree.

> differently for "small" and "large" networks.

Disagree. There's a money issue here: large networks can afford and
are willing to spend more money on multihoming than small networks.
There's no reason that both should be constrained to the same solution
at the same cost. The requirement is that whatever the scaling
solution, the per-network cost must either be trivial -or- recoverable
from the networks who instigate it.

BGP is unacceptable because it costs "the world" about $8000/yr for
every announced BGP prefix. If there was a way that "the world" could
recover that $8000 from each of the folks announcing a prefix, our
problem space would shrink considerably.

Regards,
Bill Herrin

---

From: David Meyer <[email-masked]>
Date: Mon, 27 Oct 2008 08:48:31 -0700

> problem space would shrink considerably.

	I'm curious as to where the $8000/yr number came from. Can
	you explain? Also, how is the "world" defined in this case?

	Thnx,

	Dave

---

From: "William Herrin" <[email-masked]>
Date: Mon, 27 Oct 2008 11:53:39 -0400

>        you explain? Also, how is the "world" defined in this case?

Hi David,

See http://bill.herrin.us/network/bgpcost.html

Regards,
Bill Herrin

---

From: David Meyer <[email-masked]>
Date: Mon, 27 Oct 2008 09:00:58 -0700

> See http://bill.herrin.us/network/bgpcost.html

	Thnx, I'll check it out.

	Dave

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 28 Oct 2008 16:48:14 +1300

Hi Bill,

Yes, we agree that if apps always used names, the problem
would at least be confined to the network stack. I thought
that was already fairly clear, but we'll work your points in.

Thanks for the SMTP point, it needs to be added.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 28 Oct 2008 17:02:26 +1300

> from the networks who instigate it.

I agree with Bill, and it seems to me that there's a fundamental
difference between Eliot's o(10^6) big sites and the much greater
number of small sites. The big ones are very likely to have a DMZ,
run their own servers, and have multiple points of interconnection
around the world. In other words, prime candidates for PI based
addressing and maybe a map/encap style solution.

The small ones are very likely to have a simple firewall/router
combo, outsource their publicly accessible servers, and have a single
point of attachment (or at least, several attachments in a relatively
small geographical area). They are also, I believe, much less likely
to be significantly disturbed by renumbering than the large sites.

I think there's definitely scope for two solutions.

    Brian

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 28 Oct 2008 11:33:50 +0100

Brian,
> but I will also mention the draft on the intarea list.

I completely agree that renumbering needs work, but I don't think that
we can approach the problem by looking at all the existing devices and
applications in a network and see how they can survive a renumbering
event and list all the practical problems that need to be solved.

I think that we should take a completely different approach.
I believe that the networks, small, medium or large, that are able to
easily renumber their addresses will have a competitive advantage in the
long term compared to the networks that cannot renumber because they
will be more agile and will adapt more easily to changes in providers
and so on.

The document that we need in the long term is not a list of all the
problems that we have today. It is a list of guidelines on how to build
an easily renumberable network. We should consider different types of
networks : home network, SME network, campus network, enterprise
network, small ISP network, ... and for each network define a set of
services that are used inside the network such as firewalls, DNS,
routers, applications, ... and for each service explain how it should
react to a renumbering event. I agree with you that in most cases, this
implies thu utilization of strong asset management to "automatically"
build the configuration of the services. I don't see strong asset
management as a limitation as it has other benefits in addition to
renumbering. Based on such a document, developpers of the different
services could improve their service to better support renumbering, e.g. :
- libraries used by applications could check DNS TTL
- applications could rely more on DNS than on IP addresses
- routers could better support dhcp prefix delegation
- firewalls could be configured based on information stored in DNS or
other databases instead of lists of IP addresses
- ...

I'm convinced that with some small changes we could significantly
improve the "renumberability" of services and applications.

Olivier

-- 
http://inl.info.ucl.ac.be , UCLouvain, Belgium
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Scott Brim <[email-masked]>
Date: Tue, 28 Oct 2008 11:38:21 -0700

> but I will also mention the draft on the intarea list.

Hi Brian and Ran.  Unfortunately for me you didn't produce a magic
solution for the site policy change issues (Section 4.3.4).  I think
that's the critical issue area.

swb

---

From: "William Herrin" <[email-masked]>
Date: Tue, 28 Oct 2008 14:51:48 -0400

On Tue, Oct 28, 2008 at 6:33 AM, Olivier Bonaventure
<[email-masked]> wrote:
> an easily renumberable network.

Oliver,

While that's a laudable goal, I doubt its attainable.

Renumbering problems break down into two general groups:

1. Problems which are within the administrative control of the
renumbering network.
2. Problems which are outside the administrative control of the
renumbering network.

Group #2 issues (like antispam filters and faulty DNS caching) is
where renumbering falls apart: you can control what you do and when,
but you can't control what everybody else does.

Regards,
Bill Herrin

---

From: Scott Brim <[email-masked]>
Date: Tue, 28 Oct 2008 12:19:38 -0700

> will be more agile and will adapt more easily to changes in providers

Would making it easy to renumber be easier or harder than changing the
Internet so that that didn't matter?

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 29 Oct 2008 09:30:05 +1300

> Internet so that that didn't matter?

I can't answer that question until I fully understand (a) what's
missing to make renumbering thinkable and (b) what infrastructure changes
would be needed to make renumbering unnecessary. Point (a) is why I think
the draft is worth writing, despite Olivier's point.

On 2008-10-29 07:38, Scott Brim wrote:

>   Unfortunately for me you didn't produce a magic

Sorry, our magician's handbook didn't have a spell for that ;-)

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 29 Oct 2008 09:37:08 +1300

On 2008-10-28 23:33, Olivier Bonaventure wrote:

[Several interesting paragraphs and then...]

> improve the "renumberability" of services and applications.

I wish I could be as optimistic. Unfortunately I believe we have
25 years of history to deal with (i.e. since the Berkeley socket
interface hit the streets) in which IP addresses have been treated
as de facto constants in services and applications.

   Brian

---

From: Scott Brim <[email-masked]>
Date: Tue, 28 Oct 2008 13:37:23 -0700

> the draft is worth writing, despite Olivier's point.

I see two ways to make renumbering unnecessary: (1) What Lixia calls
"separation" (map-and-encap or ubiquitous translation) or (2) GSE.  As

swb

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 28 Oct 2008 23:34:00 +0100

Brian,
> as de facto constants in services and applications.

With more recent APIs such as those described by Stuart Cheshire,
applications don't even see the IP addresses. Using applications built
on such APIs could be one way to improve their renumberability

 <http://www.stuartcheshire.org/quicktime/StuartCheshireIETF72.html>

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 28 Oct 2008 23:50:43 +0100

Scott,
> that's the critical issue area.

Let's consider the configuration of a firewall for example. Assume that
foo's firewall needs to have an access list for one prefix owned by bar.

Today, foo's sysadmin will lookup bar's prefix and will create an access
list containing bar's prefix.

Instead of using IP addresses directly, assume that bar publishes (with
a URL, an XML document or through the DNS) the list of its prefixes with
a unique name assigned to each prefix and a lifetime, e.g.
	 prefix1.bar = 1.2.3.0/24, ttl=1 day

Then, foo's firewall could be configured with prefix1.bar. foo's
firewall could contain a daemon that checks regularly bar's database to
updates its access lists. When bar needs to renumber, it will first
change its database to contain
	prefix1.bar = (1.2.3.0/24, 4.5.6.0/24), ttl=1 hour

foo's daemon will update its configurations by using the mechanism
described in

http://inl.info.ucl.ac.be/publications/preparing-network-configurations-ipv6-renumbering

Later, once renumber is finished, bar will change its database to contain
	prefix1.bar = 4.5.6.0/24, ttl=1 day

and foo's dameon will again update its access list

Of course, bar's database needs to be secure and if foo cannot contact
bar's database, it should assume that there are no changes.

I think that a similar approach could be extended for other services
where you need information about prefixes managed by a remote site.

Olivier

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 29 Oct 2008 11:53:03 +1300

>  <http://www.stuartcheshire.org/quicktime/StuartCheshireIETF72.html>

Certainly a name-based API confines the problem to the API and the
stack, but (as IPv6 has taught us) there are billions of dollars
invested in code that uses the address-based API, and the inertia
in that software is enormous.

   Brian

---

From: Robin Whittle <[email-masked]>
Date: Wed, 29 Oct 2008 11:23:10 +1100

Short version:  I can't quickly summarise this, but I have a
                tentative map of the design space regarding one or
                two solutions for "big" and "small" networks.

                Eliot's suggestion (as I understand it) to exclude
                larger end-user networks from the (presumably)
                renumbering based solution for smaller networks
                would still leave many "smaller" networks way too
                big for "routine" renumbering.

Hi Brian,

While I agree with what you wrote here, responding to Bill Herrin,

> I think there's definitely scope for two solutions.

I want to make the following points:

1 - The observations you make of smaller networks at present are
    probably generally true - such as of my little single IPv4
    address patch of the Net via an ADSL line, and 8 addresses I use
    in a hosting company in Dallas Fort Worth.  However, I think it
    would be best to avoid a dual solution arrangement which is
    predicated on particular patterns of usage which may not persist
    indefinitely.

2 - All currently potentially practical (no host changes) solutions
    (LISP, APT, Ivip, TRRP and I think Six/One Router) provide a
    single solution for all sizes of end-user network.

3 - While I am not completely opposed to there being two different
    solutions for "large" and "small" networks, however defined,
    I propose Ivip as a solution for networks of all sizes.

Also, I stand by my arguments for a single solution for networks of
all sizes:

in order to:

a - Avoid a situation where some small networks either don't adopt
    a solution (because there is no solution for big networks, and
    they intend to be big one day) or because they adopt an
    inappropriate solution (the big one) for the same reason.

b - Provide a simpler, more elegant, overall architecture with no
    debates about which solution each network should adopt.

c - To make a solution which is generally not dependent on patterns
    of usage, which are likely to change over the next few decades.

I am not saying that a dual solution is unthinkable.  Just that
there would need to be very strong arguments for it in order to
justify the extra work of devising the two solutions and coping with
the problems noted above.

There has been some suggestion of "no solution for big end-user
networks", such as by Eliot:

  http://www.irtf.org/pipermail/rrg/2008-October/000048.html

who thinks that their numbers - or at least the numbers of separate
BGP advertised prefixes - is within reasonable bounds.

I don't agree with the idea of a million separate prefixes being
within reasonable bounds of the scaling problem.

I guess it is always possible to nominate some criteria for choosing
whether an end-user network is "big", such that the long-term impact
of its BGP advertised prefixes (in the absence of adopting any
scalable routing solution) is deemed to be "acceptable".  My sense
is that any reasonable criteria of this nature would leave only a
handful of very large end-user networks, such as a few of the
biggest universities, corporations or whatever, and so relegate
virtually all end-user networks, including those of many
universities, government departments and corporations, to the class
of "small", or at least "not big enough so as to escape the
requirement that they adopt the scalable routing solution".

That would leave many genuinely large networks classified as
so-called "small".  It would not be practical to treat those the
same as a DSL-based SOHO network and expect them to do "routine
renumbering" when choosing a new ISP.

The design space in terms of end-user network size seems to be:

1 - Single solution for all sizes.  (My preference, and apparently
    the assumption behind LISP, APT, TRRP and Six/One Router.)

2 - Two separate solutions, such as:

    2a:  Core-edge separation, such as map-encap, for "big"
         networks, and something like SHIM6 and renumbering
         when choosing a different ISP (2 are needed for SHIM6)
         for small networks. (Note this precludes any solution
         for small IPv4 networks.)

    2b:  Any others?

3 - No solution for "big" networks, and one solution (I assume)
    for small networks.  I guess this is Eliot's position: no
    solution for some class of "big" networks, with another
    solution (map-encap, or SHIM6 and renumbering?) for all other
    networks, which are deemed to be "small".

(4 - A solution for big networks but no solution for "small"
     networks.  No-one is suggesting this - except perhaps
     for those who suggest IPv6-only solutions, excluding
     IPv4 networks, large and small.)

Since I think Ivip is a good solution - the best solution - for
networks of all sizes, I am happy to propose there be a single
solution for all sizes of networks.

While not wanting to suppress brainstorming, I think that if there
was going to be a serious proposal for two solutions, or for only
one solution for "small" networks and none for "big", then any such
serious proposal would need to involve a pretty robust critique of
Ivip and the other current "single solution" proposals: LISP, APT,
TRRP and Six/One Router (which is IPv6 only).

  - Robin

---

From: "William Herrin" <[email-masked]>
Date: Tue, 28 Oct 2008 22:15:17 -0400

Hi Robin,

That's not really true of TRRP. TRRP doesn't replace BGP, it
supplements BGP. At the end of the day, the individual network
operators decide which prefixes are allowed into whatever remains of
their respective BGP tables and which must use map-encap to transit
their network. Addresses remaining in BGP are usable for all purposes
they're now used for, plus providing TRRP exits and map servers.
Although I didn't describe it this way, the practical result should be
a balance affected as much by who is willing to pay to be in the BGP
table as much as it is by the overall table size.

In other words, the deployment of TRRP should, in theory, bring about
a natural, economically-driven balance between BGP-based large sites,
TRRP-based sites of various sizes and provider-aggregated sites of
various sizes.

Regards,
Bill Herrin

---

From: Scott Brim <[email-masked]>
Date: Tue, 28 Oct 2008 20:46:55 -0700

> I think there's definitely scope for two solutions.

Agreed.  Sites can look at their tradeoffs themselves and decide whether
decoupling from their upstream providers is worth the effort.
_______________________________________________
rrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/rrg

---

From: Eliot Lear <[email-masked]>
Date: Wed, 29 Oct 2008 07:58:08 +0100

Hi Robin,
>                  big for "routine" renumbering.

Again, let's separate the problem from the solution.  To put it another 
way, home and SMB networks really don't even have an option today to be 
multihomed (at least not at the network layer), and so whatever growth 
we're seeing today in the routing table is strictly that of larger 
institutions, and related traffic engineering and (to a lesser extent) 
disaggregation to protect large prefixes.

If one solution can fit all, all the better.  One is better than two, 
all other things being equal.  Whether they are or not is a fair question.

Regards,

Eliot

---

From: "Laurent Toutain" <[email-masked]>
Date: Wed, 29 Oct 2008 09:26:23 +0100

Hi Eliot,

Instead of renumbering, auto-configuration can be a more generic approach.
In SIGCOMM 2007 IPv6 workshop we proposed a solution for small sites
auto-configuration (
http://conferences.sigcomm.org/sigcomm/2007/ipv6/1569041157.pdf). This paper
proposed an extension to OSPF, but it can be easily adapted to RIP. It can
help also to solve one part of the multi-homing problem, i.e. route packets
toward the provider which assigned the prefix used in the source address of
the packet.

Laurent

On Wed, Oct 29, 2008 at 07:58, Eliot Lear <[email-masked]> wrote:

> multihomed (at least not at the network layer), and so whatever growth we're
> seeing today in the routing table is strictly that of larger institutions,
> and related traffic engineering and (to a lesser extent) disaggregation to
> protect large prefixes.
>
> If one solution can fit all, all the better.  One is better than two, all
> other things being equal.  Whether they are or not is a fair question.
>
> Regards,
>
> Eliot
>

---

From: "William Herrin" <[email-masked]>
Date: Wed, 29 Oct 2008 08:16:29 -0400

On Wed, Oct 29, 2008 at 2:58 AM, Eliot Lear <[email-masked]> wrote:
> To put it another way,
> home and SMB networks really don't even have an option today to be
> multihomed (at least not at the network layer),

Not correct. Anyone buying at least two T1s @ $600/mo or so each
qualifies for a /24 for multihoming from one of the ISPs and an AS#
from ARIN. Further, anyone with a legacy IP address block (pre-1997)
is also able to announce a block, even with just one T1.

Occasionally (though this is rare) folks with less than a T1 are also
permitted by their ISPs to participate in BGP. I have a quote from a
tunnel provider to do so for $300/mo, tunneling the resulting block to
my two home broadband links.

While most hobbyists aren't going to spend $1200/mo on Internet
connectivity, that's well within the budget of most small offices.

My point is: if we solve the problem for small offices without
disrupting or removing BGP, we'll at least minimally have an
intermediate solution where BGP growth can be constrained by moving
the dividing line between what size network is large enough to
announce into BGP and what has to use the new solution.

IMO, such would be a more modest and attainable goal than trying to
replace BGP outright.

Regards,
Bill Herrin

---

From: Eliot Lear <[email-masked]>
Date: Wed, 29 Oct 2008 13:28:40 +0100

Bill:

> On Wed, Oct 29, 2008 at 2:58 AM, Eliot Lear<[email-masked]>  wrote:
>    
>> To put it another way,
> from ARIN.

They're going from $25 - $90 or so single homed (what most pay) to $1200 
multihomed plus a BGP capable router plus the cost of the ASN, and 
that's only for failover.  If you want to actually make use of optimal 
routing, how much will that router cost?  But all of that is NOTHING in 
comparison to the costs of the expertise needed to manage such a service.

> announce into BGP and what has to use the new solution.

Who said anything about removing BGP?  Certainly not me.  Not sure what 
you mean by "disrupting."

Eliot

---

From: Scott Brim <[email-masked]>
Date: Wed, 29 Oct 2008 07:06:04 -0700

On 10/29/08 5:16 AM, William Herrin allegedly wrote:
>> To put it another way, home and SMB networks really don't even have
>> an option today to be multihomed (at least not at the network
>> layer),
> 

But the main point is whether small networks are managed differently
enough that they could be treated differently.  Multihoming is one
factor, renumbering is another, and there are certainly others.
Regardless of where you draw the line, the architectural point is that
some edge sites of the Internet can be treated as appendages on their
upstream providers, while others must be treated as independent
entities.  No?

> disrupting or removing BGP, 

or in some cases installing it!

>                             we'll at least minimally have an 

What the architecture should do is _allow_ that line to be moved, flexibly.

Scott

---

From: "William Herrin" <[email-masked]>
Date: Wed, 29 Oct 2008 12:18:02 -0400

On Wed, Oct 29, 2008 at 8:28 AM, Eliot Lear <[email-masked]> wrote:
> multihomed plus a BGP capable router plus the cost of the ASN, and that's
> only for failover.  If you want to actually make use of optimal routing, how
> much will that router cost?  But all of that is NOTHING in comparison to the
> costs of the expertise needed to manage such a service.

Hi Eliot,

A Cisco 2811 with extra RAM capable of handling full BGP for two T1s
costs about $2500. BGP-capable routers don't get expensive until you
need one that can do both BGP and a few hundred mbps of traffic. As
for expertise, the two most common configurations (primary/fail and
balanced share) could be easily commoditized with the instructions:
Here's your preconfigured router for the two ISPs you selected. If one
ISP ever acts flakey, pull the T1 cord.

The hobbyist without any expertise is going from the sub-$100 range to
$1200. Or rather, for the most part he isn't because his Internet link
isn't worth that much to him.

The small business is going from $600 to $1200, which isn't so big a jump.

> Who said anything about removing BGP?  Certainly not me.  Not sure what you
> mean by "disrupting."

My understanding is that APT replaces BGP entirely within its islands
from the outset, Ivip replaces BGP following a transition period and
LISP retasks BGP to carry routing for only the core ingress and egress
nodes. Apologies to the authors if I have that wrong; it has been a
long time since I read any of the three proposals and I could easily
be crosslinking other discussions in my head. Can you point me to
documentation that clarifies the situation?

Anyway, a retasking of BGP would qualify as "disruptive" since BGP
would cease to be usable for its original purpose of providing routing
for enduser-level addresses.

Scott,

I suspect that's mostly a red herring. It takes two to communicate. No
matter how I manage my network, I only control half that process. When
the agency I'm contracting for wants my source IP address to put in
their firewall, I don't have the luxury of saying, "Gee, I really
don't manage my network that way."

Regards,
Bill Herrin

---

From: Lan Wang <[email-masked]>
Date: Wed, 29 Oct 2008 12:30:04 -0500

Bill,

I'm just responding to your following statement about APT, so I  
removed the other text.

> documentation that clarifies the situation?
APT does not replace BGP.  In fact, BGP is still used by routers in  
ISPs to compute their routes to each other.   APT just defines how to  
distribute the mapping table among default mappers and how border  
routers communicate with the default mappers to obtain the mapping  
information.  The ITR still use BGP to figure out how to reach the  
ETR for the destination.

Lan

---

From: Robin Whittle <[email-masked]>
Date: Thu, 30 Oct 2008 13:43:22 +1100

Hi Bill,

Further to:

   http://www.irtf.org/pipermail/rrg/2008-October/000079.html
   http://www.irtf.org/pipermail/rrg/2008-October/000080.html

as with APT (and I think LISP, Six/One Router and your own TRRP
proposal), Ivip doesn't replace BGP in a global sense.

If every user network which was not on PA space from their ISP
adopted Ivip SPI (Scalable PI) space by renting it from some MAB
(Mapped Addres Block) company, then ISP networks would still all be
using BGP.

If an end-user network currently has conventional BGP-managed PI
space, and relinquishes this so they can use rented SPI space within
a MAB of some MAB company, then as far as this end-user network,
Ivip is "replacing BGP", because the network never needs to use BGP
after this.

If an end-user network with conventional BGP-managed PI space
converts their prefix (or prefixes) into a MAB (or MABs) then they
are still using BGP, since each MAB is a single advertised prefix in
BGP.  However, they can split the MAB into as many micronets as they
like, which gives a great routing scaling advantage compared to them
splitting their prefix into multiple separately advertised prefixes.

  - Robin

---

From: Robin Whittle <[email-masked]>
Date: Fri, 31 Oct 2008 12:05:27 +1100

Hi Eliot,

> disaggregation to protect large prefixes.

I agree - and I think we all agree - that most of the growth in the
DFZ routing table, currently doubling every 4 years, is from end-user
networks doing as you say.  The remainder is ISP growth - which I
think we accept as a natural, proper and generally sustainable load
for the BGP system to handle.

However my understanding of the RRG charter is not just to find a
scalable routing and addressing solution for existing PI end-user
networks - to eliminate some or all of the burden they will place on
the DFZ - but to also provide a scalable solution to the multihoming
and portability needs of millions (hundreds of millions or billions?)
of smaller networks.  These are networks which don't exist yet, or if
they do exist, don't have PI space and would generally be unable to
get it.

"Portability" is my word, since I think keeping one's address space
while choosing another ISP is the only reliable, acceptable, approach.

However, I agree that if the network only has a single IPv4 address -
such as a SOHO running from a cable modem, fibre or DSL service - it
is probably not too hard to renumber it.  If it was IPv6, I guess this
would be a /64 - or perhaps a /48.  For multihoming, the network would
have two /64s, such as one from the DSL service and one from the cable
modem service.

Still, as Bill Herrin wrote:

     No matter how I manage my network, I only control half that
     process.  When the agency I'm contracting for wants my source
     IP address to put in their firewall, I don't have the luxury
     of saying, "Gee, I really don't manage my network that way."

this supposed ease of renumbering a "small" network ignores wherever
the addresses turn up in other systems, such as ACLs as Bill
mentioned, or perhaps in DNS systems not directly controlled by the
network administrator.

Let's say I run a web server from my home network for the local
Neighbourhood Watch - then every time I get a new ISP, with new
address space, I need to get someone else to change the IP address in
the Neighbourhood Watch's DNS.  (Maybe I am mistaken - is there
another way of doing this?)

Also, with IPv6, I could in principle have a bazillion servers at
home, each with their own public IP address.  Changing over all the
DNS settings for those would be error-prone and painful, especially if
the DNS system was not something I ran on my own server.

With SHIM6 and IPv6, I need two ISPs - so changing ISPs is arguably
going to happen twice as often as with IPv4 portability only (no
multihoming), where I only need one ISP at a time.

> all other things being equal.  Whether they are or not is a fair question.

I agree.

 - Robin