Subject: [RRG] Geographic aggregation-based routing is at odds with reality

From: Robin Whittle <[email-masked]>
Date: Fri, 18 Jul 2008 11:37:20 +1000

I think that geographical-based addressing arrangements, such as
those proposed in the recent thread by Heiner Hummel and Iljitsch
ban Beijnum) are not worth considering further because:

   To the extent that routing scalability depends on geographically
   determined assignment of address space, this is completely
   incompatible with several fundamental needs of providers and
   end-user networks:

     1 - That organisations who have been assigned address space
         should be free to use it at various sites, and these
         organisations are frequently global.

           (Otherwise, each branch of an organisation - and
            there could be hundreds at the granularity required
            by the geographic aggregation system - will want to
            get a large slab of address space, to cope with the
            potential for future expansion.)

     2 - Since the scalability which geographic aggregation
         supposedly must depend on routers forwarding packets
         in part or in whole according to their destination
         (source too??) address, this is incompatible with
         the need of organisations to have packets flow along
         paths which are determined by their business relationships.

           (It is assumed that the Internet's routing and addressing
           system should not require any organisations to have a
           business relationship or handle each other's packets
           simply because they are in some kind of geographic
           proximity.)

        Bill's challenge to Heiner illustrates point 2 nicely:

          http://psg.com/lists/rrg/2008/msg01815.html
          http://psg.com/lists/rrg/2008/msg01829.html

    3 - Organisations need to choose who they connect their networks
        to according to various criteria which are at odds with
        geographical aggregation, including being free to create
        links to distant networks.  Scenarios include:

          a - Redundant paths to cope with (geographically) nearby
              failures and points of congestion.

          b - Similarly, paths (such as by a fibre link, not
              tunnelling through the Net) which enable packets
              to travel whilst meeting security and policy needs.

                (For instance, for security - not through any
                given country or company.  Also, to meet local
                Internet censorship, anti-terrorism etc. laws,
                it may be necessary to make links which avoid
                certain countries.  Encryption is not a proper
                solution, and security can be damaged just
                by analysing traffic patterns, even if the
                contents cannot be deciphered.)

          c - Efficient traffic handling within global private
              networks which nonetheless use public address space.

Having the scalability of the Internet's routing system depend on
assigning addresses according to geographical location - implicitly
with forwarding of packets being dependent upon those addresses - is
completely incompatible with the business, policy, security and
efficiency requirements of the great majority of providers and
end-user networks.

Geographical aggregation is the sort of thing which looks good on
paper, but will never be acceptable in the real world.

  - Robin

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 18 Jul 2008 14:00:00 +1200

> paper, but will never be acceptable in the real world.

In any case, it isn't forbidden, and never has been. There's
absolutely nothing to stop the ISPs in Klingonville grouping
together to set up a local IXP, which gets a short prefix from
the RIR, and then uses it as a metro area prefix for all
local customers, with whatever policies and configurations
are needed in the IXP-connected routers.

    Brian

---

From: Ricardo Oliveira <[email-masked]>
Date: Thu, 17 Jul 2008 19:16:07 -0700

Depends on how you do it. i think geo aggregation has been  
consistently associated w/ bad design because of the bad examples  
proposed in the past, that negleted the fact that the first thing to  
route by is policy (local_pref is the first thing to check in bgp  
decision process).  after fixing this, it can look good both in paper  
and in the real world ;), e.g.
http://www.cs.ucla.edu/~rveloso/papers/giro.pdf

besides aggregation, geoinfo is usefull for fault detection, TE,...  
e.g. some isps tag routes with geographical info in entry points so  
customers(and isps themselves) can improve their route selection
cheers,

--Ricardo

---

From: "William Herrin" <[email-masked]>
Date: Thu, 17 Jul 2008 22:49:51 -0400

> ban Beijnum) are not worth considering further because:

Hi Robin,

Let's be cautious about taking an absolute position against geographic
aggregation. Let me relate to you an experience I had a year or so
ago:

At the Democratic National Committee, our upstreams were handled by a
pair of Cisco 6503 switches with Supervisor 2 cards. We were an
origin-only AS, a leaf node in the BGP system.

Here's an important fact about the Sup2 card: it implements the FIB
with a TCAM capable of holding 244k entries. What size did the global
BGP table pass last year? You guessed it. And what happens when the
TCAM fills up? The least specific routes (starting with 0.0.0.0/0)
fall out of the TCAM and have to be process-switched by the main CPU
using a radix tree lookup. So the box with multiple gigabits of
hardware-switching capacity gets very slow very quickly.

My solution to the problem was, in fact, geographic aggregation.
Optimal routing to the pacific rim is not a priority for the DNC. The
target audience is primarily domestic. So I looked up the /8's for
APNIC and squashed them into a few large routes that preferred first
one path and then another if a few "trigger" routes I defined should
happen to drop out. Brought my FIB down to about 180k entries, well
inside the Sup2's limit.

If IOS supported performing this sort of calculation automatically
when it builds the FIB from the RIB, that approach could accommodate
many origin-only ASes like the DNC almost indefinitely. Doesn't
particularly help the core with the two prefixes the DNC announces
though.

All of that having been said, Heiner's approach runs counter to the
economic demand and Iljitsch has his work cut out for him if he wants
to try to demonstrate that additional geographic-centric address
assignment doesn't chase diminishing returns.

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked] [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004

---

From: Robin Whittle <[email-masked]>
Date: Fri, 18 Jul 2008 13:08:31 +1000

I am replying to Brian and Ricardo.

Hi Brian,

> are needed in the IXP-connected routers.

The above sentence, when properly interpreted in the context of the
rest of my message, means:

   Ideas to achieve a scalable routing and addressing system by
   re-engineering the routing system to work scalably with addresses
   which are assigned according to the restrictions of "geographical
   aggregation" is the sort of thing which looks good on paper, but
   will never be acceptable in the real world.

This is my opinion, of course.  I am stating it as if it is a fact
because I am sure it is true.  I am frustrated at the way
geo-aggregation keeps emerging in this discussion, which I believe
should concern proposals which have the potential to be compatible
with business, security and policy reality.

Hi Ricardo,

How could a scalable routing system which depends upon geographical
aggregation achieve its goals if there are also connections and
address usage which does not respect whatever geographical rules are
set to support this architecture?

I imagine some new architecture could "improve" scalability, such as
perhaps:

However, if you only have 50% of the Net using the rules, then the
scalability improvements won't apply to 50% of the packets, 50% of
the addresses, routes or whatever.

I am not sure that it would ever be adopted by 50% of providers and
end-user networks anyway.  What would be the benefits for early
adoptors of a new architecture which restricts their addressing and
routing systems in this way?

But even if the new routing technologies and the new geographic
restrictions on what IP address a network could have were adopted by
50% of networks, I don't see how this would solve the routing
scaling problem.

Cutting the problem in half wouldn't be good enough.  We need some
factor of 10 or more (ideally hundreds, thousands or hundreds of
thousands) scaling improvement to cope with the desired growth in
end-user networks with their own portable, multihomable, address
space.  This growth is desired and required by the end-user networks
themselves.

How do you counter my specific critiques that geo-aggregation's
requirements on addressing and paths taken by packets are at odds
with business, policy and security requirements?

 - Robin

---

From: Ricardo Oliveira <[email-masked]>
Date: Thu, 17 Jul 2008 20:40:22 -0700

> the addresses, routes or whatever.
And this applies to all new designs that involve readdressing or  
changes in routers.

> with business, policy and security requirements?
I recommend you to read the paper for that, again, all depends on how  
you define "geo-aggregation".
Cheers,

--Ricardo

---

From: "Tony Li" <[email-masked]>
Date: Thu, 17 Jul 2008 22:43:31 -0700

|In any case, it isn't forbidden, and never has been. There's
|absolutely nothing to stop the ISPs in Klingonville grouping
|together to set up a local IXP, which gets a short prefix from
|the RIR, and then uses it as a metro area prefix for all
|local customers, with whatever policies and configurations
|are needed in the IXP-connected routers.

http://tools.ietf.org/html/draft-li-ispac-00

Tony

---

From: "Tony Li" <[email-masked]>
Date: Fri, 18 Jul 2008 00:47:57 -0700

|This is my opinion, of course.  I am stating it as if it is a fact
|because I am sure it is true.  

Good to see that you're keeping an open mind.  ;-)

|I am frustrated at the way
|geo-aggregation keeps emerging in this discussion, which I believe
|should concern proposals which have the potential to be compatible
|with business, security and policy reality.

Well, unfortunately, we don't all live in my reality.  ;-)

Over in my reality, geo-aggregation has some substantial benefits (and
drawbacks).  The real key with geo-aggregation is the scope that the
aggregation happens on and the strictness with which the rules are applied.

The scope of geo-aggregation can vary widely.  One could, in extreme cases,
have geo-aggregation down to the city block level.  At the opposite extreme,
one could have continental aggregation.  [If one exists in Vint Cerf's
reality, one might even argue for planetary aggregation.  ;-)]

Similarly, one can be very relaxed with the aggregation rules, simply
distributing an aggregate at some boundary and filtering internal more
specifics and allowing more specifics to be originated outside of the
abstraction naming boundary.  Or, one can be draconian and insist that there
be no exceptions to the aggregate.

If we take for example a continental aggregation with relaxed rule keeping,
you can see that prefixes within the continent are cleanly absorbed by the
aggregate.  This allows the site to migrate within the continental providers
without renumbering and without any burden to those outside of the
continent. 

If routing optimality becomes an issue, having an abstraction action
boundary that is even one or more hops removed from the abstraction naming
boundary can further improve the situation.

The issue with this approach is again one of cooperation: those folks
performing the maintenance of the aggregation are NOT those that benefit
from it.  The up side is that if everyone were cooperative (you scratch my
back, I'll scratch yours), then mutual aggregation would distribute both the
costs AND the benefits.  If providers could be enticed to enter into such a
multilateral (and informal!) agreement, we could make significant progress.

Regards,
Tony

---

From: Robin Whittle <[email-masked]>
Date: Fri, 18 Jul 2008 18:22:32 +1000

Hi Tony,

I have some understanding of what you wrote:

> The issue with this approach is again one of cooperation: those
> folks performing the maintenance of the aggregation are NOT those
> that benefit from it.  The up side is that if everyone were
> cooperative (you scratch my back, I'll scratch yours), then
> mutual aggregation would distribute both the costs AND the
> benefits.  If providers could be enticed to enter into such a 

but I don't see how these approaches are going to cause a new system
to be adopted to the very high degree we need in order to solve the
routing scaling problem.

Whether it is making IPv4 scalable or introducing something new and
incompatible (IPv6 or anything else) we need to make the new system
so attractive in terms of "immediate" benefits to adoptors,
including the earliest adoptors, that it is adopted by the great
majority of providers and end-user networks.

"Immediate" means benefits in months, not years.  Ideally there will
be benefits the moment the new system is turned on.  The benefits
must clearly outweigh the costs (financial, risk, disruption,
unfamiliarity) etc. for most providers and end-user networks -
including especially those who adopt it first - otherwise it will
never be widely enough adopted.

If your position is that a scalable routing and addressing solution
based on geographic aggregation of address space can only be widely
adopted if most or all adoptors (or at least most or all initial
adoptors) do so out of altruism, then I would say this sort of
solution is a non-starter.

Communism would be great - if only the great majority of humans had
the required high principles, altruism, focus on the long-term good
of fellow humans etc.

How could any company with shareholders justify adopting some new
geo-aggregation scheme, which provided no direct benefit to the
company or its customers, and which involved substantial investment,
risk etc.  The shareholders neither know nor care about:

  http://bgp.potaroo.net

and its ill-effects on the Net in general, other perhaps than the
fact that it costs them a little more in general to run however many
DFZ routers they have.

  - Robin

---

From: [email-masked]
Date: Fri, 18 Jul 2008 04:22:07 EDT

In einer eMail vom 18.07.2008 03:38:22 Westeuropäische Normalzeit schreibt  
[email-masked]:

100 % d'accord.What they should not be free to do is to combine the address  
with a wrong geographical location (btw, I have different solutions in mind  
which  require either more or less accuracy wrt geographical  location.  

2 - Since the scalability which  geographic aggregation
supposedly must  depend on routers forwarding packets
in  part or in whole according to their destination
(source too??) address, this is incompatible with
the need of organisations to have packets flow  along
paths which are determined by their  business relationships.
I support this need of organisations( combine the inter-domain-ly derived  
and geographically organized topology with the entire intra-domain topology even 
 if the latter one spans the entire globe; enable inter-domain multipath as  
extensively as demonstrated on my website, of course while taking care that  
preferences can be made according to business relationships)

c - Efficient traffic  handling within global private
networks which nonetheless use public address space.
All of this can be supported much better than so far.

Geographical aggregation is the sort of thing  which looks good on
paper, but will never be acceptable in the real  world.
This is a clear statement. My answering statement is this:
IETF was the first to do route computation but  is meanwhile far  behind. We 
could do internet routing as perfectly as Google map can  compute a path from 
New York to L.A. 
What it takes is a clean vision ( which I think I have), the  proper 
computation technology (which partly exists for 20 years) and a common  process to 
develop a new Topology Aggregating Routing Architecture (TARA).

Heiner

---

From: [email-masked]
Date: Fri, 18 Jul 2008 04:47:49 EDT

In einer eMail vom 18.07.2008 05:08:49 Westeuropäische Normalzeit schreibt  
[email-masked]:

Eliminating the scalability problem once and forever is just a side effect  
(just like that problem will never arise for postal delivery no matter how many 
 new residential areas are going to be built in the future).
 
More to counter:
"One touch only forwarding" ! Multihoming ! Mobility ! real-time traffic  
balancing ! New ways to do Network Management ! Safest  (politically) routing ! 
...
 
Heiner

---

From: Robin Whittle <[email-masked]>
Date: Fri, 18 Jul 2008 19:03:31 +1000

Heiner, Iljitsch and Tony,

It seems you support or are prepared to seriously consider a
solution to the scalable routing problem in the form of a
clean-slate approach which requires some, most or all addresses to
be assigned according to the geographical location of the provider
or end-user network.

Bill and I have argued that this is not worth discussing seriously
because we believe the resulting arrangement would be far too much
much at odds with the business (security, policy ...) needs of
providers and end-user networks.

Tony admitted that organisations adopting geo-aggregation (I assume
of a kind which would somehow solve the routing scaling problem)
would need to be motivated other than by the immediate benefits they
would achieve.

  http://psg.com/lists/rrg/2008/msg01855.html

What criteria of compatibility with existing human nature and
business practices do you have in mind when considering the
practicality of a proposal for solving the routing scaling problem?

Do you think a proposal should still be taken seriously by other
people if it can be shown to be at odds with current and likely
future organisational needs?

If we could change humanity so as to make the needs and behaviour of
organisations accord with the current or future architecture of the
Internet, then we could no-doubt solve the problem much more simply
than by introducing a new addressing scheme and new routers etc.

Just upgrade the necessary subset of humanity so that those who run
end-user networks (and who use them and pay for them) are happy
about not having portable, multihomable space.  Then the demand for
PI space would diminish and the problem would be solved.

LISP, APT, Ivip, TRRP and I think Six-One Router are all intended to
work with the current version of humanity.

For any proposal, there must be some level of requisite upgrades to
humanity and organisational behaviour at which you consider the
proposal to be unworkable.

I think geo-aggregation for the purpose of solving the routing
scaling problem is far over that line.

If you think the changes in organisational behaviour your proposals
require are not over that line, perhaps you could state some greater
degree of requisite change beyond which you would agree that a
proposal was unworkable and not worth discussing seriously.

  - Robin

---

From: "Tony Li" <[email-masked]>
Date: Fri, 18 Jul 2008 08:32:38 -0700

Hi Robin,

|If your position is that a scalable routing and addressing solution
|based on geographic aggregation of address space can only be widely
|adopted if most or all adoptors (or at least most or all initial
|adoptors) do so out of altruism, then I would say this sort of
|solution is a non-starter.

No Robin.  My point was an academic one: if it was possible to get some
degree of multi-lateral cooperation from providers, then geo-aggregation at
a very coarse level with relaxed rules is _technically_ feasible.  This has
been under consideration since we first introduced CIDR and it got ejected
even then, when there were far more providers in the room.

Moreover, it's not pure altruism: it's a coalition or co-operative
agreement.  Altruism would be pure sacrafice for no return.  This clearly
has a return, it's just not tightly coupled to the investment.

Note that this same type of mechanism is also going to be necessary to deal
with some of the non-architectural deaggregation that's seen today with
traffic engineering generated more specifics.

Now, it's been a very long time since the I*TF had the kind of critical mass
of operators necessary to begin that type of agreement, and it's been even
longer since we've seen folks that had the kind of vision to collaborate for
the long term good.  More's the pity.

That said, I'm actually welcome to see geo-aggregation discussed.  It's
certainly part of the technical solution space and if someone, as part of
the conversation, has a brilliant mechanism for driving such cooperation,
then it's all worthwhile.  While we're at it, we can also toss in creating a
market for routing slots.  Same set of issues.

Tony

---

From: Peter Sherbin <[email-masked]>
Date: Fri, 18 Jul 2008 14:00:48 -0700 (PDT)

a brilliant mechanism for driving such cooperation,

it is the monetary settlement based on the amount of the exchanged traffic where the originator pays for the delivery. nothing really new here

thanks,

Peter

--- On Fri, 7/18/08, Tony Li <[email-masked]> wrote:

> From: Tony Li <[email-masked]>
> Subject: RE: [RRG] Geographic aggregation-based routing is at odds with reality
> To: "'Robin Whittle'" <[email-masked]>, "'Routing Research Group'" <[email-masked]>
> Date: Friday, July 18, 2008, 11:32 AM
> Hi Robin,
> 
>

---

From: "Tony Li" <[email-masked]>
Date: Fri, 18 Jul 2008 14:21:01 -0700

Hi Peter,

Note that the analogy is poor, at best.  It's been shown multiple times that
settlements in an IP environment do not truly reflect usage (who pays:
Google, or the guy that did the search?).  Moreover, the data plane is
significantly different than the control plane.  If you take it in that
direction, do you do settlements of routes?  If so, who pays and why?  If
the sender, then what if the sender's path is not selected as best?  If the
receiver, then what encourages folks to aggregate?

Tony

|-----Original Message-----
|From: Peter Sherbin [mailto:[email-masked]] 
|Sent: Friday, July 18, 2008 2:01 PM
|To: 'Robin Whittle'; 'Routing Research Group'; [email-masked]
|Subject: RE: [RRG] Geographic aggregation-based routing is at 
|odds with reality
|
|a brilliant mechanism for driving such cooperation,
|
|it is the monetary settlement based on the amount of the 
|exchanged traffic where the originator pays for the delivery. 
|nothing really new here
|
|thanks,
|
|Peter
|
|
|--- On Fri, 7/18/08, Tony Li <[email-masked]> wrote:
|
|> From: Tony Li <[email-masked]>
|> Subject: RE: [RRG] Geographic aggregation-based routing is 
|at odds with reality
|> To: "'Robin Whittle'" <[email-masked]>, "'Routing 
|Research Group'" <[email-masked]>
|> Date: Friday, July 18, 2008, 11:32 AM
|> Hi Robin,
|> 
|> 
|> |If your position is that a scalable routing and addressing
|> solution
|> |based on geographic aggregation of address space can only
|> be widely
|> |adopted if most or all adoptors (or at least most or all
|> initial
|> |adoptors) do so out of altruism, then I would say this
|> sort of
|> |solution is a non-starter.
|> 
|> 
|> No Robin.  My point was an academic one: if it was possible
|> to get some
|> degree of multi-lateral cooperation from providers, then
|> geo-aggregation at
|> a very coarse level with relaxed rules is _technically_
|> feasible.  This has
|> been under consideration since we first introduced CIDR and
|> it got ejected
|> even then, when there were far more providers in the room.
|> 
|> Moreover, it's not pure altruism: it's a coalition
|> or co-operative
|> agreement.  Altruism would be pure sacrafice for no return.
|>  This clearly
|> has a return, it's just not tightly coupled to the
|> investment.
|> 
|> Note that this same type of mechanism is also going to be
|> necessary to deal
|> with some of the non-architectural deaggregation that's
|> seen today with
|> traffic engineering generated more specifics.
|> 
|> Now, it's been a very long time since the I*TF had the
|> kind of critical mass
|> of operators necessary to begin that type of agreement, and
|> it's been even
|> longer since we've seen folks that had the kind of
|> vision to collaborate for
|> the long term good.  More's the pity.
|> 
|> That said, I'm actually welcome to see geo-aggregation
|> discussed.  It's
|> certainly part of the technical solution space and if
|> someone, as part of
|> the conversation, has a brilliant mechanism for driving
|> such cooperation,
|> then it's all worthwhile.  While we're at it, we
|> can also toss in creating a
|> market for routing slots.  Same set of issues.
|> 
|> Tony
|> 
|> 
|> --
|> to unsubscribe send a message to [email-masked] with
|> the
|> word 'unsubscribe' in a single line as the message
|> text body.
|> archive: <http://psg.com/lists/rrg/> &
|> ftp://psg.com/pub/lists/rrg
|
|
|      
|

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 20 Jul 2008 15:06:16 +0200

> multilateral (and informal!) agreement, we could make significant  

I don't see that.

If I want to shrink my routing tables I can filter out any Canadian  
more specifics all over the world except in Canada, and instead put in  
an aggregate that covers the address space used in Canada.

(This requires that Canadian address space can be identified through a  
small number of prefixes and that I can easily make exceptions for  
more specifics to Canadian destinations that I can't reach through  
peerig inside Canada, but only elsewhere.)

If you are also a large network with customers in Canada, I don't care  
whether you also implement this aggregation or not. I can either do  
hot potato and give you the Canadian traffic close to the source and  
thus optimize for traffic flow, or filter those more specific,  
transport the traffic to Canada and then give it to you, optimizing  
for routing table size.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 20 Jul 2008 15:16:54 +0200

> scaling problem is far over that line.

Have you read my draft? The only thing that it _requires_ is that the  
RIRs start giving out address space with geography in mind. I don't  
think this requires upgrading much humanity or even business  
processes. Actually implementing the aggregation then has some  
business impact (going from hot potato to cold potato) but since every  
AS can decide for itself if and when to do this, that's simply a  
tradeoff: what's cheaper: transporting traffic over some distance, or  
buying bigger routers?

http://www.muada.com/drafts/draft-van-beijnum-multi6-isp-int-aggr-01.txt

BTW, has anyone considered the issue where large international  
organizations with many offices may want to advertise a prefix for  
each office rather than a single one for the entire organization?

---

From: Robin Whittle <[email-masked]>
Date: Mon, 21 Jul 2008 00:25:53 +1000

Hi Iljitsch,

Heiner's system doesn't cope with the needs of ISPs and end-user
networks to be able to apply policy at their routers, to control the
flows of packets to those networks for which they know will carry
them, according to contractual agreements:

Tony seems to admits that geo-aggregation is more of an academic
than a practical idea:

  http://psg.com/lists/rrg/2008/msg01860.html

and that it would require cooperation and business motivations which
are at odds with centuries of standard business practice.

  > This clearly has a return, it's just not tightly coupled to the
  > investment.

I don't have time to read your 2003 I-D fully:

but in your Limitations section, you write, in part:

    Since this scheme depends on geography for aggregation, it only
    works well for organizations that connect to the internet in
    locations that are close together. An organization with a
    network spanning multiple countries and connecting to the
    internet in all those countries isn't geographically
    aggregatable, and neither is an organization connecting to ISPs
    very far way, for instance by means of a satellite circuit.

    These types of organizations must choose address space falling
    within a geographic area that doesn't (fully) fit if they elect
    to use the type of address space this aggregation scheme uses.

!   This choice will have consequences on routing efficiency, and
!   when the infrastructure changes, the organization may need to
!   adopt a new address range to minimize the routing efficiencies
!   created by the change.

   Although the notion of geographic aggregation has been discussed
   many times within the IETF over the past eight years or so, this
   approach is generally believed to be flawed since the topology of
*  the networks that make up the internet and the interconnection
*  between those networks doesn't align well with geography. This is
*  indeed a problem, but it doesn't automatically make geographical
*  aggregation useless, it only makes it less effective.

   Since network topology is under constant revision, and as
?  networks get faster, the main disadvantage of "scenic routing"
?  (the increased speed of light delay) becomes more acute as
   bandwidth increases, it is certainly not unthinkable for the
   topology of the internet to align itself more with geography over
   time.

Why would any organisation want to have its choice of address space
constrained by your geo-aggregation idea?

What good does it do them, especially if the constraints change and
require it to change its address assignment when the topology changes?

Geo-aggregation is trying to shoehorn the entire addressing and
interconnection arrangements of all providers and end-user networks
into a set of constraints which do not suit those organisations, for
the sole purpose of enabling the Internet to be connected with
relatively simple routers.

I think this is exactly the wrong approach.

Organisations need flexibility in choosing their addressing
arrangements, in choosing who they connect to, in choosing whose
traffic they will accept for transit to another network etc.

Geo-aggregation forces humanity to bend to meet the needs of a
dumbed down routing system.

It is non-trivial thinking of a better routing and addressing
system, but some folks have tried: LISP, APT, Ivip, TRRP, Six-One
Router.

I am sure it will be easier to achieve any of these - which attempt
to meet actual needs of organisations - than to somehow change the
thinking of the great majority of organisations to make them happy
to accept the constraints of any geographically based addressing
regime.  You would also have to change the thinking of their
shareholders.

In order to solve the routing scaling problem we need to devise a
new technology which *most* - ideally 90% or 99% or whatever - ISPs
and end-user networks will *want* to adopt, within a few years.  The
reason they want to adopt it cannot depend upon the slow-moving
global situation of the DFZ routing table, since that only becomes
less troublesome after most people adopt the new system.  They need
short-term, robust, reasons for adopting it - reasons which make
sense to their current business priorities.

This is analogous to solving the global warming problem.  It is no
good making a solution which doesn't give people some benefit
immediately.  Only a few will adopt a new solution which is less
convenient for them simply because it does something to help stop
the greenhouse effect.

We need something which is highly attractive to the great majority
of end-user networks, and their ISPs - something which also enables
the Internet to scale well.  It is not good enough if only 30% or
70% of the end-user networks adopt it.  That would only cut the
problem by these amounts.  We need to cut the problem by several
orders of magnitude in order to enable the millions of end-user
networks (including small home and office networks) to have
multihomed, portable address space - without causing scaling
problems in the DFZ routing table.

Map encap schemes have faced criticism from some folks because they
would be hard to scale to billions of EIDs.  This means those
critics must think there is a need and demand for this number of
end-user networks with their own multihomable, portable address
space.  So if your system is to cope with this, it has to minimise
the scaling problem at any one router by many orders of magnitude -
and this requires almost ubiquitous adoption, not just a mish-mash
of geo-aggregation and its associated pattern of idealised
interconnects, alongside networks connecting in the current pattern
which is highly variegated.

I understand the ideal of geo-aggregation is to make life easier for
routers, but it has no direct benefit for adoptors over its costs
and restrictions - so it could never be adopted widely enough (90%
to 99%) to be a proper solution to the routing scaling problem.

 - Robin

---

From: "William Herrin" <[email-masked]>
Date: Sun, 20 Jul 2008 12:34:29 -0400

On Sun, Jul 20, 2008 at 9:06 AM, Iljitsch van Beijnum
<[email-masked]> wrote:
> I don't see that.
>
> If I want to shrink my routing tables I can filter out any Canadian more
> specifics all over the world except in Canada, and instead put in an
> aggregate that covers the address space used in Canada.

Iljitsch,

As you recall, I refuted a geoaggregation approach you described on
ARIN PPML last year using arguments similar to the recent discussion
with Heiner. I understand your current proposal is different: it takes
those arguments into account. To wit:

1. Assignment of addresses to organizations by some geographical basis
is mandatory. When assigning addresses, registries should choose from
a block reserved for the closest geographical match for that
organization.

2. Aggregation of routes by geography is strictly opportunistic.
Routes are distributed via BGP as now. If a particular router finds
that most or RIB entries for a given geography take the same exit from
the router, it may aggregate the routes that take that exit before
passing them on. Any routes which don't take that exit are passed on
as more-specific cutouts within the aggregated prefix.

Is that correct?

Take a look at http://bill.herrin.us/network/geoag.gif . Let's sever
the link between F and G. A backhoe took it out. This should sever
nodes E and F from nodes G and H. Until the link is repaired, those
two node sets should be unable to communicate with each other.

>From your plan, the RIBs look something like this:

Node B:
B local, propagate to F, A. C
A via A, propagate to F, C
F via F, propagate to A
E via F, propagate to A
C via C, aggregate in {right area}
D via C, aggregate in {right area}
G via C, aggregate in {right area}
H via C, aggregate in {right area}
{right area} via C propagate to A

Node C:
C local, propagate to B, D, G
A via B, aggregate in {left area}
B via B, aggregate in {left area}
D via D, propagate to B, G
G via G, propagate to D, B
H via G, propagate to D, B
{left area} via B, propagate to D, G

Node G:
G local, propagate to C, H (F link is down)
{left area} via C, propagate to H
C via C, propagate to H (F link is down)
D via C, propagate to H (F link is down)
H via H, propagate to C (F link is down)

Okay, so the first thing we notice is that we get no meaningful
benefit in nodes B and C. The RIBs there are as large or larger than
they are with plain old BGP.

The next thing we notice is that there is still a path from H to E:

H->G (left area known via G)
G->C (left area known via C)
C->B (left area was aggregated via B)
B->F (E known via F)
F->E (E known via E)

This is a permission violation!

There does not appear to be a reverse path. Packets can not travel
from E to H because neither F nor E have a route to {right area}.
However, F may be able to cheat by placing a static route to {right
area} via B,

If we restore the link between F and G, F *can* cheat by receiving
routes from G but not sending any routes. If F's traffic is
predominantly inbound, this will save him some money at B and C's
expense.

In today's networks, this would be prevented by installing traffic
filters at the peering points so that F may only send packets to B
that are destined to B or A. But because of the aggregation, that
filter is no longer effective. I suppose you could filter packets from
B to F that don't originate from A or B but that adds complexity and
consumes the same kind of router resources that the FIB does.

Anyway, my concern is this: if it's that close to a full breach of the
permission constraint in an 8-node system, what does it look like in a
50-node system?

There are some other weaknesses in the plan as well but lets look at
this one for now.

Regards,
Bill Herrin

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 21 Jul 2008 09:13:27 +1200

> organizations with many offices may want to advertise a prefix for each
> office rather than a single one for the entire organization?

I believe that large international organizations with many offices
have considered this issue with great care, and sadly in many
cases the answer has proved to be NAT (to provide local PA addresses
at each connection point to the Internet).

   Brian

---

From: Peter Sherbin <[email-masked]>
Date: Sun, 20 Jul 2008 16:12:09 -0700 (PDT)

Hi Tony,

> settlements in an IP environment do not truly reflect usage

it should. it is changing slowly where operators introduce usage based billing more broadly

> (who pays: Google, or the guy that did the search?).

the amount of traffice sent is what matters. If the Google server has sent tons of bits than Google has to compensate the provider(s). The settlement between Google and the searching guy is between two of them.

> the sender, then what if the sender's path is not
> selected as best?

If the speed of the delivery matters to the sender and/or the reciever than speedy routes clearly have a chance to become a differentiator for ISPs. The competition here will take care of pricing.

Thank you,

Peter

--- On Fri, 7/18/08, Tony Li <[email-masked]> wrote:

> To: [email-masked], "'Robin Whittle'" <[email-masked]>, "'Routing Research Group'" <[email-masked]>
> Date: Friday, July 18, 2008, 5:21 PM
> Hi Peter,
>

---

From: "Tony Li" <[email-masked]>
Date: Sun, 20 Jul 2008 20:09:53 -0700

Hi Iljitsch,

|If I want to shrink my routing tables I can filter out any Canadian  
|more specifics all over the world except in Canada, and 
|instead put in  
|an aggregate that covers the address space used in Canada.

True, but if you do that unilaterally, you affect the flow of traffic as
most specifcs draw the traffic around your network, and you incur the phone
calls from Canada, who will kindly ask you to cease and desist, eh?  ;-)

We discovered this when we tried to do proxy aggregation the first time: the
abstraction action boundary MUST be carefully and thoughtfully engineered to
avoid unintended consequences.  This requires cooperation.

|If you are also a large network with customers in Canada, I 
|don't care  
|whether you also implement this aggregation or not. I can either do  
|hot potato and give you the Canadian traffic close to the source and  
|thus optimize for traffic flow, or filter those more specific,  
|transport the traffic to Canada and then give it to you, optimizing  
|for routing table size.

Almost every carrier operates with hot potato in mind, out of vested
self-interest.

Tony

---

From: Brian Dickson <[email-masked]>
Date: Mon, 21 Jul 2008 00:17:07 -0400

> calls from Canada, who will kindly ask you to cease and desist, eh?  ;-)

Yeah. We're mad as heck, and we're not going to take it for very much 
longer. :-)

> avoid unintended consequences.  This requires cooperation.

The problem is not the generation of covering aggregates, but rather the 
suppression of more-specifics.

I think there is a very suitable boundary at which this can occur, 
without impacting anything non-local.

And that is, in the RIB->FIB stage.

We are much more concerned about the size of, and to a slightly lesser 
degree, the stability of, the FIB.

It would be nice to reduce the rate of change in the RIB, and to curtail 
the growth of the RIB, but except where
it leads to bloat in the FIB, the size of the RIB isn't really important 
per se.

If there were a suitable attribute (e.g. a new BGP attribute, or a 
well-known community) that could be applied
to arbitrary prefixes in the BGP table, which triggered their 
suppression only in the RIB->FIB, but not in any other place,
then shrinking the FIB could be done locally, with no impact at all 
outside of the local system.

BGP table entries would not be suppressed, and thus would continue to be 
propagated and usable beyond the local system.

This would differ considerably from the proxy aggregation method that 
suppresses *in bgp*, the more-specific prefixes.

And, having given it quite a bit of thought, I think this mechanism 
(coupled with appropriate allocation schemes and reasonable
means of calculating and generating tables of aggregates and RIB->FIB 
suppression maps) could be just the thing needed to achieve
the goals of the RRG. (The devil is in the parentheses, BTW.)

What about the allocation schemes?
It should be noted that one size never fits all, and knowing that in 
advance should inform any allocation scheme.

I think that schemes that acknowledge the scope of the recipient, are 
most likely to be useful, widely accepted, and
even self-documenting.

For example, hierarchy of scopes for geographic allocation pools, such as:
Planets {
  earth;
  continents {
    north america;
       countries {
          canada;
             provinces {
                ontario;
                   cities {
                      toronto;
                      [etc...]
                      }
                   counties {
                       durham;
                       york;
                       [etc...]
                      }
          usa;
          mexico;
    south america;
    [etc...]
    }
}

Requests would be directed to the region+scope that best matches the 
need of the requester, based on the expected routing policy.

A bunch of small satellite sites, would each get an individual entry 
from the appropriate pools.
A big entity that has its own internal infrastructure would get an entry 
from the pool of corresponding location and scope, e.g. city-wide, 
state-wide, continent-wide, or even global (for really big telcos or 
multinationals).

Whatever aggregation is able to be done, is a consequence on the 
eventual alignment that naturally occurs due to economics.

Over the long-haul networks, aggregation is generally going to occur 
because there are just fewer long-haul links, and the unit cost of large 
capacity is smaller, as a general rule.

The alignment of allocation pools with topology can be expected to be 
proportional to the difference in scope, because of the long-haul 
aggregation.
The number of paths to all destinations in a near-by city in the same 
state/province, are generally going to be orders of magnitude greater 
than the number of paths to all destinations in a remote city on another 
continent.

The relationship between scope size, and number of entities getting 
address space at that scope, globally, is likely to be order-of-magnitude.
If there are O(N) entities with global prefixes, there will likely be 
O(N^2) continental prefixes, O(N^3) country-level, O(N^4) 
province/state, O(N^5) city/county, and O(N^6) neighborhood prefixes.

This happens to have very nice properties, BTW. If you are in any given 
scope, you would expect to be able to aggregate, to some degree, 
non-local prefixes, and need to keep all the local prefixes.
Here, "local" has more than one meaning, but the number of local entries 
contributed by each scope can expect to be nearly constant, and very 
well behaved, the sum of linear terms.

Yes, maintaining this hierarchy of pools would have administrative costs 
- but any allocation scheme has such costs.
Minimizing the costs by taking advantage of existing schemes for 
parameterization, e.g. CLLI codes, NANP info, ISO country codes, etc., 
would certainly be advisable.
Making the aggregation itself a local activity, avoids many of the 
pit-falls such as cost/benefit asymmetry, changing the economic model(s) 
of the various players, etc.

The point is to make it possible, and to do so without making 
aggregation itself mandatory.

Not having to rely on anyone else to *do* the aggregation, but only on 
the RIRs to assign addresses (and specifically, PI prefixes) according 
to geography and scope, means the problems *can* be reduced to network 
engineering problems - hardware and software.

Brian

---

From: "William Herrin" <[email-masked]>
Date: Mon, 21 Jul 2008 09:52:37 -0400

Iljitsch,

Found the breach. Look at http://bill.herrin.us/network/geoag-i1.gif

With your new strategy, packets from H to E will take the aggregated
route through C and B instead of the longer (but permitted) route
through I and J.

Nice try though.

Regards,
Bill Herrin

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 21 Jul 2008 15:58:03 +0200

> Found the breach. Look at http://bill.herrin.us/network/geoag-i1.gif

Does this preempt your earlier message or add to it?

What do the green arrows mean?

Are the letters ASes or are the letters routers with the blue shapes  
their AS?

---

From: "William Herrin" <[email-masked]>
Date: Mon, 21 Jul 2008 10:30:05 -0400

> Does this preempt your earlier message or add to it?

Adds to it.

> What do the green arrows mean?

Green arrows indicate payment of money or some other consideration in
exchange for "Internet Service." The arrow points towards the ISP
receiving payment.

The permission rule is: For any packet transiting the network, there
must be a trail of green arrows to every node it transits from either
the packet's source or its destination.

In the diagram, nodes C and G are two "Tier 1" backbone providers: all
other nodes either pay them or peer with them.

> Are the letters ASes or are the letters routers with the blue shapes their
> AS?

The letters can be treated as ASes. The blue circles indicate
geographic regions in which addresses are allocated from an aggregable
block. According to your algorithm, the nodes outside a block should
opportunistically aggregate any routes it can for that block.

Regards,
Bill Herrin

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 21 Jul 2008 16:52:56 +0200

> The letters can be treated as ASes.

In this case the geographic aggregation doesn't really apply because  
apparently each AS only operates in one region...

> According to your algorithm, the nodes outside a block should

No, aggregation is local to an AS.

---

From: "William Herrin" <[email-masked]>
Date: Mon, 21 Jul 2008 10:59:42 -0400

> No, aggregation is local to an AS.

Then you only perform aggregation in origin-only ASes (like A, D, E
and H in the diagram) and don't attempt to aggregate in the others?

---

From: Xu Xiaohu <[email-masked]>
Date: Tue, 22 Jul 2008 10:39:01 +0800

> outside of the local system.

How about replacing the "local system" by "local autonomous system"? That is
to say, those PI prefixes with a new BGP attribute will be selectively
installed into FIB by several aggregate routers of one AS, and these
aggregate routers will advertise the aggregated routes with no-export
community and draw the traffic towards those more specific prefixes. The RIB
will not be suppressed and the most specific prefixes will be advertised
outside the AS. In a word, the FIB rather than RIB is to be shrinked.
In this way, there is no need for the ISPs in some location to be
interconnected, which is the known issue with geo-aggregation approach.

Xiaohu XU

---

From: "Scott Weeks" <[email-masked]>
Date: Mon, 21 Jul 2008 20:21:03 -0700

</lurk>

As a long time operator, this would be a very important aspect for acceptance of a New Thing.

--------------------------
Organisations need flexibility in choosing their addressing
arrangements, in choosing who they connect to, in choosing whose
traffic they will accept for transit to another network etc.
--------------------------

<lurk>

scott

------------------

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 22 Jul 2008 11:40:52 +0200

> constrained by your geo-aggregation idea?

That is a very good question. Our current answer is that the ISPs  
simply buy larger routers. Would you believe that 12 years ago you  
could (barely) run BGP with a full table on a Cisco 2500 with a 68020  
CPU (at 20 MHz, I believe) and 16 MB RAM? These days I can buy a  
multilayer switch that can forward packets at gigabit line rate on  
every port for less than what that 2500 cost, but if I want to run BGP  
in any meaningful way I have to pay much, much more money.

At some point, the answer is going to be: we don't care about your / 
24, we don't have room for it in our tables unless you buy 5 gigabit  
worth of traffic. At that point the limitations of geo aggregation  
won't look so bad.

And they're not that bad, really: mainly that when you physically  
relocate your network, you can't keep your addresses. This is true for  
most other addresses, too.

> require it to change its address assignment when the topology changes?

Addresses don't change when topology changes under normal  
circumstances. If you are in London with a satellite (or fiber)  
connection to an ISP in Vancouver, and then you change to an ISP in  
Milan, you should probably change your addresses. But my assumption is  
that when we have 10 million multihomers, 99% of those will use ISPs  
in the same or a neighboring city and this won't be an issue.

> into a set of constraints which do not suit those organisations,

No. There is no shoehorning. If aggregation is possible, we aggregate.  
If aggregation isn't possible, we don't.

> for the sole purpose of enabling the Internet to be connected with

> I think this is exactly the wrong approach.

If the shoehorning were necessary, it would still be a good approach:  
as a general rule, you don't want to optimize for the stuff you need  
to do once a month or less (changing ISPs) at the expensive of the  
stuff you have to do every 10 nanoseconds (routing table lookups).

> arrangements,

That statement nicely illustrates our past failures. Organizations  
shouldn't care about addresses, only about DNS names.

> in choosing who they connect to, in choosing whose

No argument there. With geo aggregation this is still possible,  
although of course some choices won't give you good levels of  
aggregation. A new option would be that you can easily buy or traffic  
engineer connectivity towards different regions from different ISPs  
because regions are now identified by aggregates.

> dumbed down routing system.

Hm, that's something I should think about on my flight to Dublin. </ 
irony>

> system, but some folks have tried: LISP, APT, Ivip, TRRP, Six-One

Shim6?  :-)

> would be hard to scale to billions of EIDs.

Well, that's just a challenge, if we do our work we should be able to  
solve that.

The real criticism is that we ALREADY have end-user names and  
infrastructure addresses: FQDNs and IP addresses. Apparently that  
wasn't enough. How do we know that a third naming layer will do the  
trick?

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 22 Jul 2008 11:44:54 +0200

> and H in the diagram) and don't attempt to aggregate in the others?

No, of course not. Aggregation in origin-only ASes is not the main  
issue, because those ASes don't necessarily have to run default-free.

The point is aggregation in the tier-1 service provider networks where  
default routes can't be used.

---

From: "William Herrin" <[email-masked]>
Date: Tue, 22 Jul 2008 11:07:48 -0400

On Tue, Jul 22, 2008 at 5:44 AM, Iljitsch van Beijnum
<[email-masked]> wrote:
> No, of course not. Aggregation in origin-only ASes is not the main issue,
> because those ASes don't necessarily have to run default-free.
>

Then what exactly are you aggregating?

If you're aggregating routes in a tier-1 then the routes that tier-1
passes on to its neighbors have been aggregated.

If aggregation is local then you're not passing aggregated routes to
your neighbors.

What about your plan modifies these two statements so they're no
longer mutually exclusive?

Regards,
Bill Herrin

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 22 Jul 2008 17:40:03 +0200

> Then what exactly are you aggregating?

Routes received from peers.

> If aggregation is local then you're not passing aggregated routes to

That's right, there's no propagation of aggregates to peers.

> longer mutually exclusive?

Suppose networks 100, 101 and 102 that operate in regions east and  
west. East has prefix 50/8 and west has 60/8. The networks have the  
following customers:

101: east 50.1/16 AS 1001
101: west 60.2/16 AS 1002
102: east 50.3/16 AS 1003
102: west 60.4/16 AS 1004

So 100 aggregates by allowing 50/8 and disallowing 60/8 routes in the  
east and allowing 60/8 and disallowing 50/8 routes in the west. There  
is an aggregate in the east for 60/8 pointing to the west and one in  
the west for 50/8 pointing to the east.

So the 100 router in the east has the following routing table:

prefix   next hop  AS path
50.1/16  101 east  101 1001
50.3/16  102 east  102 1003
60/8     100 west

When 100 east needs to reach 60.4.0.1 then it sends the packet to the  
west, and the router in the west knows that this packet needs to go to  
AS 102.

The aggregates are aggregates and therefore not announced to peers.

The other routes are learned from peers and therefore not announced to  
peers as per Gao/Rexford.

---

From: [email-masked]
Date: Tue, 22 Jul 2008 14:57:06 EDT

In einer eMail vom 22.07.2008 11:42:06 Westeuropäische Normalzeit schreibt  
[email-masked]:

enum-inverse ( E.164 ). Just kidding.
 
Of course, exploiting the geographical coordinates would be my  answer.
 
Obviously that can be done in many ways. If done just by building  
geo-prefixes i.e. without questioning the
real cause of the scalability problem, nothing would be gained. 

Heiner