Subject: Re: [RADIR] FWD: Re: Please review: draft-narten-radir-problem-statement-01.txt

From: Jason Schiller <[email-masked]>
Date: Mon, 10 Mar 2008 16:14:37 -0500 (EST)

Danny,

I would like to get a better idea of what sort of text wrt iBGP you think
this document is lacking.

> I still believe this document needs an entire section
> devoted to iBGP implications.  I.e., the fact that if there's
> indeed a problem the first place things are going to
> break is going to be in the iBGP core of the most densely
> connected networks.
>
> The reason it's going to break there first is because of the
> richness of interconnection, both external (e.g., section 5.1)
> and internal, and perhaps more importantly, as a result of
> architectural constraints imposed by internal BGP routing
> architectures that result in anywhere from 2 to 20x the
> number of BGP paths in the internal BGP routers as opposed
> to PE devices.

You are right to point out that the BGP architecture does play a role in
the number of paths in the network.  I believe there are a number of
aspects to this, but am not sure inclusion of text about this problem
space helps define the problem.

For example in a flat iBGP mesh, all of the routers will have all of the
eBGP learned paths.  In the event that some eBGP routes are multi-homed,
with equally good metrics, then all of the routers in the iBGP full mesh
will have one path for each eBGP connection point for a given multi-homed
route.  

Hierarchy can cause some reduction in path information.  All of the
multi-homed paths at one level of the hierarchy will be exchanged, but
external to that level of the hierarchy, only one best path will be
exchanged.  If the majority of the multi-homing occurs within a single
hierarchal domain, then there will be significant path reduction outside
of that domain.  If the majority of multi-homing occurs in different
hierarchal domains, then there will be a large impact in the level of
hierarchy above where the multi-homing occurs.  In the lower levels of the
hierarchal there will be a path reduction as the maximum number of paths
will be the number of eBGP connection points for that route, plus one best
path advertised by the hierarchal domain one level up.

Hierarchy can also cause an increase in paths depending on the number of
routers that lie on the boundary.  Take route reflection for example, if a
client has two route reflection servers then the client will necessarily
have two paths for ever route that is learned via iBGP, with a possible
addition on another path for every eBGP connection point for this route on
the local router.  Thus multiple route reflection servers will create as
many paths.

Confederation BGP is slightly more complicated.  Of all the routers that
sit on a confederation boundary and eiBGP peer with each other, one will
have the lowest IP address.  That one router with the lowest IP address
will iBGP with all the other routers in the same sub-AS, including the
other eiBGP speaking routers.  Thus all of the eiBGP speaking routers will
learn paths through eiBGP and through iBGP.  Since the iBGP neighbor has
the lowest IP address, then the other routers will not announce the eiBGP
learned paths into the iBGP.   In the opposite direction, there will be as
many paths as there are eiBGP neighbors that have a lower IP address than
all of the other eiBGP speaking neighbors in the same sub-AS.

Aggregation of non-multi-homed destinations can help ease the problem.  In
the event that that hierarchy is built in the forwarding path, it may be
possible to aggregate some of the single homed destinations, by simply
drawing the traffic to an aggregate.  In the even that all entry points to
the hierarchal routing domain are equally good for all destinations, then
a simple aggregate can be announce for all more specific singlely homed
destinations.  However if some destinations are closer to one entry point,
then the addresses must be distributed accordingly, or sub-optimal
forwarding will result.

Certainly as you point out, the problem will occur in the highest levels
of the BGP hierarchy.  Certainly as you point out, the greater the
multi-homing in different routing domain one level down in the hierarchy,
the more paths that will be sent into the highest level of the
hierarchy.  Would a paragraph stating as much be sufficient to meet what
text you think is missing?

>
> There are network topology and routing architecture
> considerations, as well as protocol modifications (e.g.,
> route reflection advertisement rule sets) that can have a
> considerable impact on both churn and scale internally
> as well.

As far as solutions go, I believe we tried to avoid that topic in this
draft.  I do agree that people should carefully consider the impact of a
given BGP architecture on their network, and that part of the solution
space might include carefully crafting a network.s BGP architecture to
meet requirements while minimizing path explosion.  

However, on the other hand, I think it would be unreasonable to expect
that people could construct networks where the route-reflection routing
domains or sub-ASes are sufficiently small to prevent multi-homing, have
entry points that are equally good for all destinations, and number all
static customers out of a unique subnet for each routing
domain.  Furthermore, each pair of routing domains will be aggregated by a
pair of routers, such that for N routing domains, there will be (the
square root of N) + 1 levels of hierarchy.  

> Furthermore, one could imagine a security considerations
> section that discussed a whole slew of topics, from Transport
> layer issues to prefix filtering, update authentication and
> stability concerns.

While I beleive security is an important requirement for any routing
protocol, I think these issues may be out of the strict scope of defining
the routing and address problem wrt the RIB and FIB size issues.  

__Jason

==========================================================================
Jason Schiller                                               (703)886.6648
Senior Internet Network Engineer                         fax:(703)886.0512
Public IP Global Network Engineering                       [email-masked]
UUNET / Verizon                         [email-masked]

The good news about having an email address that is twice as long is that
it increases traffic on the Internet.

> ------- Forwarded Message
> 
> From: Danny McPherson <[email-masked]>
> To: Olaf Kolkman <[email-masked]>, Thomas Narten <[email-masked]>
> Cc: IAB IAB <[email-masked]>,
>         The Internet Engineering Steering Group <[email-masked]>
> Date: Mon, 25 Feb 2008 05:57:33 -0700
> Subject: Re: Please review: draft-narten-radir-problem-statement-01.txt
> 
> 
> On Oct 24, 2007, at 11:14 AM, Olaf M. Kolkman wrote:
> 
> >
> > Folk,
> >
> >
> > draft-narten-radir-problem-statement-01.txt has been published in  
> > October 9.
> >
> > This document targets to be _the_ problem statement. At some point  
> > this will need to go to the community, but, before defining and  
> > taking the next steps (which probably involves IETF last call under  
> > Russ' sponsorship) it would be good to have review from these two  
> > groups.
> >
> > Please assess if this document would ready for IETF last call or if  
> > not,  where further discussion should take place.
> 
> I still believe this document needs an entire section
> devoted to iBGP implications.  I.e., the fact that if there's
> indeed a problem the first place things are going to
> break is going to be in the iBGP core of the most densely
> connected networks.
> 
> The reason it's going to break there first is because of the
> richness of interconnection, both external (e.g., section 5.1)
> and internal, and perhaps more importantly, as a result of
> architectural constraints imposed by internal BGP routing
> architectures that result in anywhere from 2 to 20x the
> number of BGP paths in the internal BGP routers as opposed
> to PE devices.
> 
> There are network topology and routing architecture
> considerations, as well as protocol modifications (e.g.,
> route reflection advertisement rule sets) that can have a
> considerable impact on both churn and scale internally
> as well.
> 
> Ignoring this entirely in the document, when it's clearly
> where the most pain exists today, isn't reasonable.
> 
> Furthermore, one could imagine a security considerations
> section that discussed a whole slew of topics, from Transport
> layer issues to prefix filtering, update authentication and
> stability concerns.
> 
> I have mentioned these issues on RRG and other lists in
> the past, though it's quite likely it's been lost in the noise.
> 
> -danny
> 
> ------- End of Forwarded Message
> _______________________________________________
> RADIR mailing list
> [email-masked]
> http://www.ietf.org/mailman/listinfo/radir
> 

_______________________________________________
RADIR mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/radir

---

From: Danny McPherson <[email-masked]>
Date: Mon, 10 Mar 2008 16:04:31 -0600

> square root of N) + 1 levels of hierarchy.

Yes, I'm aware of all the above.

In addition, where things like network design have direct implications
on scale of the routing system, from which internal BGP speakers higher
in the hierarchy have 10-20x the number of paths that PE devices do, be
they as a result of RRs within a cluster with unique cluster IDs,  
client-client
reflection as opposed to fully meshed client groups, implementations  
that
model overly liberal specifications that distribute extraneous data  
(e.g.,
RR spec updates), effects of memory, CPU and other churn because of
suboptimal update packing (e.g., from extraneous transitive attributes),
and basic iBGP route operation itself.

Ignoring this and concerning ourselves only with what functions external
entities impose on a local routing domain is disingenuous, in particular
when this is the place where things are going to break first - NO DOUBT,
hardware, CPU, memory, churn, IO, etc. are most gated, and the only
place operators have any actual control today.

> the routing and address problem wrt the RIB and FIB size issues.

I'm not sure we agree on this, given the insecurities in today's
routing system that are oft deferred "because the routing system
is to large or to complex".  In particular, add SIDR, IRR-generated
inter-provider policy and other solutions to the mix, much less
Transport Layer security for BGP, with KMART, or TCPAO, or
MD5, and then there's soBGP, SBGP and the like.  Deployment,
or lack thereof, and applicability of all of these relates directly to
routing scalability.

-danny