Subject: [Int-area] Please be more specific about the problem

From: Eliot Lear <[email-masked]>
Date: Thu, 22 Mar 2007 11:27:49 +0100

With the routing and internet areas working on ID/Locator split and the 
security and application areas taking notice, can we please be more 
specific about what problem we think we're solving?

Specifically:

    * Are we concerned about routing table size?  Is there a cache
      limitation? or
    * are we concerned about routing entropy on the wire?  and/or
    * are we concerned about other control plane entropy internal to the
      router associated with routing? or
    * are we concerned about something else?

Please provide data and analysis to support your statement.  A URL is 
sufficient.  If we do not agree that we have enough data and analysis, 
perhaps our next step should be to get the data and do the analysis.  
The lack of data and analysis in this week's presentations was extremely 
disturbing.

Eliot

_______________________________________________
Int-area mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/int-area

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 22 Mar 2007 03:46:26 -0700 (PDT)

> The lack of data and analysis in this week's presentations was extremely 

In considering protocol solutions to performance problems it is often very 
difficult to come to consensus because:

a. Performance data is often highly dependent on the implementations being 
studied, and therefore it is often unclear whether the conclusions hold 
universally. 

b. There are often multiple solutions available to the problem, some of 
which will perform better than others in given use cases.  Without 
agreement on the use cases, it can therefore be difficult to agree which 
solutions are superior, even if the performance data itself is 
unassailable (which is itself rare). 

The end result of this is that in practice, standards bodies often 
face the choice of waiting for completion of a protracted 
requirements/performance test development phase, or allowing work to 
proceed without a solid basis, under the theory that "the market will 
sort it out".  While the latter approach frequently results in solutions 
in search of a problem, this may at times be preferrable to deadlock.

---

From: Russ White <[email-masked]>
Date: Thu, 22 Mar 2007 08:12:13 -0400

>    * are we concerned about something else?

I suspect it is the last--that this problem really revolves around
traffic engineering based on injecting longer prefix matches, rather
than the routing table size, strictly speaking. The problem is, at the
core, I think, that competing organizations have competing traffic
engineering requirements, and the primary way to express those competing
traffic engineering requirements is through the injection of longer
prefixes.

I think I've heard it said someplace that only about 1/4 of the typical
ISP's table is Internet routes, while the remainder are internally
injected or 2547bis, etc. Perhaps someone can confirm or deny this?
Based on Vince's presentation, about 50% of the "internet" routing table
is actually also injected for traffic engineer. Hence, we get to the
point where an overwhelming proportion of any given routing table is
there to engineer traffic flow, rather than to provide reachability.

If this is true, this puts a new spin on the set of possible solutions.

1. Provide some different form of TE that uses something other than
longest match routing.

2. Figure out where the routes injected for TE lose their usefulness,
and remove them at that point.

3. Other (more creative) ideas....

I'm not certain how a loc/id split would "solve" this issue, if this is
the issue.... One of the reasons I'd like to see a better problem
statement is to better understand the actual problem.

o Is it TE?
o Is it table size?
o Is the table size being driven by TE?
o Is the table size being driven by sheer numbers of hosts?

And then, beyond understanding that, I think it's important to look at
the tradeoffs when considering a loc/id split. I'm not saying it's a bad
idea, we just need to compare the doo we're stepping in to the doo we're
actually in before taking that step.

:-)

Russ

- --
[email-masked] CCIE <>< Grace Alone

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 22 Mar 2007 13:39:00 +0100

> injected or 2547bis, etc. Perhaps someone can confirm or deny this?

It has been a while since I've worked at a really large ISP, but yes,  
the internal routing can be significant. However, there is no clear  
requirement that you carry all internal stuff everywhere, so the size  
of this stuff is (or: should be) more managable than the DFZ stuff.

> there to engineer traffic flow, rather than to provide reachability.

Increase in the routing table was 16 or 17 % last year, but the  
number of prefixes given out by the RIRs increased from 72661 to  
78154 or a 7.5% increase.

In the RRG meeting saturday it was said that 10M total routes in 2021  
is doable, which makes for a factor 25 or so increase in 14 years or  
24% per year. I guess we don't have a problem after all, and we  
certainly don't if we can get rid of this TE or "I'm too dumb to  
aggregate" stuff.

Until everyone and their little sister requests IPv6 PI space and  
starts multihoming in IPv6, of course.

---

From: Pekka Nikander <[email-masked]>
Date: Thu, 22 Mar 2007 18:47:01 +0100

Dave,

A lot of what you wrote (and I don't quote below) has indeed been  
said many times.  However, I think you left unsaid a few important  
things that have also been said many times in the past.

> So, the remaining problems [other than routing table growth and  
> traffic engineering] of having IP Addresses be used as identifiers  
> are not IP's.  They belong to other areas.  (Or, rather, Areas...)

What was repeated at least 7 times during this morning's INT are  
meeting is really a sad piece of history:

   The majority of today's applications use some derivative
   of the Berkeley Socket API.  The only thing the Berkeley
   socket API understands (in practice) is IP addresses.

   The TCP TCBs are also bound to IP addresses; something that,
   based on our current success, must have been a very good
   design decision back when it was made.

So, if you say that the other problems (like mobility or the use of  
IP addresses within applications and management systems) belong to  
other Areas, you are basically stating that both TCP, all  
applications, and most of our current management systems must be  
changed since they have been implemented wrongly from today's point  
of view.  Now, I might buy (but don't) that as an architectural  
statement, but realistically, if our job is to keep the Internet  
running even when the world is changing, I'm afraid that we do and  
will need backwards compatibility.  (See Section 3.3. of draft- 
nikander-ram-ilse-00.txt)

[Footnote:  The reason why I don't quite buy your argument even  
architecturally is that I believe that for most functions there is a  
natural level of granularity (e.g. subnet, host, or application)  
where they apply.  If we are able to implement them within the  
protocols that act on that granularity, we can gain in terms of  
simplicity, protocol efficiency, and security.]

> For example, multi-homing is naturally a transport problem -- and  
> it is fine if the solution is embodied as a shim above IP, since it  
> also can be characterized as a shim *below* transport...

I would only agree if there was a clear distinction that the IP layer  
works in terms of interfaces and it is the transport that works in  
terms of hosts (and not even then for site multi-homing).  But that  
is not really the case.  The legacy transport (TCP and UDP) have no  
concept of a host.  Architecturally (and I am referring back to  
Salzer), the architecture has been missing, from the beginning, host- 
granularity identifiers.

--Pekka Nikander

---

From: Danny McPherson <[email-masked]>
Date: Thu, 22 Mar 2007 12:25:18 -0600

On Mar 22, 2007, at 4:27 AM, Eliot Lear wrote:

> With the routing and internet areas working on ID/Locator split and  
> the security and application areas taking notice, can we please be  
> more specific about what problem we think we're solving?
>
> Specifically:
>

I suspect one way to look at this is to ask what's going to
break first?

While I agree that FIB scaling issues exist, I suspect many
of the folks most concerned today operate routers in the
networks closest to the "Internet Core".  As the denseness
of interconnection in the global routing system increases,
it's the internal network devices that are hit the hardest.

Everyone is looking at external paths and unique FIB
entries and there's clearly concerning growth there, but
from all the data I've seen it's the number of *paths* that
are internal to a given AS that are the most immediate
problem.

As I mentioned at the microphone in IDR, the number of
paths on 'PEs' in SP networks seems to be anywhere from
300-800k (~1.5 - 3x the number of unique prefixes in the
routing system), while in the core of the network the number
of paths on iBGP speakers is often 10-30x the number of
unique prefixes in the routing system (e.g., 3 'tier 1 ISPs'
polled yesterday had from 3-5 million unique paths on
'core' route reflectors; I had routers with 2 million paths
7 years ago).

This is largely dictated by the requirement that route
reflection and/or AS confederations are needed to avoid a
full mesh, and then one, or two, or three more are required
to provide redundancy, and then that external denseness
in interconnection increases and you end up with a ton of
churn and update duplication, driving CPU cycles, DRAM,
and even data<->control plane bandwidth problems.

Furthermore, functions like BGP update packing are
negatively impacted because of the number of unique
path attributes grows (e.g., originiator IDs, cluster IDs,
IGP metrics, MEDs, next hops, etc..).

Then we make protocol changes to optimize implementations
(e.g., BGP Route Reflection changes regarding whether
client knows it's a client and whether RR reflects routes
learned from clients back to that client) and the system
level problem only gets worse.

So you can either make things flatter, add more hierarchy,
modify the protocol to support new scaling mechanisms, or
design a new protocol -- all of the above introduce the
usual offshoots (e.g., TCP session state v. route oscillation
probability, etc.)

In short, the growth curve for internal BGP speaker paths
is much steeper than that of unique prefixes and general
RIB data, and is highly topologically dependent.

I believe there is opportunity here for short-term protocols,
implementation and operational gains.

FWIW, I believe that loc/ID split makes sense longer term,
as discussed in another email I posted here earlier.

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 23 Mar 2007 12:26:55 +0100

Eliot,

On Mar 22, 2007, at 11:27 AM, Eliot Lear wrote:
> Please provide data and analysis to support your statement.  A URL  
> is sufficient.  If we do not agree that we have enough data and  
> analysis, perhaps our next step should be to get the data and do  
> the analysis.  The lack of data and analysis in this week's  
> presentations was extremely disturbing.

If the available data is such that different reasonable people can  
arrive at different conclusions, what do you think the implication is?

Thanks,

--John