Subject: [RADIR] comments on plenary slides

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 14:16:09 +0100

Please find attached planned slides for the plenary.
Comments appreciated. Do you agree with the
message?

Jari

_______________________________________________
RADIR mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/radir

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 20 Mar 2007 08:38:59 -0700

> Comments appreciated. Do you agree with the

Looks good.
I don't know if there is something useful to summarize from the RRG 
meeting that should be added.

    Erik

---

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 16:40:41 +0100

Thanks for the feedback.

Re: RRG. I was thinking that Brian should simply say it went very well
and that there was a lot of interest. (But this maybe too much
data to add to the slides.)

Jari

---

From: Jason Schiller <[email-masked]>
Date: Tue, 20 Mar 2007 13:03:33 -0500 (EST)

Slide 4 "Context (2) - Architecture"

I have no objections to this slide, but I'm not sure what exactly is you
point.  Are you saying:

1. current internetworking implements a reasonable amount of TE choices
   (and provider independence), but at the cost of extra state.  This
   extra state cannot continue to scale without negatively impacting the
   ISP operators in the DFZ.

2. The problem of the ISP operators in the DFZ can be mitigated by
   reducing the amount of state, but this will negatively impact the
   amount of TE choices (and provider independence).

3. Neither increasing the state in an unscalable way, nor a reduction
   in the amount of TE choices (and provider independence) is acceptable

4. We must find another architectural approach that allows for a
   reasonable amount of TE choices (and provider independence) without
   creating state in an unscalable way.

You might consider mentioning RFC-1887 which points out the need for RIRs
to assign large amounts of IPv6 space to ISPs and transit providers
(network providers) and for all down stream end-sites (network service
subscribers) to use a more specific route of a provider aggregate.

This also lead to the ARIN policy recommendation that aggregation is of
the utmost importance for good IPv6 stewardship.

Yet the operator community finds this unacceptable, there is a lack of PI,
and a lack of any other useful multi-homing solution for business class
end sites, so they have called for and pass a PI policy last April.  (Of
course I disagree)

The general impression of the operator community is that we need a
multi-homing solution now as it is a barrier to IPv6 adoption.  It is too
late to wait any longer.  Lets just solve this the way we know how through
de-aggregation.  And there are less than 900 prefixes in the IPv6 routing
table, we are growing at less than 100 prefixes a year, 1.1K prefixes in
two years is no big deal.  Lets just de-aggregate.  (Of course I think
this is short sighted and disagree).

================================
Slide 5 "Context (3) - Technical goals"

The slide reads:
* Solve end user problems
  - Connect to multiple ISPs w/ failover and load balancing

Suggest you change to
  - Connect to multiple ISPs w/ support for current TE capabilities

Why:

Load balancing is not the only desired TE configuration for end
sites.  Many business customers that are end sites have far more
complicated TE policies based on business practices, and cost of the links
where cost may be measured in money, performance, or some other metric.

For example:
ISP A may have good performance, but cost a lot of money.
ISP B my have poor performance, but cost less money.

Say there are three business customers cust1, cust2 and cust3.  cust1 may
prefer higher performance at the acceptable higher monetary cost, and thus
would prefer to use ISP A, and only use ISP B when ISP A fails.

cust2 may prefer the lower cost with an acceptable amount of performance
degradation.  However, in the event that the lower performance ISP B
completely fails, cust2 might prefer to then use the higher cost
link.  Thus cust2 would prefer to use ISP B and only use ISP A when ISP B
fails.

cust3 may prefer the best path (loosely measured by AS path length), where
sources closer to ISP A use ISP A and sources closer to ISP B use ISP B.

These are just simple cases, there certainly are more complicated
combinations of these, as well as dialing the amount of traffic up or down
on a particular link to avoid congestion.

===================================
Slide 6 "Context (4) - Scaling"

Don't forget all the more specific customer routes that an ISP carries,
and does not announce to the Internet table.  At one time UUNET had more
internal routes, than in the Internet table, so this problem can be as
big, if not bigger than the size of the Internet table.

====================================

Slide 11 "Is there a 'hot box' problem?"

First, there is an important point.  We all have to agree on what we are
talking about, and in what time frame.  When I entered this conversation
with Ops folks, the common response was "there are less than 900 prefixes
in the IPv6 Internet table, and its growing at less than 100 prefixes
a year...  lets just de-aggregate."

I think this is short sited.  We should not base the decision to
de-aggregate on the number of routing in the IPv6 routing table today,
when
IPv6 is mostly experimental.  We need to consider what the routing table
will look like when wide spread adoption occurs.

Based on my current calculations, if everyone did dual stack tomorrow, a
large ISP in the DFZ would have somewhere between 427,943 and 585,799
routes.  Projected out seven years and we are looking at 995,493-1,245,412
routes.  (the seven year projection may be a bit low due to negative
growth of internal routes over the last year due to some recent internal
aggregation from renumbering.)

Clearly wide spread adoption will not occur tomorrow.  So the question is
what is the soonest it might it occur?

To borrow an analogy from Thomas, this is a penguin problem.  When
penguins get hungry, they waddle over to the edge of the ice, and wonder
if a seal is waiting to eat them.  They wait and wait...  eventually one
falls in.  If it gets eaten they continue to wait.  If not they all jump
in.

I suspect there is no driving force to IPv6 adoption.  While IPv6 offers
some benefits over IPv4 such as autonumbering, with the exception of more
addresses there is no real functionality in IPv6 that cannot be found in
IPv4.  As of such IPv4 address exhaustion will likely drive IPv6 adoption.

If IPv4 exhaustion occurs around 2011, then at that time it will be
sufficiently difficult or expensive to get an IPv4 address.  This will
drive people to deploy new networks as IPv6 only.  If current IPv4 only
networks need to interact with these new networks (say they don't want to
loose customer base to a competitor), they will be driven to dual stack.

In short IPv6 will hit critical mass at some point, and this will lead to
widespread adoption.

Not only do the routers need to be build big enough, but they need to be
built in time, and not to be much more expensive than the current cost of
hardware.

For a large ISP having the router available in time, means 7 years before
it is needed.  Here is why:

IPv6 does not generate any new revue.  Old equipment has to be upgraded
to support IPv6 at line rate (in hardware).  But as IPv6 generates no new
revue, we need to do the upgrades without spending money.  That means let
other projects drive the upgrades (and get IPv6 for free) or time it to be
part of our normal equipment refresh.

We depreciate equipment over 5 years.  Lets say it take 6 months to do a
certification, and the a year and a half to do a network wide fork-lift
upgrade of the network.  That means the routers I can buy now need to be
able to support the routing table demands of 7 years from now.  And then I
have the same problem when those routers need fork-lift upgrading.

Until vendors say we can build it that big, that soon, for not much more
than current equipment, and can continue to say at least 7 years ahead of
routing table growth for the foreseeable future, we have a problem.

I haven't done scalability testing but am under the impression that the
T640 and CRS-1 top out at around 1M IPv4 FIB entries.  I expect it to be
less for mixed IPv4 and IPv6 based on past experience.

If IPv6 wide spread adoption happens by 2012 there will be over 1M
routes.  If IPv6 wide spread adoption happens by 2014 there will be close
to 1.3M routes.

That means if I replace my entire network with this gear today and
widespread IPv6 adoption occurs before 2014, then I will need to forklift
upgrade my network well before 2014.  The compression of the upgrade cycle
is not sustainable.

Business questions such as when should be upgrade the network and how much
of an investment should we make, or should we listen to PI /48
prefixes?  PA /48 prefixes from Peers? More specifics of PI and/or
PA?  All of these questions need to be answered in the short
term.  Decisions we make now will define our path forward.

So let me just say with my current gear that scales to around 350K and
700K routes in the FIB I am panicked.  And with current state of the art
equipment looking like it won't last a single refresh cycle I am
panicked.  And without vendors telling me they can build it big enough
to be 7 years ahead of my projected curve, without making it too costly I
am panicked.

So I'm not sure what is meant by:
"*Recent analysis indicates that for at least for two more generations of
  microelectronics (45nm, 32nm) this isn't a problem

   -looks like 5 to 10 years of growth; core router scaling seems to be
    dominated by line speed multiplied by functional complexity (in the
    forwarding path), not RIB/FIB size.

 * No need for panic"

As far as I can tell this is a problem that needs a solution.
===================

Slide 16 "Summary 2: Technical"

*Routing table growth is "only" an engineering issue
  -for at least one more generation of microelectronics

again I don't agree, see comments of slide 11

==================

I guess it doesn't matter if we disagree on what problems are driving a
solution.  I may think RIB and FIB size is problematic, others may not.  I
guess that actually doesn't matter as long as we agree to work on the
solution, and that solution provides a more scalable solution (there by
solving the RIB FIB problem).

I guess my only concern is if people come to believe that RIB/FIB size is
not a problem, that maybe people will treat it as a "goal" instead of a
requirment, which might open the door for a solution that solves the
other problems, but does not adress RIB/FIB size may be considered an
acceptable solution.

___Jason

==========================================================================
Jason Schiller                                               (703)886.6648
Senior Internet Network Engineer                         fax:(703)886.0512
Public IP Global Network Engineering                       [email-masked]
UUNET / Verizon                         [email-masked]

The good news about having an email address that is twice as long is that
it increases traffic on the Internet.

On Tue, 20 Mar 2007, Jari Arkko wrote:

> Date: Tue, 20 Mar 2007 14:16:09 +0100
> From: Jari Arkko <[email-masked]>
> To: [email-masked]
> Cc: Brian Carpenter <[email-masked]>
> Subject: [RADIR] comments on plenary slides
> 
>

---

From: Vince Fuller <[email-masked]>
Date: Tue, 20 Mar 2007 11:10:27 -0700

> Comments appreciated. Do you agree with the

Several minor comments:

1. The "additional reading" is almost unreadable on my laptop; it will be
   completely unreadable during the presentation.

2. The statements on slide "Context (2) - Architecture" are so abstract as
   to be a little bit obscure. I hope that you'll clarify what you mean with
   more concrete examples (i.e. use of "PI" and more-specifics for TE causing
   a "tragedy of the commons" global state growth pattern)

3. On slide "Context (4) - Scaling", I wouldn't state "1 million routes" in
   5 years as a goal unless you are are making suggestions for what hardware
   vendors should build. Adjusting the network architecture to allow it to
   scalably support millions of end-user networks is a goal; specifying that
   it should be done by building equipment to support super-linear growth of
   global routing state to 1,000,000+ routes in 2012 isn't (or shouldn't be).

4. You state at least once that ipv6 is orthogonal to the solution space. It
   is unfortunate that the designers of ipv6 completely neglected this
   problem. On the other hand, moving to a new address space and to a new
   protocol suite that still has very limited deployment may be an opportunity
   to change how that address space and protocol suite work. Two examples come
   immediately to mind: 1) minor changes to ipv6 to support GSE and 2) the
   division of the numbering space such that EID or locator use can be easily
   identified (think of a /3 set aside for EID use, for example). These sorts
   of things are easier to acheive with a protocol suite that is not yet
   widely deployed and an numering space that is not yet widely used and
   fragmented.

5. In some cases, you use the word "split" when referring to work in the
   id/locator separation space. It may seem a nit, but after reading some
   discussions on the RAM list, I now have a strong preference for the word
   "separation" since it describes the concept without implications for how
   headers/addresses/etc. wluld be "split" in design or implementation.

Hope this helps.

	--Vince

---

From: Vince Fuller <[email-masked]>
Date: Tue, 20 Mar 2007 11:14:49 -0700

> Comments appreciated. Do you agree with the

Oh, one other comment: on slide "Recent Activity (2)", I hope you'll say
a little more about the role, responsibilities, and expected activities of
RADIR... we're all dying to know :-)

	--Vince

---

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 19:21:48 +0100

I will have more comments later, but I'll get to the main

> solving the RIB FIB problem).

Exactly. The message that I get when talking to hardware engineers is that
they are not scared. The message that I get when talking to you or some
other
providers is that they are. My personal belief (and I think that is
shared by the
other RTG/INT ADs) is that there's more bang available from hardware than
what we understood to be available in Amsterdam. E.g., Scudder's
presentation
in NANOG. Note -- I'm talking about technology, but there's unfortunately
commercial issues in between. I just find it hard to believe that if we have
2M routing tables available today on DRAM and without new R&D 10M,
that by the time that we actually need them we couldn't build them in a
cost effective manner, if we wanted.

But this is of course all against the architectural principle earlier in the
slides. We're doing stuff where local events cause global effects.

At the end of the day, last six months have been spent in IMHO fairly
unnecessary debate about whether we have a problem. I would like to move
forward and state that the IETF will do something, and that we can improve
the architecture even if we don't get the final word on whether the sky
falls. That should be clear in the slides, and if its not lets modify them.

We could continue the debate, but frankly that takes more energy from
the effort than anything else. I think we have enough energy to move
forward.

> other problems, but does not adress RIB/FIB size may be considered an
That's absolutely off the table. If we are doing new work in id/loc
split it has
to address the routing community's requirements.

Bottom line: if we insist on making hard statements about hardware
infeasibility,
we will spend our time defending them to Ekr and people like him. We should
move the discussion to the improvement wrt. architectural principle.

Jari

---

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 19:26:34 +0100

Thanks for your comments, Vince! I will make changes based
on these.

Inline responses for the parts that needed a response.

>    global routing state to 1,000,000+ routes in 2012 isn't (or shouldn't be).
Ack.
>    widely deployed and an numering space that is not yet widely used and
I agree that this was unfortunate.

But here's the reality:

(a) all the solutions currently on the table work in backwards
compatible manner
(b) if you want to go back and change IPv6 in some other manner, you
will need much
      better proof that we have a problem. The community at the IETF
would be hard
      to convince.

In conclusion, I suggest dropping the historical argument (even if you
are right)
and moving on with (a).

Jari

---

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 19:27:27 +0100

> RADIR... we're all dying to know :-)
Ha! You are assuming I know :-)

(But helping with this is one of the activities.)

Jari

---

From: Thomas Narten <[email-masked]>
Date: Tue, 20 Mar 2007 19:51:29 +0100

quick comment on one point.

>       better proof that we have a problem. The community at the IETF

No, don't argue we have an urgent problem. There goes that big rathole
sound starting again that tends to get us nowhere.  :-(

Rather, we need a clear proposal with a credible transition proposal
given the reality today. Given a concrete proposal we can discuss
pros/cons/etc. and whether it makes sense to do this.

Don't forget, Microsoft Vista today ships with IPv6 enabled out of the
box.  Indeed, when both IPv4 and IPv6 are available, IPv6 is
"preferred" (their langauge). We can't just change the IPv6/TCP spec
and break Vista.

Thomas

---

From: Jason Schiller <[email-masked]>
Date: Tue, 20 Mar 2007 15:03:19 -0500 (EST)

> the effort than anything else. I think we have enough energy to move

That is fair, I just didn't quite get that impression from the slides.  

__Jason

---

From: Jari Arkko <[email-masked]>
Date: Tue, 20 Mar 2007 23:51:53 +0100

Jason,
> That is fair, I just didn't quite get that impression from the slides.  

Thanks -- and I have heard the same message from others so
the slides will be modified.

Jari

---

From: Jason Schiller <[email-masked]>
Date: Tue, 20 Mar 2007 17:57:10 -0500 (EST)

Jari,

I had two points in there and don't want you to loose the other one...

The first I think you heard loud in clear, about the debate on if there is
a short term problem, and what is short term and the like...

The second point is about slide 5 "Context (3) - Technical goals".  I want
to make clear that there are more valid multi-homing configurations than
just loading up all links.  

The slide reads:

Suggest you change to

Why:

For example:

__Jason

On Tue, 20 Mar 2007, Jari Arkko wrote:

> Date: Tue, 20 Mar 2007 23:51:53 +0100
> To: Jason Schiller <[email-masked]>
> Cc: Jason Schiller <[email-masked]>, [email-masked],
>      Brian Carpenter <[email-masked]>
> Subject: Re: [RADIR] comments on plenary slides
> 
> Jason,

---

From: Jari Arkko <[email-masked]>
Date: Wed, 21 Mar 2007 00:03:16 +0100

I got the rest of the comments, but haven't gotten to updating
the slides yet. I will get there, though.

Thanks for the feedback.

Jari

---

From: Jari Arkko <[email-masked]>
Date: Wed, 21 Mar 2007 10:03:44 +0100

>    ISP operators in the DFZ.

Yes. E.g., TE through more specifics causes global state.

>    in the amount of TE choices (and provider independence) is acceptable
Right.

>    creating state in an unscalable way.

Indeed.

> late to wait any longer. 
Right. That's why we are starting all the work.

In any case, we don't have time to go into the details. I'll just
mention the
PI example.

Got it.

Got it.

Jari

---

From: Jari Arkko <[email-masked]>
Date: Wed, 21 Mar 2007 10:08:28 +0100

New version of the slides attached. Thanks again for your
comments. This has been very helpful.

Jari