Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

From: Rob Shakir <[email-masked]>
Date: Wed, 7 Sep 2011 13:05:14 +0100

On 11 Aug 2011, at 22:40, George, Wesley wrote:

> Danny said,
> "Periodic updates of the entire routing table *with much larger and more updates* seems undesirable at best to me, particularly to 'reduce the vulnerability window for replay attacks' to 'days'."
> ---------------------------------------------------------
> I think that this is a small part of a much larger issue that has been bugging me since our meeting in Quebec City.
> After looking at Sriram's estimates around the necessary amount of memory, etc for the different algorithms, I'm bothered by what is being asked of the routing system in support of BGPSec.
> 

Hi All,

Wes' point here is really important I think, and it's a bit of a shame it hasn't got more focus. I stood up and mentioned scaling in Prague, then found myself making a very similar point again in Québec. As with Wes, I would prefer that this is not interpreted as a criticism of the goals of bgpsec, but rather is an operational perspective on the assumptions that are made for deployment of the protocols being developed in sidr.

The root assumption that we seem to be taking as read throughout such discussions is that at the point where routing infrastructure is expected to support the computational load that is demanded of it by bgpsec, there will have been sufficient growth in both the computational and memory resources that are available to these systems. Whilst I appreciate that cryptographic validation of the validity of a route can be performed on the edge system(s), I think there are two very key concerns here.

- Historically, do we see Internet routing systems matching the rate of general computational resource availability? My view on this is that we do not - there are plenty of edge systems out there that are relatively modern (and capable of routing tens of gigabits of traffic) that do not have a control-plane that is as powerful as some of the software devices that existed prior to it - as such, this means either this was a lower priority to deliver (which may well be true), or that it was not practical to do so. With increasing demands on control-plane resources of IP systems (be they Internet edge or otherwise) - there is surely some query as to whether having a large amount of memory or crypto offload hardware will be a priority over other features. The question Wes raised as to whether we, as a working-group, are happy with the assumption that this is the case, is very valid. It would seem to me that we are being somewhat prescriptive with prioritising what hardware vendors' solutions should look like, and what operator's scaling approaches will look like - without necessarily explicitly discussing why this is reasonable to do so.

- Are the assumptions that we are making regarding how those autonomous systems delivering an "Internet" product set are architected valid? There is some assumption (AIUI, please feel free to correct me) that we are looking at networks where we have explicit edge devices which can scale to meet the validation workload that we're putting on it. This would seem (again, to me) to be based around assumption that there are a relatively high number of less-dense edge devices. I would be interested whether it's the feeling of the WG that my reading of what we're assuming is correct - and if so, whether the WG feels that we can be prescriptive like this for how one should architect an Internet network.

I'm again very supportive of what problems bgpsec is trying to solve - and do not want to put a dampener on any of the excellent efforts that have gone into it - but think that we cannot wholly abstract ourselves away from the actual deployability of the solutions that are being developed. After all, we will not solve the problems that the WG is chartered to solve if the fruits of the labours herein are not actually deployed.

I am not sure of exactly how we progress discussing the scalability of the solution - but with Sriram's analysis, is it possible to discuss within this forum, and perhaps others concerned with the general scaling of IP routing elements, why the scaling considerations/assumptions of bgpsec are valid, and where a departure is made from the longer term view of scalability that is presented in existing IETF documents.

Thanks for reading this - and your consideration!
r.

---

From: Jakob Heitz <[email-masked]>
Date: Wed, 7 Sep 2011 09:12:41 -0400

While a router that performs BGPSEC may not be more expensive in 5 years than one that does not today, that is not relevant. A router that performs BGPSEC in 5 years will most definitely cost more to produce as well as cost more to run than a router that does not perform BGPSEC in 5 years.

So, a question for you Rob. Will your customers pay the premium for BGP security?

--
Jakob Heitz.

On Sep 7, 2011, at 5:05 AM, "Rob Shakir" <[email-masked]> wrote:

> _______________________________________________
> sidr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidr

---

From: Randy Bush <[email-masked]>
Date: Wed, 07 Sep 2011 15:36:58 +0200

lots of great opinions, but are there actually any measurements or
models?

randy

---

From: Rob Shakir <[email-masked]>
Date: Wed, 7 Sep 2011 15:48:10 +0100

> So, a question for you Rob. Will your customers pay the premium for BGP security?

Hi Jakob,

This is of course an interesting question - which comes down to the question of whether the threats that are being addressed by bgpsec are common-place. I definitely have customers that would pay a premium to mitigate this as a DoS vector, or malicious interception mechanism, but equally, have customers who would not, based on their current experience.

From what I have seen of the demand for origin validation at the current time, I would say that my personal opinion (and no dataset to support this, sorry) is that any willingness to pay a premium will grow relatively slowly. As such, this makes the point about trying to ensure that we have a deployable protocol that attempts to represent the smallest step change it can in terms of computational requirements more important to me - since this will mean that it is easier to begin deploying, and meeting the demand.

Kind regards,
r.

---

From: "George, Wesley" <[email-masked]>
Date: Wed, 7 Sep 2011 11:09:18 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jakob Heitz
Sent: Wednesday, September 07, 2011 9:13 AM
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

WEG] This question is a good one, as it makes the difference between a capex driver with no projected bottom-line improvement and a potential revenue stream. We've seen the "unfunded mandate" movie before. It's one of the reasons that IPv6 deployment took so long -
"yeah we have to spend $xxM to upgrade the network and our systems to support IPv6."
"Ok, how much extra can we charge for that?"
"Um...yeah, about that... but if we don't have it soon, our customers will leave us and we might run out of IP addresses..."
"I haven't had any customers ask me for it...when is soon?"
"Maybe 12-18 months?"
"Ok, let's do it next year then."
(lather, rinse, repeat for several years)

My guess is that the union of who is willing to pay for it and what they're willing to pay likely won't cover the SP's costs to implement it. Generally security is one of those things that either is critical, cost-no-object or is seen as optional if the threat of impact is low enough compared with the cost of avoidance/prevention. I think that the threat risk BGPSec is addressing is seen as pretty low by the vast majority of small to medium enterprises if they've never been hit by it. Heck, we've recently seen some very large companies get burned for not having properly invested in security in areas where the threat model was a bit more obviously risky (Sony)...
The sales pitch for BGPSec, especially in an incremental deployment model will have a lot of bearing on that perceived level of risk and our collective ability to mitigate it. If we're successful, maybe it becomes a sustainable investment. However, it's risky to assume that this will cover the added cost burden completely.

Wes George

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: "t.petch" <[email-masked]>
Date: Wed, 7 Sep 2011 17:26:06 +0200

----- Original Message -----
From: "Rob Shakir" <[email-masked]>
To: "Jakob Heitz" <[email-masked]>
Cc: "sidr wg list" <[email-masked]>
Sent: Wednesday, September 07, 2011 4:48 PM
> > While a router that performs BGPSEC may not be more expensive in 5 years
than one that does not today, that is not relevant. A router that performs
BGPSEC in 5 years will most definitely cost more to produce as well as cost more
to run than a router that does not perform BGPSEC in 5 years.
> This is of course an interesting question - which comes down to the question
of whether the threats that are being addressed by bgpsec are common-place. I
definitely have customers that would pay a premium to mitigate this as a DoS
vector, or malicious interception mechanism, but equally, have customers who
would not, based on their current experience.
>
> From what I have seen of the demand for origin validation at the current time,
I would say that my personal opinion (and no dataset to support this, sorry) is
that any willingness to pay a premium will grow relatively slowly. As such, this
makes the point about trying to ensure that we have a deployable protocol that
attempts to represent the smallest step change it can in terms of computational
requirements more important to me - since this will mean that it is easier to
begin deploying, and meeting the demand.

My own experience of promoting security (and for that matter resilience) is that
very few organisations are willing to spend until after disaster strikes.  And
when that happens, anyone without a solution ready is in trouble, so the onus on
us is to have specified a viable solution, the implementation of which is as
cheap as possible but no cheaper.  Then, when the evil empires turn to Internet
routing, as opposed to, say, e-mail, at least we can say that we did our part to
prevent it.

Promoting new functionality that turns straightaway into more revenue is the
easy part and it is rare for security to come in that category.

Tom Petch

---

From: "George, Wesley" <[email-masked]>
Date: Wed, 7 Sep 2011 14:57:21 -0400

-----Original Message-----
From: Randy Bush [mailto:[email-masked]]
Sent: Wednesday, September 07, 2011 9:37 AM
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

WEG] Please be more specific. What other models and measurements than the one already provided by NIST are you asking for? That model already tells me that BGPSec will significantly change the routing table scaling curve from today, with some variation on the steepness of the change possible based on variables and optimization. Your non-response to the concern about whether the situation articulated by 4984, & 6227 has changed in some material way as to make it no longer an issue for BGPSec is telling. Let's keep turning up the heat and hope that the frog doesn't notice until it's too late...

My guess from your response above is that you're looking for some anecdotal evidence of equipment lifetimes in real networks to support Rob's assertion that 5 years may not be a realistic event horizon, so I'll take a stab at that.

There are plenty of 15-year old routing platforms still in service, despite YFV's best efforts to make them obsolete.
Only recently has a combination of End of Support and a need for new features and higher density driven the oldest of the hardware (GSR E0, 1, 2, 4/4+, 10720, the 7200 and 7500 in Ciscoland) out of the network. Some of that stuff dated back to the early 2000s and was just removed in the last 12-24 months. Most accounting still assumes at least a 7-year depreciation cycle, and that seems somewhere between appropriate and optimistic, given that a lot of the gear was closer to 8 or even 10 years old at retirement.
On the route processor side, which is probably most pertinent to this discussion, upgrades were almost exclusively driven by the memory requirements of the routing table + the code footprint. RPs are either purchased with max supported memory so that they don't have to be touched again, (because touches are expensive) or they are upgraded periodically because the max has been increased. So the next replacement-level upgrade happens when the current max (4Gig in the last round of upgrades) isn't enough, which will take a while with just organic DFZ routing growth. My recollection was about a 6 year total replacement cycle. However, in hard-core bellhead beancounter circles, the concept of needing to invest in upgrades to a routing platform beyond adding raw port capacity every few years is totally foreign, because they have DMS-250 and DCS systems that have been in the network since Moses gave them the owner's manual on stone tablets and are still generating revenue with only occasional replacement parts. Something like BGPSec makes the discussion with the finance people all the more fun, because now we're having to justify spending significantly more on gear that does not directly translate to additional revenue port capacity, vs justifying "must do x to keep the network from falling over."
In zero-margin "race to the bottom" networks, that's not trivial.

Thanks,

Wes

---

From: "Smith, Donald" <[email-masked]>
Date: Wed, 7 Sep 2011 14:19:35 -0600

I would guess you would get a very low take rate on "secure bgp" even if you gave it away (no premium).
It would be interesting to see how many customers implemented MD5 for bgp peering sessions when it became freely available.
It wasn't very widely deployed the last time I heard.
It got a ton of push back on the NANOG list when I recommended it.

Ignorance is Bliss. "Bliss (Basic Language for Implementation of System Software) was a
systems programming language originally for the PDP-10 and DECsystem-20 written at CMU." Kevin Oberman RTD
[email-masked]

> Sent: Wednesday, September 07, 2011 7:13 AM
> To: Rob Shakir
> Cc: sidr wg list
> While a router that performs BGPSEC may not be more expensive in 5
> years than one that does not today, that is not relevant. A router that
> performs BGPSEC in 5 years will most definitely cost more to produce as
> well as cost more to run than a router that does not perform BGPSEC in
> 5 years.
>
> >> "Periodic updates of the entire routing table *with much larger and
> more updates* seems undesirable at best to me, particularly to 'reduce
> the vulnerability window for replay attacks' to 'days'."
> >> I think that this is a small part of a much larger issue that has
> been bugging me since our meeting in Quebec City.
> >> After looking at Sriram's estimates around the necessary amount of
> memory, etc for the different algorithms, I'm bothered by what is being
> asked of the routing system in support of BGPSec.
> >>
> >
> > Hi All,
> >
> > Wes' point here is really important I think, and it's a bit of a
> shame it hasn't got more focus. I stood up and mentioned scaling in
> Prague, then found myself making a very similar point again in Québec.
> As with Wes, I would prefer that this is not interpreted as a criticism
> of the goals of bgpsec, but rather is an operational perspective on the
> assumptions that are made for deployment of the protocols being
> developed in sidr.
> >
> > The root assumption that we seem to be taking as read throughout such
> discussions is that at the point where routing infrastructure is
> expected to support the computational load that is demanded of it by
> bgpsec, there will have been sufficient growth in both the
> computational and memory resources that are available to these systems.
> Whilst I appreciate that cryptographic validation of the validity of a
> route can be performed on the edge system(s), I think there are two
> very key concerns here.
> >
> > - Historically, do we see Internet routing systems matching the rate
> of general computational resource availability? My view on this is that
> we do not - there are plenty of edge systems out there that are
> relatively modern (and capable of routing tens of gigabits of traffic)
> that do not have a control-plane that is as powerful as some of the
> software devices that existed prior to it - as such, this means either
> this was a lower priority to deliver (which may well be true), or that
> it was not practical to do so. With increasing demands on control-plane
> resources of IP systems (be they Internet edge or otherwise) - there is
> surely some query as to whether having a large amount of memory or
> crypto offload hardware will be a priority over other features. The
> question Wes raised as to whether we, as a working-group, are happy
> with the assumption that this is the case, is very valid. It would seem
> to me that we are being somewhat prescriptive with prioritising what
> hardware vendors' solutions should look like, and what operator's
> scaling approaches will look like - without necessarily explicitly
> discussing why this is reasonable to do so.
> >
> > - Are the assumptions that we are making regarding how those
> autonomous systems delivering an "Internet" product set are architected
> valid? There is some assumption (AIUI, please feel free to correct me)
> that we are looking at networks where we have explicit edge devices
> which can scale to meet the validation workload that we're putting on
> it. This would seem (again, to me) to be based around assumption that
> there are a relatively high number of less-dense edge devices. I would
> be interested whether it's the feeling of the WG that my reading of
> what we're assuming is correct - and if so, whether the WG feels that
> we can be prescriptive like this for how one should architect an
> Internet network.
> >
> > I'm again very supportive of what problems bgpsec is trying to solve
> - and do not want to put a dampener on any of the excellent efforts
> that have gone into it - but think that we cannot wholly abstract
> ourselves away from the actual deployability of the solutions that are
> being developed. After all, we will not solve the problems that the WG
> is chartered to solve if the fruits of the labours herein are not
> actually deployed.
> >
> > I am not sure of exactly how we progress discussing the scalability
> of the solution - but with Sriram's analysis, is it possible to discuss
> within this forum, and perhaps others concerned with the general
> scaling of IP routing elements, why the scaling
> considerations/assumptions of bgpsec are valid, and where a departure
> is made from the longer term view of scalability that is presented in
> existing IETF documents.
> >

This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 07 Sep 2011 22:44:07 +0200

Hi,

Securing BGP really means transitioning from 100% distributed control of 
today's BGPv4 based Internet to new layer of Internet control to be 
build for BGPSec (for that matter for BGP Origin Validation too).

IMHO scaling aspects are serious, but are not the most serious reg 
discussion of acceptance of not for this new layer of control. And that 
is not something one can address by dedicated crypto engine processing 
or better PR CPU.

It would be IMHO very interesting to see more discussions in the 
community on the fundamental aspects of risks such new layer of control 
may bring or cause to the Internet as we know it today.

Rgs,
R.

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Wed, 7 Sep 2011 17:51:45 -0400

> From: [email-masked] [mailto:[email-masked]] On Behalf Of George, Wesley
> Sent: Thursday, August 11, 2011 5:41 PM
> 
-- snip --
>
> As far as where this leaves BGPSec, I don't know. Perhaps a solution comes in the form of
> some of us starting to champion the necessary work to move one or more of the better ideas
> from RRG into implementation so that the underlying scaling problem is being addressed in
> parallel. It should be quite possible to incorporate improvements to route security into
> whatever must already be done to improve scale, and even if not, these are two things that
> both share a barrier to deployment, especially incrementally, and would certainly benefit
> from being designed  and deployed in concert instead of in separate silos.
> 
> Thanks
> Wes George

Wes,

Sorry for the belated response -- I was away in India on vacation :) 
We had some discussion earlier on this list about
BGPSEC in conjunction with an RRG scalability solution (e.g., LISP).
(You might have seen these posts back in May, but just in case.)

http://www.ietf.org/mail-archive/web/sidr/current/msg02836.html 
http://www.ietf.org/mail-archive/web/sidr/current/msg02837.html 
http://www.ietf.org/mail-archive/web/sidr/current/msg02847.html 

These posts relate only to the above noted comment from you
(but not directly to a different question you've raised 
w.r.t. the concerns in 4984/6227).  

Sriram

---

From: Shane Amante <[email-masked]>
Date: Wed, 7 Sep 2011 19:35:56 -0600

Wes,

Excellent points, which I agree with.  One additional point below.

On Sep 7, 2011, at 12:57 PM, George, Wesley wrote:
> On the route processor side, which is probably most pertinent to this discussion, upgrades were almost exclusively driven by the memory requirements of the routing table + the code footprint. RPs are either purchased with max supported memory so that they don't have to be touched again, (because touches are expensive) or they are upgraded periodically because the max has been increased. So the next replacement-level upgrade happens when the current max (4Gig in the last round of upgrades) isn't enough, which will take a while with just organic DFZ routing growth.

Let's also not forget that all routing platforms deployed in the field /ARE NOT/ created "equal" with respect to the _potential_ for just Routing Engine upgrades, just to gain more CPU and/or DRAM.  Specifically, I'm referring to Router HW architectures where the Routing Engine is co-resident on the same physical LC, (sometimes the RE is a daughtercard, other times it is not), on a centralized Switch Fabric Module.  Up until now, the upgrade strategy to get faster CPU or more DRAM, from the vendors who have shipped these platforms, is to replace the entire LC (that means: CPU, DRAM (RE) _and_ the SFM) ... even if the chassis/system did not require the additional forwarding capacity offered by the new SFM.  Or, worse, sometimes said fabric upgrades cause "legacy" LC's to no longer be supported!!! -- hello whole chassis upgrade ...  Of course, this could change for future generations of platforms, but it doesn't solve for what's already deployed and continues to be deployed, today, in the field.

-shane

> My recollection was about a 6 year total replacement cycle. However, in hard-core bellhead beancounter circles, the concept of needing to invest in upgrades to a routing platform beyond adding raw port capacity every few years is totally foreign, because they have DMS-250 and DCS systems that have been in the network since Moses gave them the owner's manual on stone tablets and are still generating revenue with only 
> occasional replacement parts. Something like BGPSec makes the discussion with the finance people all the more fun, because now we're having to justify spending significantly more on gear that does not directly translate to additional revenue port capacity, vs justifying "must do x to keep the network from falling over."

---

From: "t.petch" <[email-masked]>
Date: Thu, 8 Sep 2011 18:04:44 +0200

----- Original Message -----
From: "Smith, Donald" <[email-masked]>
To: "'Jakob Heitz'" <[email-masked]>; "'Rob Shakir'" <[email-masked]>
Cc: "'sidr wg list'" <[email-masked]>
Sent: Wednesday, September 07, 2011 10:19 PM

I would guess you would get a very low take rate on "secure bgp" even if you
gave it away (no premium).
It would be interesting to see how many customers implemented MD5 for bgp
peering sessions when it became freely available.

<tp>
None, when first I became involved in BGP but then, 18 months later, it was
widely deployed as a result of a 'disaster' of the kind I alluded to in my
previous post.  I never
did hear directly what happened, and it may be an European affair that did not
impact America.  I think that I learnt most about it via the RIPE Routing group
but it was several years ago, and my memory of it is now quite vague.  But
it was a good example of nothing being done with the available technology until
the disaster happened, and then it was all hands to the pumps.

Tom Petch

</tp>

Ignorance is Bliss. "Bliss (Basic Language for Implementation of System
Software) was a
systems programming language originally for the PDP-10 and DECsystem-20 written
at CMU." Kevin Oberman RTD
[email-masked]

> > Thanks for reading this - and your consideration!

This communication is the property of CenturyLink and may contain confidential
or privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.
_______________________________________________
sidr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/sidr

---

From: Rob Shakir <[email-masked]>
Date: Thu, 8 Sep 2011 19:18:39 +0100

> My guess from your response above is that you're looking for some anecdotal evidence of equipment lifetimes in real networks to support Rob's assertion that 5 years may not be a realistic event hor…

I'd like to lend some support to Wes' message here, the situation described therein is something I've seen before at numerous operators.

Randy - do you think that with the modelling that NIST provided, it is possible for the WG to map (from a historical dataset of control-plane growth) why the time-scale envisaged for hardware to reach the level required for fairly ubiquitous bgpsec support is reasonable? This would seem to answer at least the queries raised here around whether the resource level envisaged is supportable within the quoted timeframe.

Many thanks,
r.

---

From: "George, Wesley" <[email-masked]>
Date: Thu, 8 Sep 2011 15:13:19 -0400

-----Original Message-----
From: Sriram, Kotikalapudi [mailto:[email-masked]]
Sent: Wednesday, September 07, 2011 5:52 PM
To: George, Wesley; sidr wg list
Subject: RE: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

Wes,

WEG] Sriram - thanks for the pointers, but all this really tells me is that I'm not the first one to point out a potential problem here. The issue with making any assumption of a solution means that we're essentially closing our eyes and hoping that someone else figures this out before the world ends. My point is that we have to take a more active role in bringing this back to the forefront of the discussion in IETF because we stand to be impacted by delays in finding a solution. If the timing is wrong, it ends up being gating to deployment of BGPSec. If the timing is right, it probably requires a lot of redesign work and additional investment, neither of which are particularly optimal. I'd prefer that we document up-front that there is a real concern here and that IETF needs to get moving on the scale problem, in some way other than endless debate over options within RRG. Even making a decision as to which direction to go (map and encap vs L/I split) would move a long way towards allowing other things like BGPSec to optimize their designs accordingly.

Wes George

---

From: Jakob Heitz <[email-masked]>
Date: Thu, 8 Sep 2011 15:46:28 -0400

A router that implements BGPSec will *always* use more memory
and more CPU that one that doesn't.

CPU
---
A router has a powerful CPU mainly to converge when it comes up
and when a neighboring router comes up or goes down. During normal
operation, it is largely unused.

If we reprioritise BGPSec, we could make it work. Sugestion:
during bringup, send routes without the BGPSec attribute.
Once converged, during low CPU usage, send them again with
the BGPSec attribute. Call this the first beacon.
When receiving routes, delegate the BGPSec
verification to a low priority task.

Memory
------
Because BGPSec operations do not affect convergence (we send
without BGPSec attribute to achieve convergence), it can use
slower, cheaper memory like a hard disk or flash disk.

Downside
--------
There will be a few minutes after router bringup when unverified
routes exist. Unverified routes can be given a lower preference.

On Thursday, September 08, 2011 12:13 PM, George, Wesley <> wrote:

>> As far as where this leaves BGPSec, I don't know. Perhaps a solution
>> comes in the form of some of us starting to champion the necessary
>> work to move one or more of the better ideas from RRG into
>> implementation so that the underlying scaling problem is being
>> addressed in parallel. It should be quite possible to incorporate
>> improvements to route security into whatever must already be done to
>> improve scale, and even if not, these are two things that both share
>> a barrier to deployment, especially incrementally, and would
>> certainly benefit from being designed  and deployed in concert
>> instead of in separate silos.   
> 
> Wes,
> 
> WEG] Sriram - thanks for the pointers, but all this really tells me
>  is that I'm not the first one to point out a potential problem here.
> The issue with making any assumption of a solution means that we're
> essentially closing our eyes and hoping that someone else figures
> this out before the world ends. My point is that we have to take a
> more active role in bringing this back to the forefront of the
> discussion in IETF because we stand to be impacted by delays in
> finding a solution. If the timing is wrong, it ends up being gating
> to deployment of BGPSec. If the timing is right, it probably requires
> a lot of redesign work and additional investment, neither of which
> are particularly optimal. I'd prefer that we document up-front that
> there is a real concern here and that IETF needs to get moving on the
> scale problem, in some way other than endless debate over options
> within RRG. Even making a decision as to which direction to go (map
> and encap vs L/I split) would move a long way to wards allowing other
> things like BGPSec to optimize their designs accordingly.            
> 
> Wes George
> 
> This E-mail and any of its attachments may contain Time Warner Cable
> proprietary information, which is privileged, confidential, or
> subject to copyright belonging to Time Warner Cable. This E-mail is
> intended solely for the use of the individual or entity to which it
> is addressed. If you are not the intended recipient of this E-mail,
> you are hereby notified that any dissemination, distribution,
> copying, or action taken in relation to the contents of and
> attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.

-- 
Jakob Heitz. x25475. 510-566-2901

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 08 Sep 2011 22:58:23 +0200

Hi Jakob,

> during bringup, send routes without the BGPSec attribute.

I think there is a significant risk that this would cause unnecessary 
Internet wide churn.

I would propose and alternative approach. To send the routes with 
whatever attributes sender decides to send then with and do it only once.

Up-on reception if indeed inline CPU processing is an issue on given 
platform store the BGPSec related attributes without any processing then 
when CPU get's idle .. verify them.

Hoping that only small fraction of all routes could be considered 
invalid you may later withdraw them.

Of course this only attempts to fix inbound processing issue. You can 
not do the same when signing the routes outbound.

---

It keeps me puzzled  why sidr is so stuck on BGP inbound solution as 
opposed on number of proposals which deliver the same results using out 
of BGP bound control plane verification.

Cheers,
R.

> On Thursday, September 08, 2011 12:13 PM, George, Wesley<>  wrote:
>

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Fri, 9 Sep 2011 09:46:14 -0400

-- snip --
>not do the same when signing the routes outbound.

Jakob:
Robert:

The protocol permits the ideas you are proposing -- see Section 4.2.
http://tools.ietf.org/html/draft-ietf-sidr-bgpsec-protocol-00#section-4.2

The key wording there is:
.... it is RECOMMENDED that if the BGPSEC speaker 
chooses to propagate the route that it
generate an update message containing the BGPSEC_Path_Signatures
attribute.  However, a BGPSEC speaker MAY propagate a route
advertisement by generating a (non-BGPSEC) update message that does
not contain the BGPSEC_Path_Signatures attribute. 

Also, the optimizations you mention are included/discussed 
in Section 5.4 of the draft-sriram-bgpsec-design-choices document:  
5.4. Temporary Suspension of Attestations and Validations
http://tools.ietf.org/html/draft-sriram-bgpsec-design-choices-00#section-5.4

Robert seems to be asserting that the router must always 
sign and propagate the update (if it selected a signed update for best path). 
However, the current protocol specification allows for dropping BGPSEC_Path_Signatures
attribute (potentially for cases when the route-processor is in overload state). 

Sriram

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Fri, 9 Sep 2011 10:59:27 -0400

>If the timing is wrong, it ends up being gating to deployment of BGPSec. 
>If the timing is right, it probably requires a lot of redesign work 
>and additional investment, neither of which are particularly optimal. 
>I'd prefer that we document up-front that there is a real concern here and that 
>IETF needs to get moving on the scale problem, 
>
>Wes George

The devil is always in the details. But I am thinking (hope not too simplistically)
that once you have a solution agreed upon and deployed for scaling the FIB, then
all it does is slow down (hopefully quite significantly) the growth of the
FIB and RIB sizes. It makes no changes to the eBGP protocol as such.
(Only adds the additional step of mapping look up at the ingress routers.)
It would appear that "BGPSEC protocol specification" by itself has no dependency 
on the FIB scalability solution (except that the potential impediments for 
BGPSEC deployment may be substantially alleviated -- benefiting from possibly
significant reductions in RIB size, # updates/beacons, and route-processor workload).
FIB scalability solution would ultimately reduce the cost of deployment
of BGPSEC, but the specifications of the two solutions/protocols need not be
intertwined. Having said that, I am all for full scale co-operation between the
two efforts -- much as you have suggested.  

Sriram

---

From: Randy Bush <[email-masked]>
Date: Fri, 09 Sep 2011 17:06:37 +0200

> Randy - do you think that with the modelling that NIST provided

i am not very confident in any modeling done to date and am not making
grand opinions until i, or others, have time to do credible measurement.

but i do have some thought about what i want to model, mostly stolen
from others

    due to design lead times and pricing, what will be on routing engine
    boards in three to five years is whatever is available as commodity
    today.  i.e. what does a hot westmere chip give us?  with a cheap
    crypto chip?

    as a vendor friend says, if ipv6 deploys, insha allah, we're gonna
    be upgrading those routers to do real v6 forwarding.  if it does not
    deploy, you will be deploying massively bigger boxes to nat your ass
    into hell.

randy

---

From: Russ White <[email-masked]>
Date: Fri, 09 Sep 2011 11:12:16 -0400

>     deploy, you will be deploying massively bigger boxes to nat your ass

There are two possible results, it seems to me:

1. The cost of deploying IPv6 will "bury" the cost of doing BGPsec, so
that BGPsec essentially becomes "free" in the IPv6 upgrade.

2. The cost of deploying BGPsec will be significant enough that it can't
be "buried," in any other costs.

The question is --which is true? I'm guessing #2 is going to be true,
but many others seem to think #1 is true. The reality is that the market
will decide, and no study is going to be able to answer this question fully.

Russ

---

From: Randy Bush <[email-masked]>
Date: Fri, 09 Sep 2011 18:19:19 +0200

> The question is --which is true?

as i have no data, any guess i make would be bullshit, would it not?

randy

---

From: Russ White <[email-masked]>
Date: Fri, 09 Sep 2011 12:22:08 -0400

> as i have no data, any guess i make would be bullshit, would it not?

Does anyone have the data needed to answer which is true? My guess is
based on the cost of hardware in general --even small hardware costs
don't normally end up being "swamped"-- and the cost of network
convergence times, etc.

Using different assumptions, you can come to different conclusions. I
don't know how you can build a study that would tell you which set of
assumptions is "correct."

OTOH, it would be interesting to try.

Russ

---

From: Randy Bush <[email-masked]>
Date: Fri, 09 Sep 2011 18:23:50 +0200

> OTOH, it would be interesting to try.

gambatte!

i am hoping to find time to try some simpler things.

randy

---

From: Jakob Heitz <[email-masked]>
Date: Fri, 9 Sep 2011 12:33:17 -0400

By doing "converge first verify later", the router can be made cheaper.

However, there is still the cost of the RPKI and the cost of running/maintaining it.
My guess is that will bury the cost of the routers.
Just a guess, Randy.

--
Jakob Heitz.

---

From: Russ White <[email-masked]>
Date: Fri, 09 Sep 2011 12:42:52 -0400

> By doing "converge first verify later", the router can be made cheaper.

I'm not certain I'd agree with this assessment... I'd guess you can make
convergence faster this way, but the memory and other costs on the
router are going to be the same. You could make it cheaper by moving
specific things off the router and onto auxiliary boxes, but the point
of inline authentication is _not_ to move anything onto boxes outside
the router (or even onto line cards within the router) --this is the
reason all overlay proposals were rejected up front.

> My guess is that will bury the cost of the routers.

I would guess this, as well --and I would guess that the cost of the
extra memory and processing to perform what BGPsec is asking for will
never be buried. IMHO, there will always be a cost differential between
a device capable of what BGPsec requires, and a device that's not capable.

Security will always cost something, no matter how you slice it. The
more "perfect" you try to make it, the more it will cost. SIDR seems to
have started out with "let's engineer the most perfect security we can
imagine," end of the stick --something most engineers (including myself)
tend to do a lot.

Russ

---

From: Christopher Morrow <[email-masked]>
Date: Sun, 11 Sep 2011 23:25:45 -0400

> as i have no data, any guess i make would be bullshit, would it not?

maybe what Wes is asking here is really:
"Could someone model the load on a router doing bgpsec, in a world of
bgpsec speaking devices?"

Something like, for a core network edge device (say sprint, C&W, TWTC,
UU/vzb,ATT an edge connecting device in their worst metro):
  o number of updates today/second (steady state and 'worst case')
  o projected growth of update stream (given historical data)
  o projected 'cost' (cpu cycles) of un-assisted bgpsec
  o projected RIB RAM size (use historical data to project forward)
  o projected beacons/second (which really just look like updates in
the update stream)
  o routing table size (projected forward from historical data)

It seems most of that data exists in one form or another, it seems
that running the math isn't "hard". There's a question of the validity
of the model... but that's always the case.

Wes, is this sort of thing what you're asking for?

-Chris

---

From: Randy Bush <[email-masked]>
Date: Mon, 12 Sep 2011 08:16:37 +0200

> bgpsec speaking devices?"

as i have said, i hope to get the time to do real measurement and
modeling.  until i, or someone whose methods i trust does so, all is
conjecturbation.

randy

---

From: Jakob Heitz <[email-masked]>
Date: Mon, 12 Sep 2011 02:21:52 -0400

> modeling.  until i, or someone whose methods i trust does so, all is

Do you trust this?

http://www.ietf.org/proceedings/81/slides/sidr-2.pdf

-- 
Jakob Heitz.

---

From: Randy Bush <[email-masked]>
Date: Mon, 12 Sep 2011 08:26:22 +0200

> http://www.ietf.org/proceedings/81/slides/sidr-2.pdf

no.  way too much excel model based on predictions of future growth
rates and how operators will act and way too little hard data.

randy

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Mon, 12 Sep 2011 08:16:06 -0400

In a kinder, gentler world, Randy used to speak like this, or at least he did once!
http://www.ietf.org/mail-archive/web/sidr/current/msg02845.html

Sriram

P.S. Regarding Toni's question in that email (to which Randy responded),
I did extend the model to include separate projected growth rates for IPv4 and IPv6.
_______________________________________
From: [email-masked] [[email-masked]] On Behalf Of Randy Bush [[email-masked]]
Sent: Monday, September 12, 2011 2:26 AM
To: Jakob Heitz
Cc: [email-masked]
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

randy

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Mon, 12 Sep 2011 08:40:15 -0400

Jakob,

Three key specific questions one would ask about a model are:

Q1. Does it model the underlying physical system accurately.
In this instance, would it correctly compute the RIB size, given
a number of routes (i.e., total # prefix-paths) in the RIB?

Q2: Is it well rooted in the historical measurement data
for projecting into the future?

Q3: Are additional assumptions made for projecting into the
future reasonable? In this instance, are the assumptions regarding
the BGPSEC take rate reasonable?

Answers (IMHO):

For Q1: Yes. There is no guess work involved here.
I feel we've accurately factored in the update overheads 
of BGPSEC based on draft-ietf-sidr-bgpsec-protocol-00.

For Q2: We used up to date historical measurement data from

---

From: "Sriram, Kotikalapudi" <[email-masked]>
Date: Mon, 12 Sep 2011 09:30:11 -0400

Sorry, I accidentally hit Send (instead of Save) halfway through in the previous email.
Here is the complete email.

Jakob,

Q2: Is it well rooted in the historical measurement data
for its projections into the future?

Answers (IMHO):

For Q2: We used up to date historical measurement data from 
http://www.potaroo.net/ispcol/2009-03/bgp2008.html 
(also Geoff's 2009 predictions in that paper came out nearly accurate
through the next two years into 2010 and 2011.)
We also used current measurements (as of March 2011)
from a large Tier 1 ISP regarding the total # routes they
observed in the PE routers and in Route Reflectors.
We used the observed growth rates for IPv4/IPv6 from the past 
several years to project into the future.
(Needless to say, predicting the future is never a hard science.)
We have provided the excel speadsheet so anyone can 
inject their world view regarding growth rates and see what happens.      
http://www.antd.nist.gov/~ksriram/BGPSEC_RIB.xls

For Q3: This is the hardest part of the modeling -- 
very tricky and no one can speak to it with certainty. 
We did overlay a BGPSEC take rate model (slides 5 and 6)
http://www.ietf.org/proceedings/81/slides/sidr-2.pdf
This was broadly based on historical view of take rates of new technologies in general.
(It is usually a Normal distribution curve.)
This was felt to be OK within the BGPSEC design team discussions.
Once again, anyone curious can play with the speadsheet to input
their assumptions/knowledge regarding their projected take rates for BGPSEC.

So at the minimum, if you just wish to find out the RIB size corresponding to
a given number of routes (i.e., total # prefix-paths), the model would
give you that accurately. You are further at liberty to input into the speadsheet
any assumptions regarding the growth rates and take-rates per your view of the world. 

I hope this helps some.

Sriram

---

From: "George, Wesley" <[email-masked]>
Date: Mon, 12 Sep 2011 09:39:25 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Russ White
Sent: Friday, September 09, 2011 11:12 AM
To: [email-masked]
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

>     be upgrading those routers to do real v6 forwarding.

WEG] Two or three years ago I might have at least partially agreed with the above. Now, most large-scale ISPs have IPv6 deployed, using hardware that they have been assured (through testing and contracts) can forward IPv6 traffic at linerate. I believe that there are plenty of places where there is real traffic running such that the concept that somehow we're all cruising towards disaster if this IPv6 thing ever gets real is FUD at best anymore. There is also a big difference in my mind between forwarding hardware and routing hardware, so talking about v6 forwarding is conflating the two in an unhelpful way.
However, rather than an assumption that the two share a timeline, I think that this implies that IPv6 deployments may have the net effect of knocking some of the oldest crap out of the network, which makes more sense to me. Even then, all it'll do is bring things up to today's state of the art (+/- 2 years), so I think Randy's assumption about today's state of the art being the absolute best that you can assume when modeling for 3-5+ years from now is a good one.

Wes

---

From: Randy Bush <[email-masked]>
Date: Mon, 12 Sep 2011 16:33:11 +0200

> Q2: Is it well rooted in the historical measurement data for its
> projections into the future?
> 
> Q3: Are additional assumptions made for projecting into the future
> reasonable? In this instance, are the assumptions regarding the BGPSEC
> take rate reasonable?

as humans and as engineers, we are well known for poor predictions of
the future.  we each have a model and they differ widely.  and if you
extrapolate, especially with a complex model, i become very suspicious
of any and all of your data.

this is why the data i give on rpki-rtr is what we have measured and
only what we have measured, 10u on a gsr/xr to validate one incoming
prefix, 4-7 seconds on gsr/xr, 7206/ios, and juniper m7 and m240 to load
360k roas built to map the current route views table.  this is from a
freebsd 8.2 server over both local gige and wan with 63ms rtt, seems to
make little difference.  we have not yet been able to make finer grained
measurements,

you are welcome to extrapolate all you wish.  i do not wish to.  i have
no idea what the timing would be with twice that many roas.  i would
guess it would take more time, but i really have no idea, it could be a
flat constant!

randy

---

From: "George, Wesley" <[email-masked]>
Date: Mon, 12 Sep 2011 10:40:08 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Russ White
Sent: Friday, September 09, 2011 12:43 PM
To: Jakob Heitz
Cc: [email-masked]
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

You could make it cheaper by moving

WEG] I think that this looks at the separation between forwarding and compute resources a bit too narrowly. As Shane pointed out in an earlier thread, there are some implementations where the forwarding hardware and the routing/general compute hardware are far too closely linked together in the way that they grow and scale. Perhaps the solution is to stop trying to cram new magic into RP slots and start pushing the external processing model where the compute resources are physically, but not logically separate (eg a blade center with a fabric interface, not a standalone route server) so that computing (routing) and forwarding can scale independently. It still ends up with a model where you're throwing more CPU resources at the problem instead of fixing the underlying internet routing scale problem, but perhaps that kicks the can far enough down the road to be acceptable because it puts things back on a commodity PC hardware growth and cost curve.

Wes

---

From: Jakob Heitz <[email-masked]>
Date: Mon, 12 Sep 2011 10:44:53 -0400

10u to validate an incoming prefix.
How long to validate an incoming path?
How long to generate/sign an outgoing path?
To one neighbor? To all neighbors?

--
Jakob Heitz.

---

From: Randy Bush <[email-masked]>
Date: Mon, 12 Sep 2011 16:51:55 +0200

> 10u to validate an incoming prefix.

rpki-based origin validation, NOT bgpsec.

> To one neighbor? To all neighbors?

how many times to i need to repeat that i have no fracking idea?  if i
had measured it, i would have told you 35 messages ago.

randy

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 12 Sep 2011 17:10:01 +0200

Hi Wes,

IMHO offloading essentially very similar computation which is 
architected to happen today on each ASBR (both inbound and outbound) to 
control plane device or set of devices residing anywhere within given AS 
which is _not_ an existing router would completely change the 
requirements as well as enable easy deployment of security in BGP, CCNx, 
you name it.

And this is not only for CPU/Memory issue.

You need to wait today 2-3 years for even minor feature enhancement from 
major vendors. Now add to this zoo of vendors and different dev 
priorities of each - anyone will clearly see that considering the 
reality deploying end to end Internet wide extension on 
existing/deployed equipment is just not going to happen.

Cheers,
R

PS. And as you said boxes which support IPv6 at line rate are deployed 
and on some of them there are no plans for new major software upgrades. 
What do you do with those considering that they are just fine to act  as 
routers for some time ?

---

From: "George, Wesley" <[email-masked]>
Date: Mon, 12 Sep 2011 14:28:17 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christopher Morrow
Sent: Sunday, September 11, 2011 11:26 PM
To: Randy Bush; George, Wesley
Cc: Russ White; [email-masked]
Subject: Re: [sidr] BGPSec scaling (was RE: beacons and bgpsec)

WEG] Yes, to some extent, but you're right that the model is the hard part, not the math. In trying to unwind a similar problem of how to characterize steady-state and peak CPU load on a L3VPN PE router so that there are real rules of thumb for capacity management and scaling, we discovered a couple of things -
1) (some) Vendors are quite bad at providing reasonably accurate multi-dimensional scaling models based on testing or real-world results. They tend to give a lot of single-dimension scale limits (eg with this knob turned to 11, you can get this value), but are very conservative and mumbly when it comes to what the actual real-life limits are, YMMV, etc. As a result, sometimes you end up finding out about the scaling cliff as you're falling over it, or you pay for hardware that you can never fully use because you stick to very conservative limitations.
2) a corollary: behavior at scale becomes increasingly non-deterministic the more variables you're working with simultaneously. Even worse, it's difficult to account in a model for things that work well enough at moderate scale, but are not efficient enough for high scale, or suffer some sort of secondary impact due to dependencies, etc.
3) some routers are very bad at providing useful data about critical scaling vectors (updates per sec, changes in multicast state, etc). Coupled with the fact that each router's numbers can be wildly different, it's difficult to characterize a "common" router, let alone a common network.
4) there are widely varying opinions among vendors and operators as to what is an acceptable level of performance at scale i.e. time to convergence of last route, steady-state CPU utilization (how much headroom is enough), stability during system or network events.

I think that what is coming up here are concerns in a couple of different categories:
1) Short-term hardware scale - is BGPSec supportable with what is realistically available today? For how long? Is that long enough?
2) Long-term hardware scale (5+ years) - What's the next breakthrough? How long does that buy us? Is that long enough? What does it do to our time remaining before we have to redesign the routing system to make it keep scaling?
This is where we should be considering RFC4984 and either updating or affirming the guidance there.
3) Cost for both - what is an acceptable assumption of the cost premium for BGPSec, in both capital and personnel?

On the hardware side, we're in a discussion that sounds a lot like predicting peak oil - when do we run out of scale growth on Moore's law with the current overall Internet architecture, and will BGPSec be just "one more gas-guzzler on the road" or the straw that broke the camel's back?

I don't know that we're going to get a definitive answer from modeling, and I'm not trying to bring on analysis paralysis either. Randy's (and mine, and everyone else's) guess may be BS, but even making a gut check based on what info we have available and documenting the assumptions we're basing our decision on would be a good thing.

Wes

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 12 Sep 2011 15:09:37 -0400

> I don't know that we're going to get a definitive answer from modeling, and I'm not trying to bring on analysis paralysis either. Randy's (and mine, and everyone else's) guess may be BS, but even m…

I agree with the above, and the last comment really was what I was
aiming at.. If someone were to model the 6-ish items I outlined, and
properly documented their test-harness (and maybe provided it out so
folk could test with their favorite settings?) that would help us get
around this paralysis problem. At least we'd feel a bit more
comfortable having something to check against.

-chris

---

From: Rob Shakir <[email-masked]>
Date: Tue, 13 Sep 2011 19:25:15 +0100

> I don't know that we're going to get a definitive answer from modeling, and I'm not trying to bring on analysis paralysis either. Randy's (and mine, and everyone else's) guess may be BS, but even m…

+1 to all Wes said in this post.

I've spent quite some time on similar problems (particularly around control-plane scale for network elements), and agree that modelling does not necessarily yield a definitive answer - however, some guidance as to what we might expect to need to be able to cater for is of great advantage from my perspective.

r.