Subject: RE: ISPs offering VPN service

From: [email-masked]
Date: Sat, 9 Sep 2000 08:29:06 +0100

<snip> Randy Bush  Thursday, September 07, 2000 8:21 PM 
> Subject:	Re: ISPs offering VPN service
> 
> 
> the problems with bgp scaling is not a matter of ram.  for a scary example
> of one small aspect of the current bgp mess see
> 
>     http://www.acm.org/sigcomm2000/conf/paper/sigcomm2000-5-2.ps.gz
> 
	[Harrison,N,Neil,INC1 X]  Thanks for this reference Randy.  Very
interesting paper.  Some observations I draw from a 1st pass on this paper
in the context of the current thread discussion are:
	-	these effects are a function of BGP per se, but they can be
'made worse' by poor (or at least non-harmonised)  implementations
	-	the fact that BGP is being used to also distribute labels is
incidental to the observed BGP performance
	-	all forwarding behaviour (be this Diffserv say (at either IP
or MPLS layers) or LSPs within an MPLS layer network itself for example) is
generally predicated on a stable routing environment.  So QoS-type
metrics/objectives (eg loss, delay) are *only* relevant to the
up-state.....whatever this means, since it is not defined as yet for IP/MPLS
networks......and should *not* be associated with the down-state, ie
availability and QoS SLA-type considerations should be orthogonal, otherwise
the former screws-up the latter's statistical significance.  A corollary
observation here is that the data/user-plane forwarding QoS performance
*must* always be worse than that determined by the stability of the
control-plane, ie we need to add (it will not be linear) user-plane
defects/impairments to those arising from control-plane mechanisms.
	-	and from all the above it seems I can conclude that: Any VPN
service offered on an IP/MPLS basis looks like it will experience QoS/outage
impairments significantly larger (and more frequent?) than from traditional
WAN technologies of TDM leased-lines, FR or ATM.  This concerns me since, as
an operator, at some stage I will have to make availability and QoS
statements in SLAs to customers of VPNs.
	-	the paper deals with the effects of routing information
change, but a further factor we have to consider is the mechanisms for
causing that information to change and, in particular, what sort of physical
server layer disturbances (eg short breaks) would give rise to this
behaviour.
	-	and this all brings me back to a point I have raised a
couple of times (and to which only Eric Gray and Curtis Vilamizar responded
- thanks) and that is the expected dynamic defect behaviour of different
types of label distribution techniques when nested.  And it would appear,
from this paper, that those predicated on BGP (but to be fair I guess this
would apply to any such interdomain cases?) are likely to get quite large
outage events.

	Regards, Neil

---

From: Eric Rosen <[email-masked]>
Date: Mon, 11 Sep 2000 12:55:17 -0400

If one believes  the various claims made about the poor  state of the public
Internet  routing system,  then the  one thing  which is  clear is  that one
should not  use a  VPN scheme  which is based  on IP  or IPsec  tunnels that
traverse the  public Internet  and then terminate  on the customer  site.  I
think  that these  schemes  are likely  to  have the  availability/reliabil-
ity/quality problems that you mention. 

It's quite ironic  that those who are shouting loudest  about the poor state
of  Internet routing  seem to  favor CPE-based  IP tunneling  schemes, which
depend ENTIRELY on Internet routing.

While there are a number of problems in the distribution of public IP routes
around the Internet, I haven't seen any clear statement or reasoned argument
as to why these problems should affect the distribution of the VPN-IP routes
in the more controlled environment offered by a VPN service provider.

---

From: Randy Bush <[email-masked]>
Date: Mon, 11 Sep 2000 22:53:45 +0200

> Internet routing system

before leaping to conclusions, one probably needs to understand it (the real
operational network) as well.

randy

---

From: Randy Bush <[email-masked]>
Date: Tue, 12 Sep 2000 06:37:56 +0200

> operational network) as well.

apologies.  i was a bit abrupt.  late and tired.  sorry.

the net does work.  i did get your message.  and packets do flow.

it comes down to scaling.  operators have watched the size of our networks
more than double every year, and spend our days and nights maintaining
stability and service despite protocols in which we find continual growing
pains (e.g. craig's paper on the needed bgp fix) and which we know do not
scale linearly.

so we are very sceptical of adding weight or complexity in the center, and
prefer to offload as much as possible to the edges.  and 2547[bis] looks as
if it adds weight and complexity in the center, and it is not at all clear
how it would scale when we're talking gillions of vpns and links.

and we do not succumb to the mpls fantasy that, because they look like
circuits, they are circuits and there are not a few layers of the usual
causes of worry below them.  and, as many of us are considering deploying
mpls as part of cbr traffic engineering, the issues of igp, inter-provider,
etc. etc. keep us very cautious.

randy