Subject: ISPs offering VPN service

From: Juan Diego Otero <[email-masked]>
Date: Mon, 04 Sep 2000 12:04:17 +0100

Hi,

I would like to know if there is any ISP currently

offering (or if it is planning to do it)

a VPN service based on MPLS.

Thanks a lot

Diego Otero

---

From: Cor van Rij <[email-masked]>
Date: Mon, 4 Sep 2000 12:25:04 +0200

Hi

VersaTel will offer it within the next 6 months

regards

cor

> -----Original Message-----
> From:	Juan Diego Otero [SMTP:[email-masked]]
> Sent:	maandag 4 september 2000 13:04
> To:	MPLS WG
> Subject:	ISPs offering VPN service
> 
> Hi, 
>

---

From: Jessica Yu <[email-masked]>
Date: Thu, 7 Sep 2000 09:47:03 -0700 (PDT)

--- Robert Raszuk <[email-masked]> wrote:

> 
> >   how will the pes perform?
> 
> During my current deployment experience with
> multiple customers there
> was never an issue with the PE scaling. Why ? -
> since I have never came
> across any VPN site which would not fit to the PE.

It would depend on how many VPNs a PE suports and how
large each VPN is, and what kind of hardware used for
PE in the deployment cases you mentioned.

> So as long as this
> condition holds when one PE get's full (I mean he
> memory is getting full
> or CPU for what ever reason is dying) one can easily
> add another PE and
> still grow easily. 

Yes, except adding more PEs means adding more PEs to
the iBGP mesh or adding more burdens to Route
Reflector if RR is used, which itself may need other
mechanisms to reliefe burdens, as you stated below.

> 
> Ok now depending what hardware/software you are
> using you may get some
> scaling limitations on vpnv4 reflector simply due to
> the fact that in
> some cases you may find end customers networks
> messed up big time
> without any hope for summarization. So what we can
> do here is a few
> things: reflector partitioning (based on RT value),
> (including pushing
> ORF to PEs), use router with more memory/cpu power
> or even use zebra.

......
> 
> There are different ways to provide VPNs and all
> have it's own usage. It
> was said a lot of time but ipsec has slight issue
> with overlay scheme
> associated maintenance. Setting thousands of vpns
> with nice tool may not
> be an issue but reconfiguring it based on the
> customer's open tickets is
> a good job security for some time in the NOC. 

It's not necessary the case. Directory based VPN
management tool is able to handle both building and
reconfiguring ipsec based VPN in a more scalable way.
There is at least one such product that I am aware of.

Cheers!

                                 --jessica

=====

__________________________________________________
Do You Yahoo!?
Yahoo! Mail - Free email you can access from anywhere!
http://mail.yahoo.com/

---

From: "Chase, Christopher J (Chris), ALNTK" <[email-masked]>
Date: Mon, 11 Sep 2000 13:40:22 -0400

Enforcing a topology within the provider portion of the VPN makes perfect
sense, rather than expecting the VPN owner to enforce a topology.

Customers may not have the capability to implement or manage TE tunnels to
enforce a topology on top of the any-to-any connectivity.  In addition, that
would require running MPLS to the CE instead of providing a VPN service that
uses standard IP CE-PE interface (e.g., FR or PPP).

Examples of wanting other topologies: inter corporate extranets, network
based firewall services, regional company gateways.

One big advantage of the MPLS VPN hub-and-spoke topology over the
private-line/VC model is circuit aggregation.  The hub or data center site
only needs to terminate a single logical connection independent of the
number of remote sites.  I know a lot of large enterprise networks using the
PVC hub-and-spoke model having to spread those PVCs across many ports and
routers at the hub site.

Chris Chase
AT&T IP Enabled FR Architect

> From: Christian Kuhtz [mailto:[email-masked]]
> Sent: Thursday, September 07, 2000 11:16 AM
> To: 'mpls wg'
> Subject: RE: ISPs offering VPN service
> 
> 
> 
> 
> > > That is not too say that I personely like hub and spoke. 
> In fact I do
> > > agree with you that full mesh is the way to go and 
> mpls-vpns easily
> > > allow it.
> >
> > while i may agree with you technically. enterprises are 
> used to it, and may
> > be slow to make the cultural change.  so i expect customers 
> to be in a
> > common process of evolving reconfiguration.
> 
> I'm lost with this argumentation.
> 
> In MPLS VPNs, default is that traffic starts to automatically 
> flow shortest
> path/fully meshed by the very nature of it.  Basically, all 
> sites of a given
> customer just attach to a common net.  So, that said, what 
> evolution is there
> for customers to go thru?
> 
> No "evolving reconfiguration"...
> 
> But, if you really want them to, you can just make MPLS VPNs 
> look like hub &
> spoke, why you would ever want to force that inside the vpn 
> (you can always do
> it on a TE level regardless, if you desire to) escapes me, 
> but it is possible.
> 
> Cheers,
> Chris
> 
> --
> Christian Kuhtz, Sr. Network Architect              
> Architecture, BellSouth.net
> <[email-masked]> -wk, <[email-masked]> -hm                 
>       Atlanta, GA
>                                                     "Speaking 
> for myself only."
>

---

From: "Chase, Christopher J (Chris), ALNTK" <[email-masked]>
Date: Mon, 11 Sep 2000 14:12:39 -0400

Adding new PEs to handle new VPNs is fine.  However, I do have to apriori
anticipate the number of routes in a single VPN.  So I have to limit the
number of VPNs on a given PE.  As a particular VPN adds sites, all the
routes for that VPN show up at every PE the VPN touches.

If the PE routing table is reaching capacity because of the growth of
existing VPNs then adding an additional PE will not alleviate that problem.
Without very conservative provisioning assignment rules in place I may have
to "rehome" VPNs off of the constrained PE.  I would prefer not to rehome -
so instead I try to be very conservative in provisioning assignment.  As a
result it may mean an interim of underutilized PEs until individual VPNs
stabilize in size (it can take up to a year or more for some enterprises to
deploy a complete network).

Additionally, to aid in the scalability of VPNs on a PE we feel that it is
important that the PE be able to limit the total routes in a vrf.  I do not
want a customer to accidentally inject Internet routes into his VPN and
consume all the route table resources on the PE - thus affecting other VPNs
on the same VPN.

My goal is that the customer should pay proportionately to the amount of PE
resources he consumes.  Thus the limit on the number of routes in the VPN
should be proportional to the number of sites in the VPN.  Most business
models I have seen the revenue for a VPN scales linearly with the number of
sites.  

Making this association is not unreasonable since large enterprise networks
I am familiar with have the number of routes scale as 2*N+C where N is the
number of sites and C is a constant representing the routes at the hub site.
Large enterprises with hundreds or thousands of sites have a single network
(LAN) at all their "remote" sites.  (The multiplier is 1 if the CE-PE link
is a subnet of the site summarized network).

I can not summarize as a MPLS VPN provider in the middle of an enterprise
network.  But if the customer is paying proportionately to the utilized
network resources then I don't have a problem with not summarizing.  As long
as the ratio of total number of PEs (and RRs) to total number of CEs stays
constant as the network grows then I am fine.  If I need more PE's then it
means I am making more money.

If a VPN is serving a carrier application rather than enterprise then the
2N+C model does not hold.  But that VPN is ideal for carrier's carrier VPN
where only N site routes need to be directly exposed in the provider VPN.

Chris Chase
AT&T IP Enabled Frame Relay Architect

> From: Robert Raszuk [mailto:[email-masked]]
> Sent: Wednesday, September 06, 2000 3:37 PM
> To: Sitarama R Kalipatnapu
> Cc: [email-masked]; [email-masked]; 'Randy Bush'; 'mpls
> wg'
> Subject: Re: ISPs offering VPN service
> 
> 
> 
> Sitarama,
> 
> > 1. When do you decide to add another PE router, as the number of
> >     VPNs supported is increasing ?
> 
> Let me say once again that number of VPNs does not trigger addition of
> another PE. Why ? Simply because the overhead of vrf is memory
> consumption wise not significant. What does matter is the number of
> routes your VPNs give you as you need to store them in the vpnv4 bgp
> table as well in RIBs & FIBs for given vrfs. So it really 
> depends on the
> operator's comfort with a given router to run at X% of memory
> utlization. Notice that in moders routers the memory limit and upgrade
> flexibility is growing so even that may not be a very significant
> factor. 
> 
> The same applies to CPU.
> 
> > 2. What makes us add a second PE router ?
> 
> See above.
> 
> > 3. Is this connected to BGP ?
> 
> No. In a peering VPN model you need to maintain customer routes. What
> ever protocol you would use you will still have a potential of hitting
> the hardware limitations at some point and bgp is quite ortogonal to
> this. 
> 
> Just one more comment on your last question:
> 
> > > > 3.  Is it possible to unload the demands on BGP 
> (carrying  MPLS labels )
> > > >      in the BGP/MPLS VPNs model ?
> 
> Using bgp to carry labels for vpn routes is totally not an issue as it
> just extending the NLRI size by 20 bits without any 
> processing penalty.
> BGP is more or less just a vehicle here.
> 
> R.
> 
> > 
> > Thanks,
> > 
> > --Sitaram
> > 
> > Robert Raszuk wrote:
> > 
> > > Sitarama,
> > >
> > > BGP is not a choke point. First it is already there in 
> all edge routers
> > > in all ISPs I know of and adding the new AF to it does 
> not cause it to
> > > choke in the right implementation. Notice that the 
> "choking" effect all
> > > depends how one sets the timers (per AF of course - so it does not
> > > effect plain IPV4 updates or other bgp IPV4 mechanisms). 
> As a contary to
> > > what you are saying many customers are setting the VPNv4 
> timers more
> > > aggresively then ipv4 timers to allow equal to IGP 
> convergence and even
> > > that is not causing bgp to "choke" in a production networks.
> > >
> > > The scalability of MPLS-VPNs is achived by the fact that 
> BGP is being
> > > used as a tool there so it would be a mistake to even attempt it's
> > > replacement by some other means.
> > >
> > > R.
> > >
> > > > On the note of scalability of BGP/MPLS VPNs, adding 
> more hardware
> > > > seems to do the trick. But on a close observation it 
> appears, BGP is the
> > > > choke point. BGP running in the provider edge seems to 
> peer with the
> > > > customer edges and also distribute MPLS labels across 
> PE edges. The
> > > > questions are:
> > > >
> > > > 1.  Do we have alternate solutions to provide MPLS VPNs 
> without BGP ?
> > > > 2.  If so are they effective ?
> > > > 3.  Is it possible to unload the demands on BGP 
> (carrying  MPLS labels )
> > > >      in the BGP/MPLS VPNs model ?
> > > >
> > > > Thank You.
> > > >
> > > > --Sitaram
> > > >
> > > > Eric Rosen wrote:
> > > >
> > > > > Brijesh> I think you are probably right. It seems 
> that the model in RFC 2547
> > > > > Brijesh> may be flawed because it fails to separate 
> internal VPN control and
> > > > > Brijesh> configuration  within  a  domain  from the  
> routing  control  plane
> > > > > Brijesh> primarily  designed  for  carrying  external 
>  inter-domain  transit
> > > > > Brijesh> traffic.
> > > > >
> > > > > No it  doesn't.  It does  use BGP for  both purposes 
> though.  The  fact that
> > > > > technology which is primarily designed  for one 
> application is also used for
> > > > > a second application is not generally considered to 
> be a bad thing.
> > > > >
> > > > > Brijesh> adding  VPN  related policies  (basically,  
> internal  traffic of  a
> > > > > Brijesh> domain) to something that was originally 
> designed to provide policy
> > > > > Brijesh> control of inter-domain traffic doesn't seem 
> obviously appropriate.
> > > > >
> > > > > The "VPN-related policies" are primarily expressed as 
> filtering on community
> > > > > attributes, which is actually fits in quite well with 
> normal BGP functions.
> > > > >
> > > > > Brijesh> Besides the scalability problems you mentioned,
> > > > >
> > > > > The  only  scalability problem  Randy  mentioned is  
> that  as  you get  more
> > > > > customers, you might have to add more equipment.  I'd 
> like to see the scheme
> > > > > in which the hardware cost is independent of the load!
> > > > >
> > > > > Brijesh> It is  not going  to help busy  network 
> administrators who  need to
> > > > > Brijesh> manage large number  of BGP nodes, and 
> resolve  policy conflicts in
> > > > > Brijesh> increasingly large and complex networks.
> > > > >
> > > > > This is  sophistry.  You  could argue against  any 
> VPN scheme  whatsoever by
> > > > > pointing out that it creates  some administrative 
> burden, when compared with
> > > > > not having  any VPN scheme at  all.  That's not very  
> interesting, it's only
> > > > > interesting   to   compare   two   VPN   schemes,   
> for   instance   RFC2547
> > > > > vs.  configuring,  managing,  and   maintaining  
> hundreds  of  thousands  of
> > > > > point-to-point tunnels.
>

---

From: [email-masked]
Date: Tue, 12 Sep 2000 10:09:23 +0100

Eric Rosen wrote on Monday, September 11, 2000 5:55 PM 	Re: ISPs
offering VPN service 

> It's quite ironic  that those who are shouting loudest  about the poor
> state
> of  Internet routing  seem to  favor CPE-based  IP tunneling  schemes,
> which
> depend ENTIRELY on Internet routing.
		[Harrison,N,Neil,INC1 X]  Quite.  If I assume the results
given in the paper on 'Delayed Internet Routing Convergence' from Michigan
University are correct (and I see no reason why they should not be) then I,
as an operator, would have some real concerns about the nature of the SLAs I
could agree with my VPN customers.  Maybe some VPN customers (who are used
to trad L1/2 VPN solutions) might not be so happy about this type of
performance?  And in which case I would need an alternative solution that is
*proved* to be stable enough.......and that includes both the dynamic defect
behaviour from control-plane aberrations as well as those arising from
data-plane aberrations (which are likely to be sourced from L1, and where
the data-plane behaviour could also impact the control-plane behaviour).

> While there are a number of problems in the distribution of public IP
> routes
> around the Internet, I haven't seen any clear statement or reasoned
> argument
> as to why these problems should affect the distribution of the VPN-IP
> routes
> in the more controlled environment offered by a VPN service provider.
		[Harrison,N,Neil,INC1 X]  I agree.  However, I have seen
many pleas for 'comments from operators' and I have given one of my concerns
here, which as yet is still unresolved to my satisfaction.....so I will
briefly repeat it as I think it is relevant to trying to compare different
methods of delivering VPNs:

		An SLA related to QoS is all about the up-state forwarding
behaviour when everything is working as expected.  A more important SLA
however is related to when things are not working as expected and there are
outages.  I can easily imagine a situation where the QoS SLAs are common (ie
the same or very similar) across the majority of VPNs, but where the
availability SLAs need to be very different across VPNs (or even across
different workgroups in the same VPN).  I can't appeal to DiffServ here
since this assumes an up-state (and as an aside observation DiffServ will
mutate failures into QoS hits, skewed by class wrt to forwarding 'survival'
priority....which is not really what I want).  So what I need is a better
handle on LSP failure behaviour and tools to provide this....hence the OAM
ID which some of us are now working on.

		To provide VPNs using MPLS one has to have at least a label
stack depth of 2.  Adding other considerations (eg TE) will also increase
the label stacking depth.  I now have 2 degrees of freedom wrt to
client/server LSP relationships:
		-	nesting within a single LDP-type
		-	nesting across multiple LDP-types
		What I want to understand better is the expected defect
behaviour for various combination of LDP-type nesting, ie stacking of LSPs
using vanilla-LDP, CR-LDP, RSVP-LDP and BGP4-LDP in various arrangements.  I
would expect there to be some differences and I think this needs to be
explored and better understood by operators.

		If we understood the above better, operators would be able
to say something more concrete about MPLS VPN behaviour and the SLAs we
could set/enforce......and from your observations here Eric, I would also
suggest we have to add other methods of VPN construct (eg IPSec) to the list
for evaluation/comparison purposes.  These all ought to be benchmarked
against the trad L1/L2-type of VPN behaviour.  And yes I am well aware of
the config/scaling issues here, but if a particular VPN technology cannot,
for whatever reason, offer the QoS/availability that some customers will
demand then we should know about this.

		Regards, Neil

---

From: GUESDON Herve FTRD/DAC/ISS <[email-masked]>
Date: Mon, 25 Sep 2000 15:33:24 +0200

Hello Henrich

I think also that it is a good idea to aggree on some figures regarding IP
VPN, especially regarding the service requirements. I aggree on the numbers
you gave, but they are related to the definition of IP VPN for entreprises
where one site equals one IP subnet. I think you should also take into
account the number of routes per VPN and the different flavours of IP VPN
mainly VPN of VPN and community of interests. Then the number of VPN and
routes per VPN can be higher.

Regards

hervé

PS : I include the nbvpn mailing list, which is involved in defining IP VPN.

-----Message d'origine-----
De : Hummel Heinrich [mailto:[email-masked]]
Envoyé : lundi 25 septembre 2000 14:37
À : [email-masked]'
Objet : Re: ISPs offering VPN service

Hi all,

Is it possible or do you all think it will be helpful to first of all agree
on a set of  fictitious but reasonable numbers, e.g. like

envisioned number of small-sized vpns of up to 10 sites       = 100 000
                number of  medium-sized vpns of up to 300 sites = 2000
                number of large-sized vpns of up to 2000 sites     = 500

number of nationally operating SPs = 500
number of nations =200

number of internationally operating SPs = 100

 Are there other important figures and differentiations ? And what would be
the values everybody will agree on ?

I think the dispute on RFC 2547 and on BGP concerning chokes and limitations
would have a better fundament and would
yield better results, if all agree first on a set of such figures.

Heinrich Hummel
Siemens

[email-masked]

---

From: Hummel Heinrich <[email-masked]>
Date: Wed, 27 Sep 2000 11:40:49 +0200

I am certainly not aiming for IPSEC nor for any other quick solution.

But if we just want to cement what so far has been worked out, then we wouldn't need a new nbvpn-WG, would we ?

Routing to a (different) destination VPN  in the first place as to get (finally) to a destination user: 
 This is an issue and all solutions should be described. A tunnel between the VPN border nodes is certainly one solution.
Should there be others in the long run ? I don't see any VPN -sensitive addressing in IPv6 (am I wrong ?).

Coming back to my initial proposal to agree on some expected deployment figures: In case they are big (which I believe), it is worth
to think about long-term solutions. It would be bad if even IPv6 addressing becomes an obstacle for future-proven solutions, will say if
tunneling is enforced just due to IPv6-addressing defficiencies. 

Heinrich
> -----Urspr> üngliche Nachricht-----
> Von:	Curtis Villamizar [SMTP:[email-masked]]
> Gesendet am:	Dienstag, 26. September 2000 17:50
> An:	Hummel Heinrich
> Cc:	[email-masked]'; [email-masked]'; [email-masked]'
> Betreff:	Re: AW: ISPs offering VPN service 
> 
> 
> In message <DB74A4E69C7CD311B740006008136E078D349B@MCHH213E>, Hummel Heinrich w
> rites:
> > 
> > 2) Establishing a VPN by a network administrator completely from his desk. 
> >    Juha's "soft-LSP" scenario   should even be extended to a "3rd-party LSP s
> > etup" scenario. 
> 
> Use IPSEC and don't bother your provider at all.
> 
> > 3) Routing between different (and not allied) VPNs while utilizing the instal
> > led source- and destination-VPNs:
> 
> Use IPSEC and don't worry about who in the VPN uses which provider.
> 
> > What do you think?
> >    
> > Heinrich
> 
> You want to use IPSEC for the above.
> 
> Some providers don't want to hear that because it means no added
> revenue for maintaining the VPN but a while back they didn't want to
> hear about IP because they couldn't charge per connection or per byte.
> 
> If you want a single provider VPN where the customer has to have no
> involvement in the management just pays someone (and relies on their
> security), then use RFC2547.
> 
> Curtis

---

From: Jieyun Jessica Yu <[email-masked]>
Date: Wed, 27 Sep 2000 18:37:43 -0700

>security), then use RFC2547.

I do not disagree with you but just want to add a point: VPN built with
IPsec/VR model allows a VPN customer to outsource management to its SP as
well.

                                --Jessica