Subject: RE: Last Call, draft-ietf-l3vpn-vpn-vr-01.txt and <draft-ietf-l3v pn-as-vr-01.txt>

From: "Paul Knight" <[email-masked]>
Date: Thu, 1 Apr 2004 14:11:20 -0500

Hi Mark and all,

I have a complete (I hope) Section 10 below, along with some responses
marked [pk].

> -----Original Message-----
> From: Mark Duffy [mailto:[email-masked]] 
> Sent: Thursday, April 01, 2004 12:23 PM
> To: Knight, Paul [BL60:1A14:EXCH]; Ross Callon; [email-masked]
> Subject: RE: Last Call, draft-ietf-l3vpn-vpn-vr-01.txt and 
> <draft-ietf-l3v pn-as-vr-01.txt>
> 
> 
> At 07:06 PM 3/31/2004 -0500, Paul Knight wrote:
> ...
> >[pk] Yes, it does need to be clarified... New text:
> >The backbone VR is a mechanism to enhance scalability.  The use of
> >backbone VRs is OPTIONAL in VR-based VPNs. When backbone VRs 
> are used, 
> >they SHOULD be configured on all PEs which participate in 
> VPNs carried 
> >over the backbone VRs.
> 
> That seems fine to me.  It doesn't explicitly address the 
> point Ross made 
> that the backbone VR at any given point could be replaced by 
> an actual 
> router adjacent to the PE.  But I think that is an arcane 
> enough point that 
> it can be left to the reader to deduce :-)
> 
[pk]Yes, Ross's point is exactly right, that an actual adjacent router could
provide the same functionality as a VR.  However, the recommended approach
is to configure backbone VRs instead of sticking another box into the
network!
> 
>   ...
> >[pk] Is this more clear?
> 
> Somewhat.  But maybe not enough on the first approach described...
> 
> >New text:
> >
> >10. Carrier's Carrier Case
> >
> >In some cases, the customer of a VPN is a service provider or carrier
> >offering VPN services for its own customers.  We can 
> describe this as a 
> >VPN hierarchy, with the "carrier's carrier" providing 
> backbone services to 
> >a "sub-carrier."
> >
> >The VR model provides several approaches to implement this VPN 
> >hierarchy.
> >
> >In one approach, tunnels are built from the VRs of the carrier's 
> >carrier
> >to the CEs of the customers of the sub-carrier.
> 
> I'm assuming this use of tunnels is in the same sense that 
> they may be used 
> in the non-carriers-carrier case, as described in sect. 3:   
> "CE devices 
> can be attached to VRs over any type of access link (e.g. 
> ATM, frame relay, 
> ethernet, PPP or IP tunneling mechanism such as IPSec, L2TP 
> or GRE tunnels)."
> 
> But if those "access link" tunnels go from the carriers' 
> carrier VRs to the 
> end-user CEs, as I believe you are describing, that would 
> seem to largely 
> bypass the sub-carrier.  The sub-carrier could fill a purely 
> business role 
> as a reseller (which is out of scope of this draft).  The sub 
> carrier could 
> also provide IP connectivity between the end-customer CEs and 
> the carriers' 
> carrier PEs.  But this IP connectivity would not be 
> VPN-aware!  It makes 
> the sub-carrier more like an ISP.  Is this what you have in 
> mind?  If so it 
> could use more description.
[pk] Yes, the sub-carrier is acting as an ISP, providing connectivity but
not a VPN-aware service.
> 
> 
> >   In this case, the VRs of the carrier's carrier provide 
> VPN service 
> > to
> > the remote CEs. This can be particularly useful in cases 
> where the CEs of 
> > the sub-carrier are themselves VRs

[pk] Sorry, the text is not precise enough.  "CEs of the sub-carrier" means
the sub-carrier's devices which look like CEs to the carrier's carrier (the
sub-carrier's P or PE devices), while "remote CEs"="CEs of the customers of
the sub-carrier".

[pk] So (referring to your comment now far below) I'm not *TRYING* to
describe a three-level hierarchy!  I'll try to clarify the text again.

[pk] Also, I managed to sort through the final approach described in section
10, and rewrite it in terms of carrier's carrier and sub-carrier.  By the
way, can you think of better terminology?

So: Section 10 in its entirety:

10. Carrier's Carrier Case

In some cases, the customer of a VPN is a service provider or carrier
offering VPN services for its own customers.  We can describe this as a VPN
hierarchy, with the "carrier's carrier" providing backbone services to a
"sub-carrier." The VR model provides several approaches to implement this
VPN hierarchy. 

In one approach, tunnels are built from the VRs of the carrier's carrier to
the CEs of the customers of the sub-carrier. In this case, the VRs of the
carrier's carrier provide VPN service to the remote CEs. The sub-carrier
provides transport but does not participate in the VPN services. This can be
particularly useful in cases where the sub-carrier's PE or P devices (which
appear as CE devices to the carrier's carrier) are themselves VRs (which may
be instantiated within the same device as the VRs of the carrier's carrier)
and where the sub-carrier is outsourcing the management of its customers'
VPN services. From the carrier's carrier perspective, this is sometimes
called "VPN wholesaling."  The carrier's carrier may support multiple
sub-carriers within a single PE device, through this approach using VRs.

Another approach is where the sub-carrier's VPN services are completely
transparent to the VRs of the carrier's carrier. This is the default case.
It is up to the sub-carrier's VPN service to distribute VPN reachability
among the CEs of its customers.

Another approach is for the sub-carrier's VPN service to implement the VR
architecture, using backbone VRs as described in section 5.3. In this
approach, the backbone VRs of the sub-carrier can establish tunnels to
backbone VRs of the carrier's carrier. Compared to the first approach, this
can reduce the number of VRs needed on the carrier's carrier devices, since
the backbone VRs can multiplex different VPNs. 

> 
> Hmm.  This sounds to me like you are now describing a *three* level 
> hierarchy of providers.  We have carriers' carrier (company 
> A), sub-carrier 
> (company B), and sub-carrier's customer (company C) at whose 
> site the CE 
> sits.  But when you make that CE into a VR you make company C a 
> sub-sub-carrier, no?
> 
> 
> >(which may be instantiated within the same device as the VRs of the
> >carrier's carrier) and where the sub-carrier is outsourcing 
> the management 
> >of its customers' VPN services.
> 
> So now the CEs belonging to company C are actually VRs 
> belonging to company 
> C.  And these VRs "belonging" to company C are in the same PE devices 
> (owned by company A) as the VRs operated by company A, and 
> connected to 
> them by IP tunnels within the PE?  I can see how the 
> protocols could do 
> that but I am at a loss to understand the business model.  
> And what does 
> company B provide in this case?
> 
> I can only guess that my interpretation is not what you have in mind.
[pk]  Yes, two levels is enough to attempt to describe!
> 
> 
> >Another approach is where the sub-carrier's VPN services are 
> completely
> >transparent to the VRs of the carrier's carrier. This is the 
> default case. 
> >It is up to the sub-carrier's VPN service to distribute VPN 
> reachability 
> >among the CEs of its customers.
> 
> Now that I understand.  :-)
> 
> --Mark
> 
Regards,
Paul

---

From: Mark Duffy <[email-masked]>
Date: Thu, 01 Apr 2004 12:23:04 -0500

>backbone VRs is OPTIONAL in VR-based VPNs. When backbone VRs are used, 
>they SHOULD be configured on all PEs which participate in VPNs carried 
>over the backbone VRs.

That seems fine to me.  It doesn't explicitly address the point Ross made 
that the backbone VR at any given point could be replaced by an actual 
router adjacent to the PE.  But I think that is an arcane enough point that 
it can be left to the reader to deduce :-)

  ...

>offering VPN services for its own customers.  We can describe this as a 
>VPN hierarchy, with the "carrier's carrier" providing backbone services to 
>a "sub-carrier."
>
>The VR model provides several approaches to implement this VPN hierarchy.
>
>In one approach, tunnels are built from the VRs of the carrier's carrier 

I'm assuming this use of tunnels is in the same sense that they may be used 
in the non-carriers-carrier case, as described in sect. 3:   "CE devices 
can be attached to VRs over any type of access link (e.g. ATM, frame relay, 
ethernet, PPP or IP tunneling mechanism such as IPSec, L2TP or GRE tunnels)."

But if those "access link" tunnels go from the carriers' carrier VRs to the 
end-user CEs, as I believe you are describing, that would seem to largely 
bypass the sub-carrier.  The sub-carrier could fill a purely business role 
as a reseller (which is out of scope of this draft).  The sub carrier could 
also provide IP connectivity between the end-customer CEs and the carriers' 
carrier PEs.  But this IP connectivity would not be VPN-aware!  It makes 
the sub-carrier more like an ISP.  Is this what you have in mind?  If so it 
could use more description.

>   In this case, the VRs of the carrier's carrier provide VPN service to 
> the remote CEs. This can be particularly useful in cases where the CEs of 

Hmm.  This sounds to me like you are now describing a *three* level 
hierarchy of providers.  We have carriers' carrier (company A), sub-carrier 
(company B), and sub-carrier's customer (company C) at whose site the CE 
sits.  But when you make that CE into a VR you make company C a 
sub-sub-carrier, no?

>carrier's carrier) and where the sub-carrier is outsourcing the management 

So now the CEs belonging to company C are actually VRs belonging to company 
C.  And these VRs "belonging" to company C are in the same PE devices 
(owned by company A) as the VRs operated by company A, and connected to 
them by IP tunnels within the PE?  I can see how the protocols could do 
that but I am at a loss to understand the business model.  And what does 
company B provide in this case?

--Mark

---

From: Mark Duffy <[email-masked]>
Date: Fri, 02 Apr 2004 10:32:53 -0500

Paul, this is coming along.  But I don't think it is quite there 
yet.  Comments below...

At 02:11 PM 4/1/2004 -0500, Paul Knight wrote:
>a "sub-carrier." The VR model provides several approaches to implement 
>this VPN hierarchy.
>
>to the CEs of the customers of the sub-carrier ("remote CEs"). In this 
>case, the VRs of the carrier's carrier provide VPN service to the remote CEs.

nit: I'd suggest adding '("remote CEs")' as I have done in your text just 
above, in order to establish the meaning of that term.

>The sub-carrier provides transport but does not participate in the VPN 
>services. This can be particularly useful in cases where the sub-carrier's 
>PE or P devices (which appear as CE devices to the carrier's carrier)

It doesn't seem accurate in this case to say that the sub-carriers's 
devices appear as CE devices to the carrier's carrier.  I think the "remote 
CEs" are the ones that appear as CE devices to the C's C.  The 
sub-carrier's devices are just part of an IP access network connecting C's 
C and the remote CEs, no?  Perhaps just omit the whole phrase in parentheses?

>  are themselves VRs (which may be instantiated within the same device as 
> the VRs of the carrier's carrier) and where the sub-carrier is 
> outsourcing the management of its customers' VPN services.

I'm not sure what value these intermediate VRs (the ones that are "owned" 
by the sub-carrier) provide in this case.  If you have anything to add 
about that it might help.

>  From the carrier's carrier perspective, this is sometimes called "VPN 
> wholesaling."  The carrier's carrier may support multiple sub-carriers 
> within a single PE device, through this approach using VRs.

I think these 2 sentences apply to C's C in general (all 3 approaches) so 
you might want to move these statements to the top or bottom of the section.

>backbone VRs of the carrier's carrier.

Do you really mean that?  I would expect that in this case the backbone VRs 
of the sub-carrier would appear to the C's C as CE's, and this would 
connect to "regular" (customer-facing) VRs operated by the C's C (and not 
to backbone VRs of the C's C which would face towards the C's C 
backbone).  And in this case tunnels would not usually be used because the 
sub-carrier's backbone VR and the C's C VR would usually be directly 
adjacent.  I.e. I am thinking that this case is just the "default" case but 
in which the sub-carrier happens to be using backbone VRs (which would 
actually be transparent to the C's C).  If you have something else in mind 
maybe it needs a bit more explanation.

>  Compared to the first approach, this can reduce the number of VRs needed 
> on the carrier's carrier devices, since the backbone VRs can multiplex 
> different VPNs.

Doesn't the C's C need one VR per sub-carrier per POP, regardless?

Thanks, Mark

---

From: "Paul Knight" <[email-masked]>
Date: Fri, 2 Apr 2004 12:25:49 -0500

Hi Mark,

This helps a lot. See inline below...

> Sent: Friday, April 02, 2004 10:33 AM
> To: Knight, Paul [BL60:1A14:EXCH]; Mark Duffy; Ross Callon; 
> [email-masked]

[pk]Good!
> C and the remote CEs, no?  Perhaps just omit the whole phrase 

[pk]Yes, you're right, they are transparent to the C's C... I'll omit it.
> 
> >  are themselves VRs (which may be instantiated within the 
> same device 
> > as

[pk] These VRs provide a point of control and aggregation for the
sub-carrier, just like a separate router providing transport service.  For
example, the sub-carrier may be able to define its customer-facing circuits
(usually VCs) on these VRs.

I propose expanding the parenthetical expression, like:
(which may be instantiated within the same device as the VRs of the
carrier's carrier, aggregating the connections from the remote CEs)

Does this help?
> you might want to move these statements to the top or bottom 

[pk] Yes, I'll do that.
> >among the CEs of its customers.

[pk] Unless I get any objections from the other authors, I think it will be
best to simply remove the third case discussed in section 10 (and below),
since it is not clearly distinguished from the default case, and does not
appear to provide any clear advantages to it.

> >Another approach is for the sub-carrier's VPN service to 
> implement the 
> >VR
[pk] Yes, one per "POP" (although that's not really a term defined in this
context, we'd probably have to say "PE of the sub-carrier").  I think the
third case can provide a reduction in VRs compared to the first case, where
it needs one VR per VPN customer of the sub-carrier at best, or one VR per
remote CE at worst... But I really think it's not a clear value compared to
the second case, so I propose to omit this third case unless another author
wants to clarify it.

Thanks and regards,
Paul

---

From: Mark Duffy <[email-masked]>
Date: Mon, 05 Apr 2004 10:18:24 -0400

Hi Paul,

Thank you, I am in agreement.

--Mark

At 12:25 PM 4/2/2004 -0500, Paul Knight wrote:

> > From: Mark Duffy 
> [<mailto:[email-masked]>mailto:[email-masked]]
>example, the sub-carrier may be able to define its customer-facing 
>circuits (usually VCs) on these VRs.
>
>[pk] Unless I get any objections from the other authors, I think it will 
>be best to simply remove the third case discussed in section 10 (and 
>below), since it is not clearly distinguished from the default case, and 
>does not appear to provide any clear advantages to it.
>
>third case can provide a reduction in VRs compared to the first case, 
>where it needs one VR per VPN customer of the sub-carrier at best, or one 
>VR per remote CE at worst... But I really think it's not a clear value 
>compared to the second case, so I propose to omit this third case unless 
>another author wants to clarify it.
>
>Thanks and regards,
>Paul
> >
> >
> > Thanks, Mark
> >
> >