Subject: [Idr] FW: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Fri, 20 Jun 2014 02:54:45 +0000

Hi all, 

This is a short draft to make RTC applicable in hierarchical RR scenario.

Comments are welcome.

Best regards,
Jie

> -----Original Message-----
> 
> A new version of I-D, draft-dong-idr-rtc-hierarchical-rr-00.txt
> has been successfully submitted by Jie Dong and posted to the IETF repository.
> 
> Name:		draft-dong-idr-rtc-hierarchical-rr
> Revision:	00
> Title:		Extensions to RT-Constrain in Hierarchical Route Reflection
> Scenarios
> Document date:	2014-06-19
> Group:		Individual Submission
> Pages:		5
> URL:
> http://www.ietf.org/internet-drafts/draft-dong-idr-rtc-hierarchical-rr-00.txt
> Status:
> https://datatracker.ietf.org/doc/draft-dong-idr-rtc-hierarchical-rr/
> Htmlized:
> http://tools.ietf.org/html/draft-dong-idr-rtc-hierarchical-rr-00
> 
> 
> Abstract:
>    The Route Target (RT) Constrain mechanism specified in RFC 4684 is
>    used to build a route distribution graph in order to restrict the
>    propagation of Virtual Private Network (VPN) routes.  In network
>    scenarios where hierarchical route reflection (RR) is used, the
>    existing RT-Constrain mechanism cannot build a correct route
>    distribution graph.  This document refines the route distribution
>    rules of RT-Constrain to address the hierarchical RR scenarios.
> 
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 20 Jun 2014 11:27:31 +0200

Hello Jie,

Please let me understand why instead of defining new BGP term of "the
most disjoint alternative route" between RR-1 and RR-2 and between
RR-1 and RR-3 you would not simply run add-paths on the RTC address
family ?

I believe that even add-paths = 2 where next hop is different on the
set of advertised paths will solve your issue.

Likewise if add-paths is not supported on any of those RRs in question
one can also fix the scenario by using concept of diverse-path (RFC
6774) without any additional changes to BGP advertisement machinery on
BGP route reflectors.

Isn't it always an advantage to use existing mechanism to solve
potential deployment problems ?

Comments ?

Best regards,
R.

On Fri, Jun 20, 2014 at 4:54 AM, Dongjie (Jimmy) <[email-masked]> wrote:
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: <[email-masked]>
Date: Fri, 20 Jun 2014 12:03:40 +0000

Hi Jie,

Why not just applying the same behavior as for Originator ID for cluster list ? So erasing it, and putting RR Cluster ID only in Clist ...
Is there some concern about update loop between RRs in RTC scenario that I missed ?

Stephane

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Dongjie (Jimmy)
Sent: Friday, June 20, 2014 04:55
To: [email-masked]
Subject: [Idr] FW: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi all, 

Comments are welcome.

Best regards,
Jie

> Please note that it may take a couple of minutes from the time of 
> submission until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 20 Jun 2014 14:08:07 +0200

What happens if RR-2 has session with RR-3 - on purpose or by mistake ?

I would not go that far ...

Cheers,
R.

---

From: <[email-masked]>
Date: Fri, 20 Jun 2014 12:44:11 +0000

I was thinking about the situation where RR of same level has session to each other ...
I think the issue that you would have with RTC, is only unefficient VPN route distribution.

Normally you have the same kind of issue with OriginatorID ... isn't it ? 

Considering now we have this session between RR2 and RR3 and consider that RR1 does not have any client owning VPN1, the more efficient distribution method would be that distribution tree does not include RR1 ...

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, June 20, 2014 14:08
To: LITKOWSKI Stephane SCE/IBNF
Cc: Dongjie (Jimmy); [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Cheers,
R.

On Fri, Jun 20, 2014 at 2:03 PM,  <[email-masked]> wrote:
> Subject: [Idr] FW: New Version Notification for 
> draft-dong-idr-rtc-hierarchical-rr-00.txt
>
> Hi all,
>
>> http://www.ietf.org/internet-drafts/draft-dong-idr-rtc-hierarchical-r
>> r
>> -00.txt
>> Status:
> ______________________________________________________________________
> ___________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations 
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.

---

From: <[email-masked]>
Date: Fri, 20 Jun 2014 12:53:13 +0000

Robert,

Regarding add-path or some other knob, I agree that this would work but when you have a basic hierarchical RR design for VPN service, you can expect that RTC works without having to tweak BGP. 
IMO, the basic spec is really broken for this scenario and we have to address it.

Best Regards,

Stephane

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, June 20, 2014 11:28
To: Dongjie (Jimmy)
Cc: [email-masked]
Subject: Re: [Idr] FW: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hello Jie,

Please let me understand why instead of defining new BGP term of "the most disjoint alternative route" between RR-1 and RR-2 and between
RR-1 and RR-3 you would not simply run add-paths on the RTC address family ?

I believe that even add-paths = 2 where next hop is different on the set of advertised paths will solve your issue.

Likewise if add-paths is not supported on any of those RRs in question one can also fix the scenario by using concept of diverse-path (RFC
6774) without any additional changes to BGP advertisement machinery on BGP route reflectors.

Isn't it always an advantage to use existing mechanism to solve potential deployment problems ?

Comments ?

Best regards,
R.

On Fri, Jun 20, 2014 at 4:54 AM, Dongjie (Jimmy) <[email-masked]> wrote:
>> r-00.txt

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 20 Jun 2014 14:57:28 +0200

Hi Stephane,

I am not worried about efficiency here.

Imagine worst case of three RRs in this picture having sessions to
each other and each deletes and puts it's own CLUSTER LIST before
sending the update out.

What would stop the RTC SAFI update loop between those RRs if RRs will
not be able to tell looking at the CLUSTER_LIST if they have already
processed the update before ?

Perhaps some implementations would indeed recognized that the same
path exist already and suppress generation of new update, but even
then when you have more RRs in such setup you are now depending on the
timing issues.

We are also subject to run into the problem (which Danny has been
pointing out many times before)  of  sending by to clients in the same
peer group all paths including his own advertisements, but now without
CLUSTER_LIST due to broken (and blessed by the spec) split-horizon in
RR peer groups.

r.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 20 Jun 2014 15:03:21 +0200

Hi Stephane,

My personal believe is that hierarchical RRs is really bad idea
especially for VPN cases. In VPNs RRs are the "weakest" points of
route accumulation and adding hierarchy there makes the split of VPN
routes based on the RTs grouping much harder.

Moreover RRs now run just fine on shelf x86 machines and single box
can handle easily millions of routes and thousands of sessions so I do
not think hierarchy buys you much.

If anything IMO we should rather focus to make sure RTC works without
RRs in an automatically created full mesh ;).

Cheers,
R.

---

From: <[email-masked]>
Date: Fri, 20 Jun 2014 13:43:37 +0000

Hi,

I'm not pushing for hierarchical RR at all :) But I can imagine than some people use it as for Internet networks ....

Stephane

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, June 20, 2014 15:03
To: LITKOWSKI Stephane SCE/IBNF
Cc: Dongjie (Jimmy); [email-masked]
Subject: Re: [Idr] FW: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Stephane,

My personal believe is that hierarchical RRs is really bad idea especially for VPN cases. In VPNs RRs are the "weakest" points of route accumulation and adding hierarchy there makes the split of VPN routes based on the RTs grouping much harder.

Moreover RRs now run just fine on shelf x86 machines and single box can handle easily millions of routes and thousands of sessions so I do not think hierarchy buys you much.

If anything IMO we should rather focus to make sure RTC works without RRs in an automatically created full mesh ;).

Cheers,
R.

On Fri, Jun 20, 2014 at 2:53 PM,  <[email-masked]> wrote:
> Subject: Re: [Idr] FW: New Version Notification for

---

From: <[email-masked]>
Date: Fri, 20 Jun 2014 13:47:34 +0000

I completely agree that touching loop prevention mechanism may cause some issue ... 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, June 20, 2014 14:57
To: LITKOWSKI Stephane SCE/IBNF
Cc: Dongjie (Jimmy); [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Stephane,

Imagine worst case of three RRs in this picture having sessions to each other and each deletes and puts it's own CLUSTER LIST before sending the update out.

What would stop the RTC SAFI update loop between those RRs if RRs will not be able to tell looking at the CLUSTER_LIST if they have already processed the update before ?

Perhaps some implementations would indeed recognized that the same path exist already and suppress generation of new update, but even then when you have more RRs in such setup you are now depending on the timing issues.

We are also subject to run into the problem (which Danny has been pointing out many times before)  of  sending by to clients in the same peer group all paths including his own advertisements, but now without CLUSTER_LIST due to broken (and blessed by the spec) split-horizon in RR peer groups.

r.

On Fri, Jun 20, 2014 at 2:44 PM,  <[email-masked]> wrote:
> Subject: Re: [Idr] New Version Notification for 
>> _____________________________________________________________________
>> _ ___________________________________________________
>>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Fri, 20 Jun 2014 19:02:18 +0000

Why? A picture would be quite useful here. I am struggling to understand a hierarcacal design that then needs RT-C?? Maybe if the VPNs are highly localized.. BTW isn't RT-C a good candidate to off net to an SDN controller? IMO that should be the future.

Jim Uttaro

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Dongjie (Jimmy)
Sent: Thursday, June 19, 2014 10:55 PM
To: [email-masked]
Subject: [Idr] FW: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi all, 

Comments are welcome.

Best regards,
Jie

---

From: Vitkovský Adam <[email-masked]>
Date: Sat, 21 Jun 2014 17:19:22 +0000

Hi Jie,
Well the correctly designed MPLS networks use unique per PE per VRF RDs in form of RID:xxxxx. 
Unique per PE RDs makes all copies of the same VPN prefix received from different PEs unique so RRs will select all as best path and distribute them accordingly, which solves this and many other issues. 

In other words if someone tries to shoot himself in the foot I wouldn't develop bulletproof boots for him. 

adam
> Sent: Friday, June 20, 2014 4:55 AM
> To: [email-masked]

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 21 Jun 2014 19:52:09 +0200

Hi Adam,

Jie is describing RT routes not VPN routes. Your story does not apply
to RT routes as format of RT NLRI is well defined by RFC ;)

Besides MPLS network or not has nothing to do with it ....

Best,
R.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Sat, 21 Jun 2014 18:56:14 +0000

+1

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Vitkovský Adam
Sent: Saturday, June 21, 2014 1:19 PM
To: Dongjie (Jimmy); [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Jie,

adam

---

From: Vitkovský Adam <[email-masked]>
Date: Sat, 21 Jun 2014 19:19:36 +0000

Aah I see :)
Well we could have that one solved if RD would be part of the RTC routes :) (though I understand one could use RTC for non VPN routes as well so...)
Anyways so than as you mentioned add-path on the RTC AF seems as the best option then.

adam
> Sent: Saturday, June 21, 2014 7:52 PM
> To: Vitkovský Adam
> Jie is describing RT routes not VPN routes. Your story does not apply to RT
> routes as format of RT NLRI is well defined by RFC ;)
> 
> > Unique per PE RDs makes all copies of the same VPN prefix received from
> different PEs unique so RRs will select all as best path and distribute them
> accordingly, which solves this and many other issues.
> >
> >> > http://www.ietf.org/internet-drafts/draft-dong-idr-rtc-hierarchical
> >> > -rr
> >> > -00.txt
> >> > Status:

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 23 Jun 2014 02:01:51 +0000

Hi Robert, 

Sorry for the late reply.

First of all, this draft describes the problem of RTC in hierarchical RR scenario, and I think this problem is ACKed.

Regarding the proposed solution, to be honest the co-authors discussed several alternate solutions and add-path was one of them. The concern was that with add-path=2, in some tricky topology both the 2 paths advertised may include the same CLUSTER_ID or ORIGINATOR_ID that may result in both paths being discarded due to loop detection. 

I totally agree that using existing mechanisms is always preferred, while like Stephane said we'd also expect RTC to work in basic hierarchical RR case without assistance of other BGP mechanisms. And since recently we are making several updates to RTC, it may be the right time to address this problem together.

Best regards,
Jie

> Sent: Friday, June 20, 2014 5:28 PM
> To: Dongjie (Jimmy)
> Cc: [email-masked]
> Please let me understand why instead of defining new BGP term of "the most
> disjoint alternative route" between RR-1 and RR-2 and between
> Likewise if add-paths is not supported on any of those RRs in question one can
> also fix the scenario by using concept of diverse-path (RFC

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 23 Jun 2014 02:16:07 +0000

Hi Stephane, 

I think Robert has pointed out the potential risk of erasing the cluster list information, which is also our concern. And IMO the rule defined in this draft can also work for the ORIGINATOR_ID manipulation, which makes it generic for the RTC route advertisement...

We are open to the discussion of the best solution for this problem.

Many thanks,
Jie

> From: Idr [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Friday, June 20, 2014 8:04 PM
> Why not just applying the same behavior as for Originator ID for cluster list ? So
> erasing it, and putting RR Cluster ID only in Clist ...
> ________________________________________________________________
> _________________________________________________________
> 
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou
> copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le
> signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration, Orange decline toute
> responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed, used
> or copied without authorisation.

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 23 Jun 2014 02:41:39 +0000

Hi Jim,

Here is the example I previously sent to the list: 

PE1---RR2
         \
          \    
PE2-       RR1
    \     /
     \   /
      RR3
     /
    /
PE3-

RR1 is level-1 RR, and RR2 and RR3 are level-2 RRs.

1. All the PEs advertise RTC route of RT-X to their connecting RRs.
2. RR2 and RR3 select the best RTC route, add their cluster_id to the cluster_list and advertise the RTC route to RR1 respectively. 
3. RR1 selects one RTC route as best (let's say RTC route from RR2), adds its cluster_id into the cluster_list and advertises it to RR2 and RR3. 
4. Then RR2 would receive the RTC route from RR1 with its own cluster_id in the cluster_list and will drop this RTC route.
5. As a result, RR2 will not advertise VPN routes with RT-X to RR1.

For a network with hierarchical RR, RTC can also be useful to control the VPN routes being distributed only to PEs which have the corresponding customers connected.

I'm not quite clear about your comments about RTC and SDN controller, do you mean RTC mechanism can be used to distribute routes to SDN controller? I'd agree that is another possible use case of RTC.

Best regards,
Jie

> From: UTTARO, JAMES [mailto:[email-masked]]
> Sent: Saturday, June 21, 2014 3:02 AM
> To: Dongjie (Jimmy); [email-masked]'
> Subject: RE: New Version Notification for
> Why? A picture would be quite useful here. I am struggling to understand a
> hierarcacal design that then needs RT-C?? Maybe if the VPNs are highly
> localized.. BTW isn't RT-C a good candidate to off net to an SDN controller?
> IMO that should be the future.
> 
> Jim Uttaro
>

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 23 Jun 2014 02:59:24 +0000

Hi Adam,

If RD is part of the RTC routes, this problem will not occur:) While as analyzed in rfc 4684, this requires more RTC routes being exchanged and kept in the local AS.

I agree that add-path is a candidate solution to this problem, while when enabling add-path on RTC AFI/SAFI, we still need to specify the rules of which RTC routes should be advertised to solve this problem. Or we just update the rules of RTC advertisement as the other RTC related new drafts? 

Many thanks,
Jie

> From: Vitkovský Adam [mailto:[email-masked]]
> Sent: Sunday, June 22, 2014 3:20 AM
> To: Robert Raszuk
> Subject: RE: [Idr] New Version Notification for
> Well we could have that one solved if RD would be part of the RTC routes :)
> (though I understand one could use RTC for non VPN routes as well so...)
> > draft-dong-idr-rtc-hierarchical- rr-00.txt
> >
> > Hi Adam,
> >
> > On Sat, Jun 21, 2014 at 7:19 PM, Vitkovský Adam
> > <[email-masked]>
> > wrote:
> > > Hi Jie,
> > > Well the correctly designed MPLS networks use unique per PE per VRF
> > > RDs
> > in form of RID:xxxxx.
> > different PEs unique so RRs will select all as best path and
> > distribute them accordingly, which solves this and many other issues.
> > >
> > > In other words if someone tries to shoot himself in the foot I
> > > wouldn't
> > develop bulletproof boots for him.
> > >
> > > adam
> > >> > has been successfully submitted by Jie Dong and posted to the
> > >> > IETF
> > >> repository.
> > >> >
> > >> > https://datatracker.ietf.org/doc/draft-dong-idr-rtc-hierarchical-
> > >> > rr
> > >> > /
> > >> > Htmlized:

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 23 Jun 2014 08:08:00 +0200

Hi Jie,

> we still need to specify the rules of which RTC routes should be
advertised to
> solve this problem.

Not really,  Just use add-paths option "all" and your scenarios will work
just fine ;)

​Cheers,
R.​

​> ​
On Mon, Jun 23, 2014 at 4:59 AM, Dongjie (Jimmy) <[email-masked]>
wrote:

> Hi Adam,
>
> If RD is part of the RTC routes, this problem will not occur:) While as
> analyzed in rfc 4684, this requires more RTC routes being exchanged and
> kept in the local AS.
>
> I agree that add-path is a candidate solution to this problem, while when
> enabling add-path on RTC AFI/SAFI, we still need to specify the rules of
> which RTC routes should be advertised to solve this problem. Or we just
> update the rules of RTC advertisement as the other RTC related new drafts?
>
> Many thanks,
> Jie
>
> > > >> > Title:              Extensions to RT-Constrain in Hierarchical
> Route
> > Reflection
> > > >> > Scenarios

---

From: <[email-masked]>
Date: Mon, 23 Jun 2014 07:31:36 +0000

I can remember some people that does not want to implement “add-path all” ☺

From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Monday, June 23, 2014 08:08
To: Dongjie (Jimmy)
Cc: [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Jie,

> we still need to specify the rules of which RTC routes should be advertised to
> solve this problem.

Not really,  Just use add-paths option "all" and your scenarios will work just fine ;)

​Cheers,
R.​

​> ​
On Mon, Jun 23, 2014 at 4:59 AM, Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Adam,

Many thanks,
Jie

> From: Vitkovský Adam [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: Dongjie (Jimmy); [email-masked]<mailto:[email-masked]>
> > From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert
> > Raszuk
> > Cc: Dongjie (Jimmy); [email-masked]<mailto:[email-masked]>
> > <[email-masked]<mailto:[email-masked]>>
> > wrote:
> > > Hi Jie,
> > >> From: Idr [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Dongjie
> > >> (Jimmy)
> > >> To: [email-masked]<mailto:[email-masked]>
> > >> > submission until the htmlized version and diff are available at
> > tools.ietf.org<http://tools.ietf.org>.
> > >> >
> > >> > The IETF Secretariat
> > >>
> > >> [email-masked]<mailto:[email-masked]>
> > > [email-masked]<mailto:[email-masked]>
_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 23 Jun 2014 07:47:35 +0000

Hi Robert,

Maybe add-paths option “all” is somewhat overkill for solving this scenario? :)

Best regards,
Jie

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Monday, June 23, 2014 2:08 PM
To: Dongjie (Jimmy)
Cc: Vitkovský Adam; [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Jie,

​Cheers,
R.​

​> ​

Many thanks,
Jie

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 23 Jun 2014 12:06:51 +0200

Well I don't think this is overkill. Keep in mind that this is only
localized to your RR1-RR2 and RR1-RR3 sessions ...

R.

On Mon, Jun 23, 2014 at 9:47 AM, Dongjie (Jimmy) <[email-masked]>
wrote:

> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Robert
> Raszuk
> *Sent:* Monday, June 23, 2014 2:08 PM
> *To:* Dongjie (Jimmy)
> *Cc:* Vitkovský Adam; [email-masked]
>
> *Subject:* Re: [Idr] New Version Notification for
> > > tools.ietf.org.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 23 Jun 2014 11:24:15 +0000

Jie,

	I still do not understand the architecture of hierarcacal RRs for the VPNV4 AF. As evidenced by your diagram and associated text probably not a good idea.  Regardless of that here are my thoughts, concerns and the where RT-C may have utility.

RT-C Premise..

The reason articulated in the original RT-C draft was based on the notion that VPNs would be highly localized in nature and where there associated routing state could also be contained in a sub-graph of the topology. My experience is that this is not the case. The major advertisers of routing state are big organizations that have footprint spanning multiple continents across the globe. 

A benefit of RT-C may be to limit the amount of routing state sent to a PE when a soft refresh, VRF add or re-boot occurs. The savings here is CPU cycles, ease of HOL blocking for those implementations that do not discriminate between routes incrementally added and a new VRF and reducing reboot time.

For these limited benefits full blown RT-C is not something I would deploy as I risk fate sharing, disconnected VPNs ( Based on Topology ), etc... For all of the business needs competing for cycles/windows it is difficult to understand the cost/benefit..

SDN Control

 RT-C like filter deployed from an SDN controller may be the way to go. As the RR/Route Server topology can easily scale to millions of routes we can focus our energy on reducing the flood of routing state to an individual PE, between providers or even between RRs although the last one holds little interest for me.

Jim Uttaro

-----Original Message-----
From: Dongjie (Jimmy) [mailto:[email-masked]] 
Sent: Sunday, June 22, 2014 10:42 PM
To: UTTARO, JAMES; [email-masked]'
Subject: RE: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Jim,

PE1---RR2
         \
          \    
PE2-       RR1
    \     /
     \   /
      RR3
     /
    /
PE3-

Best regards,
Jie

---

From: Vitkovský Adam <[email-masked]>
Date: Mon, 23 Jun 2014 14:14:53 +0000

James,
Think of the filters between inter-as-RRs and PEs for example used to limit the exposure of MNCs prefixes versus the local corporations prefixes to other regions. These could be replaced by the RT-C not requiring "manual" maintenance. 
But I agree that the risk of discontinuous VPNs based on some arbitrary topology between RRs is an unacceptable burden for full blown RT-C deployment, though my understanding is that this could only happen in hierarchical RR topologies.
However I see no need to alleviate the number of iBGP sessions between intra-as-RRs with yet another level of hierarchy. 

adam
> From: Idr [mailto:[email-masked]] On Behalf Of UTTARO, JAMES
> Sent: Monday, June 23, 2014 1:24 PM
> To: 'Dongjie (Jimmy)'; [email-masked]'
> 	I still do not understand the architecture of hierarcacal RRs for the
> VPNV4 AF. As evidenced by your diagram and associated text probably not a
> good idea.  Regardless of that here are my thoughts, concerns and the where
> RT-C may have utility.
> 
> RT-C Premise..
> 
> The reason articulated in the original RT-C draft was based on the notion that
> VPNs would be highly localized in nature and where there associated routing
> state could also be contained in a sub-graph of the topology. My experience
> is that this is not the case. The major advertisers of routing state are big
> organizations that have footprint spanning multiple continents across the
> globe.
> 
> A benefit of RT-C may be to limit the amount of routing state sent to a PE
> when a soft refresh, VRF add or re-boot occurs. The savings here is CPU
> cycles, ease of HOL blocking for those implementations that do not
> discriminate between routes incrementally added and a new VRF and
> reducing reboot time.
> 
> For these limited benefits full blown RT-C is not something I would deploy as
> I risk fate sharing, disconnected VPNs ( Based on Topology ), etc... For all of
> the business needs competing for cycles/windows it is difficult to understand
> the cost/benefit..
> 
> SDN Control
> 
>  RT-C like filter deployed from an SDN controller may be the way to go. As the
> RR/Route Server topology can easily scale to millions of routes we can focus
> our energy on reducing the flood of routing state to an individual PE,
> between providers or even between RRs although the last one holds little
> interest for me.
> 
> Jim Uttaro
> 
> 2. RR2 and RR3 select the best RTC route, add their cluster_id to the
> cluster_list and advertise the RTC route to RR1 respectively.
> 3. RR1 selects one RTC route as best (let's say RTC route from RR2), adds its
> cluster_id into the cluster_list and advertises it to RR2 and RR3.
> 4. Then RR2 would receive the RTC route from RR1 with its own cluster_id in
> the cluster_list and will drop this RTC route.
> For a network with hierarchical RR, RTC can also be useful to control the VPN
> routes being distributed only to PEs which have the corresponding customers
> connected.
> 
> I'm not quite clear about your comments about RTC and SDN controller, do
> you mean RTC mechanism can be used to distribute routes to SDN
> controller? I'd agree that is another possible use case of RTC.
> 
> Best regards,
> Jie
> 
> > Why? A picture would be quite useful here. I am struggling to
> > understand a hierarcacal design that then needs RT-C?? Maybe if the
> > VPNs are highly localized.. BTW isn't RT-C a good candidate to off net to an
> SDN controller?
> > > http://www.ietf.org/internet-drafts/draft-dong-idr-rtc-hierarchical-
> > > rr
> > > -00.txt
> > > Status:

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 23 Jun 2014 14:24:57 -0400

> I am not worried about efficiency here.

I am. :-)

I've been quiet on the thread partially because I don't have any better
observations on the nature of the fix to contribute.  But I do have an
observation about the shape of the solution.

Solutions that rely on meddling with the existing route reflection semantics
in the cluster list will have to deal with the fact that there can be
significant "noise" in convergence events as part of a hierarchical RR
environment.  The sort of "ping pong" that happens is certainly one negative
thing that discourages such deployments.

Where this is particularly messy for VPN environments is that if the full
set of VPN routes is prevented from being flooded across RRs in the
hierarchy with whatever solution we come up with, then a RT-C reconvergence
event could result in a massive thrashing of routes among the reflectors.

Except in very strict hierarchies, RT-C is probably not beneficial to VPN
route distribution.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 23 Jun 2014 14:26:19 -0400

> Not really,  Just use add-paths option "all" and your scenarios will work

Add-paths are local between pairs of routers.  We'd really need distinct
RD-type identifiers to cover the problem.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 23 Jun 2014 20:56:39 +0200

Hi Jeff,

I don't think so.

Notice that the issue is not that we are suppressing presence of RTC routes
due to selection of some other paths. That is the only case which unique RD
for RTC NLRI could help with.

The issue here is with filtered distribution vector betwen top hierarchy
and lower layer for one RR peer which happen to be the src of the best path
of such RTC NLRI.

Also while digesting the problem keep in mind that operation of RTC is
session oriented and results in suppression of real VPN route distribution
to top level RR via such node.

In fact while thinking about your mail I think there is even simpler
alternative to use of add-paths as the solution ... Just use at least three
lower layer RRs - that way top hierarchy will always have at least two VPN
paths and that should be sufficient :) Operationally that would be
additional IBGP session from each PE.

Best,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 23 Jun 2014 15:04:20 -0400

[reformatting to avoid the top-post]

> for RTC NLRI could help with.

The suppression is incidental to the problem: RT-C needs the route to bounce
back from a RR-client when it's another RR.  Cluster list is causing that
filtering to happen.

As I try to get a handle on possible solutions, one thought had been that
the reflection was itself already that bounce-back.  An implementation
simply keeps the looped route (many discard such loops) and uses that for
it's RT-C flooding state.

However, the RR ping-pong due to inter-cluster/client changes may result in
partial meshes of flooding forming among the various RRs with resultant
thrashing of VPN routes.  This is a Bad Thing.

> and lower layer for one RR peer which happen to be the src of the best path

I agree with your observation from this perspective.

I urge you to reframe this perspective to arbitrary meshes of reflectors,
perhaps with multiple cluster members.

That said, the simplest solution spaces will be easiest to see (I think) in
the simple example Jie presented.

> to top level RR via such node.

I'm always keeping this in mind. :-)

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 23 Jun 2014 21:24:02 +0200

Hi Jeff,

Maybe I am missing something fundamental, but the lecture of your below
note sort of shed a completely new light on this entire "issue".

Let's step back for a second and ask ourselves why top level RR would not
send default RT (send all) RTC NLRI to it's lower layer clients ?

That way top level RR will learn all VPN routes and distribute them as
seems fit to lower level RRs according to atomic RTC routes they will push
to him.

If so I would call this entire "issue" a misconfiguration and move on ;)

best,
r.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 23 Jun 2014 22:41:24 +0000

Adam,

	I am uncomfortable turning over route distribution to a protocol. IMO the risk for a L3VPN network with sites that are geographically spread does not warrant the gain. I would suggest the working group tee this up with the IRS2 team as a use case.

Jim Uttaro

-----Original Message-----
From: Vitkovský Adam [mailto:[email-masked]] 
Sent: Monday, June 23, 2014 10:15 AM
To: UTTARO, JAMES; 'Dongjie (Jimmy)'; [email-masked]'
Subject: RE: New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

James,

adam

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 24 Jun 2014 01:33:54 +0000

The sideeffect of sending default RT  by the top-level RR is that some VPN routes would be unnecessarily sent to that top-level RR. How about adding RD to the RT membership NLRI (i.e., the RT membership NLRIs originated from different PEs are distinguishable) while on a session only advertising one RT membership NLRI for a given RT (i.e., all the other RT membership NLRIs containing that RT would be suppressed)?

Best regards,
Xiaohu

From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Tuesday, June 24, 2014 3:24 AM
To: Jeffrey Haas
Cc: [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Jeff,

Maybe I am missing something fundamental, but the lecture of your below note sort of shed a completely new light on this entire "issue".

Let's step back for a second and ask ourselves why top level RR would not send default RT (send all) RTC NLRI to it's lower layer clients ?

That way top level RR will learn all VPN routes and distribute them as seems fit to lower level RRs according to atomic RTC routes they will push to him.

best,
r.

On Mon, Jun 23, 2014 at 9:04 PM, Jeffrey Haas <[email-masked]<mailto:[email-masked]>> wrote:
[reformatting to avoid the top-post]

-- Jeff

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 24 Jun 2014 01:59:43 +0000

Hi Jeff,

> observation about the shape of the solution.

Your comments are always welcome:)

> Solutions that rely on meddling with the existing route reflection semantics in
> the cluster list will have to deal with the fact that there can be significant
> "noise" in convergence events as part of a hierarchical RR environment.  The
> sort of "ping pong" that happens is certainly one negative thing that
> discourages such deployments.

Agree that we need to avoid the oscillation of RTC route. Since the solution proposes to send an alternate RTC route only to the source of best route and increase its visibility to the RTC routes, with a well designed RR topology this should not change the best path selection on the source of the best RTC route. Or maybe you have a specific topology in mind where the "ping pong" of RTC route could happen? 

BTW, the existing RTC also modifies the route reflection semantics with the rules in section 3.2 of RFC 4684.

> Where this is particularly messy for VPN environments is that if the full set of
> VPN routes is prevented from being flooded across RRs in the hierarchy with
> whatever solution we come up with, then a RT-C reconvergence event could
> result in a massive thrashing of routes among the reflectors.

That is exactly what we should avoid:) We need to minimize the RTC route reconvergence and make sure each convergence can produce the correct VPN route distribution graph.

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 24 Jun 2014 02:47:08 +0000

Hi Jim,

> Sent: Monday, June 23, 2014 7:24 PM
> 	I still do not understand the architecture of hierarcacal RRs for the VPNV4
> AF. As evidenced by your diagram and associated text probably not a good idea.
> Regardless of that here are my thoughts, concerns and the where RT-C may
> have utility.

Maybe the diagram is not clear enough. In one word the problem is the RTC route advertised by higher layer RR is discarded by the lower layer RR, which would block the VPN route distribution from lower layer RR to the higher layer RR, and cause some PEs cannot receive those VPN routes.

You may also find some description about the problem in Robert and Jeff's mails:)

> state could also be contained in a sub-graph of the topology. My experience is
> that this is not the case. The major advertisers of routing state are big
> organizations that have footprint spanning multiple continents across the globe.
> 
> A benefit of RT-C may be to limit the amount of routing state sent to a PE when
> a soft refresh, VRF add or re-boot occurs. The savings here is CPU cycles, ease
> of HOL blocking for those implementations that do not discriminate between
> routes incrementally added and a new VRF and reducing reboot time.
> 
> For these limited benefits full blown RT-C is not something I would deploy as I
> risk fate sharing, disconnected VPNs ( Based on Topology ), etc... For all of the
> business needs competing for cycles/windows it is difficult to understand the
> cost/benefit..

Agree that RTC may not be the same beneficial for all networks, and analysis of the cost/benefit is needed before deployment. As contrast to full blown RTC, are you indicating you may deploy RTC partially?

> our energy on reducing the flood of routing state to an individual PE, between
> providers or even between RRs although the last one holds little interest for me.

IMO one of the objectives of RTC in traditional scenarios is also to reduce the unnecessary flooding of VPN routes to PEs.

Agree that controlling the route distribution in SDN scenario would be a valid use case. And if the controller and the PE are direct BGP peers, ORF may also help.

Many thanks,
Jie

> 2. RR2 and RR3 select the best RTC route, add their cluster_id to the cluster_list
> and advertise the RTC route to RR1 respectively.
> 4. Then RR2 would receive the RTC route from RR1 with its own cluster_id in the
> cluster_list and will drop this RTC route.
> I'm not quite clear about your comments about RTC and SDN controller, do you
> mean RTC mechanism can be used to distribute routes to SDN controller? I'd
> agree that is another possible use case of RTC.
> 
> Best regards,
> Jie
>

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 24 Jun 2014 09:19:27 +0200

Hi Xu,

IMO changing the format of RTC NLRI is the worst idea of all discussed so
far on this thread. Sorry.

It is not backwards compatible and will result in heavy deployment mess.

Note that this will likely require new SAFI and transition scheme as well
as moving current RFC4864 to historic.

NLRI is a database key in RTC.

So by all means you are free to propose a new draft on that and let the IDR
WG (maybe with L3VPN input) decide.

If anything I am eager to read the "Justification section" of the new
proposal ....

Cheers,
R.

On Tue, Jun 24, 2014 at 3:33 AM, Xuxiaohu <[email-masked]> wrote:

>  The sideeffect of sending default RT  by the top-level RR is that some
> VPN routes would be unnecessarily sent to that top-level RR. How about
> adding RD to the RT membership NLRI (i.e., the RT membership NLRIs
> originated from different PEs are distinguishable) while on a session only
> advertising one RT membership NLRI for a given RT (i.e., all the other RT
> membership NLRIs containing that RT would be suppressed)?
>
>
>
> Best regards,
>
> Xiaohu
>
>
>

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 24 Jun 2014 08:38:58 +0000

Hi Robert,

I agree that changing the format of RTC NLRI is the worst idea from the backwards compatibility perspective. However, if you need the RTC to be applicable in any BGP topology (e.g., arbitrary meshes of reflectors), it seems worthwhile to consider that possibility. Furthermore, with that approach, the following complex advertisement rules (see below) would be greatly simplified, in other words, less changes to the basic BGP advertisement procedures would be required.

When processing RT membership NLRIs received from internal iBGP
   peers, it is necessary to consider all available iBGP paths for a
   given RT prefix, for building the outbound route filter, and not just
   the best path.

   In addition, when advertising Route Target membership information
   sourced by the local autonomous system to an iBGP peer, a BGP speaker
   shall modify its procedure to calculate the BGP attributes such that
   the following apply:

   i.   When advertising RT membership NLRI to a route-reflector client,
        the Originator attribute shall be set to the router-id of the
        advertiser, and the Next-hop attribute shall be set of the local
        address for that session.

   ii.  When advertising an RT membership NLRI to a non-client peer, if
        the best path as selected by the path selection procedure
        described in Section 9.1<http://tools.ietf.org/html/rfc4684#section-9.1> of the base BGP specification [4<http://tools.ietf.org/html/rfc4684#ref-4>] is a
        route received from a non-client peer, and if there is an
        alternative path to the same destination from a client, the
        attributes of the client path are advertised to the peer.

   The first of these route advertisement rules is designed such that
   the originator of an RT membership NLRI does not drop an RT
   membership NLRI that is reflected back to it, thus allowing the route
   reflector to use this RT membership NLRI in order to signal the
   client that it should distribute VPN routes with the specific target
   towards the reflector.

   The second rule allows any BGP speaker present in an iBGP mesh to
   signal the interest of its route reflection clients in receiving VPN
   routes for that target.

BTW, it said in RFC4684

   An alternative solution to the procedure given above would have been

   to source different routes per PE, such as NLRI of the form

   {originator-id, route-target}, and to aggregate them at the edge of

   the network.  The solution adopted is considered advantageous over

   the former in that it requires less routing-information within a

   given AS.

IMHO, by using the suppression trick as mentioned before, it seems that the approach as mentioned above doesn’t require more routing-information within a given AS☺

Best regards,
Xiaohu

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Tuesday, June 24, 2014 3:19 PM
To: Xuxiaohu
Cc: Jeffrey Haas; [email-masked]
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Hi Xu,

IMO changing the format of RTC NLRI is the worst idea of all discussed so far on this thread. Sorry.

Note that this will likely require new SAFI and transition scheme as well as moving current RFC4864 to historic.

So by all means you are free to propose a new draft on that and let the IDR WG (maybe with L3VPN input) decide.

If anything I am eager to read the "Justification section" of the new proposal ....

Cheers,
R.

On Tue, Jun 24, 2014 at 3:33 AM, Xuxiaohu <[email-masked]<mailto:[email-masked]>> wrote:
The sideeffect of sending default RT  by the top-level RR is that some VPN routes would be unnecessarily sent to that top-level RR. How about adding RD to the RT membership NLRI (i.e., the RT membership NLRIs originated from different PEs are distinguishable) while on a session only advertising one RT membership NLRI for a given RT (i.e., all the other RT membership NLRIs containing that RT would be suppressed)?

Best regards,
Xiaohu

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 24 Jun 2014 09:10:54 -0400

Jie,

> BTW, the existing RTC also modifies the route reflection semantics with the rules in section 3.2 of RFC 4684.

Arguably, incompletely working through the impacts of this are what got us
into this problem. :-)

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 24 Jun 2014 09:20:20 -0400

Robert,

> If so I would call this entire "issue" a misconfiguration and move on ;)

As mentioned elsewhere in-thread, this does break some of the semantic of
the route distribution graph.  But let's focus on this for a moment as
a point of discussion.

Assume for a moment that we decide that the route selection behavior of a
mesh of reflectors is something we don't care to meddle with too much in
order to fix RT-C's behavior in this scenario.  What we then have is
something that more strongly resembles the more typical deployment scenario
for RT-C: PE's run RT-C toward their first-hop RR and the prime benefit is
pruning paths from PE through the core to another PE.

If we treat the/a mesh of RRs as a place where we don't try to prune paths but
realize that much of our benefit remains near the edges, then reducing this
to the case where we distribute all VPN routes within the mesh may simply
solve the problem.  

In such a case, we wouldn't want to generate a default RT-C route for a few
reasons:
- We can't properly limit its distribution to the RR infrastructure.
- We don't want to interfere with a legitimate default route.

We should still have implementations *act* as if they received default - or
effectively just don't implement RT-C filtering.  The main circumstance
necessary to do this is to know when the remote RR-client is also an RR.
Nothing in the protocol is going to tell us this a priori; we need this via
configuration.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 24 Jun 2014 09:23:15 -0400

> It is not backwards compatible and will result in heavy deployment mess.

Agreed.  But the fun part is the original authors told us this had been
considered.  From RFC 4684:

:  An alternative solution to the procedure given above would have been
:  to source different routes per PE, such as NLRI of the form
:  {originator-id, route-target}, and to aggregate them at the edge of
:  the network.  The solution adopted is considered advantageous over
:  the former in that it requires less routing-information within a
:  given AS.

The aggregation procedures, however, would have been messy.  The thought of
doing community coloring to limit the RT distribution graph for specific
AFI/SAFI has the same issue.  While I think the idea is still worth
considering, working through the implementation details for that have given
me a nasty headache.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 24 Jun 2014 09:28:00 -0400

Jim,

> 	I am uncomfortable turning over route distribution to a protocol. IMO the risk for a L3VPN network with sites that are geographically spread does not warrant the gain. I would suggest the working…

While certainly worth discussing on the i2rs list, I'm not 100% clear on
what you think the proposal would be.

If the general observation is around "discontiguous VPN islands", static
provisioning (with API help from something like i2rs) is certainly one way
to tackle that.  But even so, I've been kicking around the idea that tools
like RT-C (similar in form but not in exact detail) may be useful for
helping to determine the shape of such islands.

Presented somewhat abstractly, it'd be nice to keep large amounts of state
outside of devices that don't need to carry it.  Discovering the set of
devices that must carry that state and building dynamic routing connectivity
among them is interesting in an academic sense - but I suspect violates many
principles that operators need for predictability in their networks.

-- Jeff

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Tue, 24 Jun 2014 14:18:01 +0000

Jeff,

	My thinking is that we have the routing state available in the topology and we can export that to a controller which can then determine the specific distribution tree for a given VPN. I2RS can subsequently "program" the PEs and/or RRs ( Don't really want to limit the route at the RRs ) accordingly. Actually another approach could be looking at the import list of all of the VRFs and construct the distribution tree without looking at each path. This may be simpler if we are only concerned with the state being sent to the PEs..

Jim Uttaro

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Tuesday, June 24, 2014 9:28 AM
To: UTTARO, JAMES
Cc: 'Vitkovsk? Adam'; 'Dongjie (Jimmy)'; [email-masked]'
Subject: Re: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-rr-00.txt

Jim,

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 24 Jun 2014 10:56:44 -0400

Jim,

> 	My thinking is that we have the routing state available in the topology and we can export that to a controller which can then determine the specific distribution tree for a given VPN. I2RS can sub…

That work is definitely in-scope for I2RS.

-- Jeff

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 25 Jun 2014 01:42:34 +0000

Jeff, 

So let's work through it completely for valid network scenarios:)

Best regards,
Jie

> Sent: Tuesday, June 24, 2014 9:11 PM
> To: Dongjie (Jimmy)
> Cc: Jeffrey Haas; Robert Raszuk; [email-masked]
> Arguably, incompletely working through the impacts of this are what got us into
> this problem. :-)
> 
> -- Jeff

---

From: Xuxiaohu <[email-masked]>
Date: Wed, 25 Jun 2014 03:51:57 +0000

> Sent: Tuesday, June 24, 2014 9:23 PM
> To: Robert Raszuk
> Cc: Xuxiaohu; Jeffrey Haas; [email-masked]

If I understand the above text correctly, it seems that " aggregate them at the edge of the network" just meant to aggregate them (i.e., from {originator-id, route-target} to {origin AS, route-target}) when advertising them outside of the originating AS.

> The aggregation procedures, however, would have been messy.  The thought

Compared to the aggregation procedure which is executed on eBGP sessions, the suppression procedure which would be executed on IBGP sessions is a real headache.

> of doing community coloring to limit the RT distribution graph for specific
> AFI/SAFI has the same issue.  While I think the idea is still worth considering,
> working through the implementation details for that have given me a nasty
> headache.

How about making the suppression as an option? For example, if it is safe to enable the suppression in some topology(e.g., the hierarchical RR topology ), then enable it. Otherwise (e.g., arbitrary meshes of RRs), disable it.  Compared to the current procedure which is based on the granularity of {origin AS, route-target}, for those scenarios which could be dealt with by the current procedure (e.g., the hierarchical RR topology), the new procedure could deal with them at almost the same cost (i.e., no more routing-information by enabling the suppression). For those scenarios which could hardly be dealt with by the current procedure (e.g., arbitrary meshes of RRs), the new procedure could deal with them as well just at a relatively higher cost (i.e., by disabling the suppression), anyway, better than nothing.

Best regards,
Xiaohu

---

From: Vitkovský Adam <[email-masked]>
Date: Fri, 27 Jun 2014 11:52:33 +0000

Hi folks,

Regarding the rule 3.2.i of RFC4684. 

First consider scenario a):
a)
RR----PE1

Why would I want the RR to learn the PE's prefixes if there's no one interested in them (no other clients interested to receive NRLI for given RTs) as PE1 is the only one that advertised the Route Target membership information to RR. 

Now let's consider scenario b):
b)
PE2----RR----PE1

Now there is another client (PE2) connected to the RR and it happens to be interested in receiving NRLI for a given RT as designated by the advertised Route Target membership information for the given RT. 
Ok now that there are interested receivers for a given NRLI on both sessions - RR should start learning the NRLIs from clients so that it can disseminate the information accordingly. 

In order to start learning the NRLIs RR has to advertise a desire to receive them and to do that we can use modified rule 3.2.ii.

        When advertising an RT membership NLRI to a client peer, if
        the best path as selected by the path selection procedure
        described in Section 9.1 of the base BGP specification [4] is a
        route received from that client peer in particular, and if there is an
        alternative path to the same destination from another client or a non-client, the
        attributes of the other path are advertised to the peer. 

Now thinking about it only one rule for reflection of RT membership NLRI is enough I guess. 
If you received RT membership NLRI from only one peer don't advertise it until you have received the RT membership NLRI with the same RT from another peer. 
Then you can advertise the alternate (not the one that the peer sent you) RT membership NLRI to a given peer (simple split horizon). 

Then there's no need to modify the update so that it can be accepted by the originator (as in 3.2.i.). 
And this also solves the hierarchical RR problem as well. 

I'd like the think of it as with multicast flow where RR is like RP and PEs are like Sources + Receivers. 
There's no need for the RP (RR) to receive the mcast stream (e.g. VPN Prefixes) when there's just one Source/Receiver (PE) connected to it. 
A simple state on the RP saying there is a particular stream available at this particular source is enough (i.e. RT membership NLRI on RR). 
Once other source/receivers (PEs) are connected to the RP(RR) and signal their desire to receive the stream (via RT membership NLRI). 
Then RP (RR) decides the desire receive the stream (RT membership NLRI) is worth spreading and disseminates the information to the interested sources/receivers (PEs). 
Once that happens sources/receivers (PEs) will start sending the mcast stream (e.g. VPN Prefixes) towards the RP (RR) which than reflects this mcase streams to all interested sources/receivers (PEs). 

I believe the concerns presented in draft-dong-idr-rtc-hierarchical-rr-00 could be resolved by add-path where at least one additional path is advertised along with the best-path. 
But still rule 3.2.i. in RFC4684 is very harmful in scenarios where RRs are clients of other RRs (same level or different level). 

 adam

> Sent: Monday, June 23, 2014 4:59 AM
> To: Vitkovský Adam; Robert Raszuk
> Cc: [email-masked]
> Subject: RE: [Idr] New Version Notification for draft-dong-idr-rtc-hierarchical-
> rr-00.txt
> 
> Hi Adam,
> 
> If RD is part of the RTC routes, this problem will not occur:) While as analyzed
> in rfc 4684, this requires more RTC routes being exchanged and kept in the
> local AS.
> 
> enabling add-path on RTC AFI/SAFI, we still need to specify the rules of which
> RTC routes should be advertised to solve this problem. Or we just update the
> rules of RTC advertisement as the other RTC related new drafts?
> 
> Many thanks,
> Jie
> 
> > Well we could have that one solved if RD would be part of the RTC
> > routes :) (though I understand one could use RTC for non VPN routes as
> > well so...) Anyways so than as you mentioned add-path on the RTC AF
> > seems as the best option then.
> >
> > adam
> > > Jie is describing RT routes not VPN routes. Your story does not
> > > apply to RT routes as format of RT NLRI is well defined by RFC ;)
> > >
> > > > Well the correctly designed MPLS networks use unique per PE per
> > > > VRF RDs
> > > in form of RID:xxxxx.
> > > >> draft-dong-idr-rtc- hierarchical-rr-00.txt
> > > >>
> > > >> Hi all,
> > > >>
> > > >> > http://www.ietf.org/internet-drafts/draft-dong-idr-rtc-hierarch
> > > >> > ic
> > > >> > al
> > > >> > -rr
> > > >> > -00.txt
> > > >> > Status:
> > > >> > Please note that it may take a couple of minutes from the time
> > > >> > of submission until the htmlized version and diff are available
> > > >> > at
> > > tools.ietf.org.
> > > >> >
> > > >> > The IETF Secretariat
> > > >>

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Sat, 28 Jun 2014 01:48:58 +0000

Hi Adam,

Thanks for your analysis, and please see my replies inline:

> Why would I want the RR to learn the PE's prefixes if there's no one interested
> in them (no other clients interested to receive NRLI for given RTs) as PE1 is the
> only one that advertised the Route Target membership information to RR.

Agree that in this case RR does not need to advertise RT membership info to PE1. 

> Now there is another client (PE2) connected to the RR and it happens to be
> interested in receiving NRLI for a given RT as designated by the advertised
> Route Target membership information for the given RT.
> Ok now that there are interested receivers for a given NRLI on both sessions -
> RR should start learning the NRLIs from clients so that it can disseminate the
> information accordingly.
> 
> In order to start learning the NRLIs RR has to advertise a desire to receive them
> and to do that we can use modified rule 3.2.ii.
> 
> If you received RT membership NLRI from only one peer don't advertise it until
> you have received the RT membership NLRI with the same RT from another
> peer.
> Then you can advertise the alternate (not the one that the peer sent you) RT
> membership NLRI to a given peer (simple split horizon).

> And this also solves the hierarchical RR problem as well.

Partly agree. Advertising an alternate RT membership info route is also proposed in draft-dong-idr-rtc-hierarchical-rr, in addition the draft also specifies constraints on that "alternate path" considering the BGP loop detection mechanisms. When the RR has multiple alternates, there has to be some rule for choosing the alternate and advertising it to the sender of the best.

I'd agree that we may generalize the rule for the distribution of RT membership info if it works through all possible legitimate scenarios.

> There's no need for the RP (RR) to receive the mcast stream (e.g. VPN Prefixes)
> when there's just one Source/Receiver (PE) connected to it.
> A simple state on the RP saying there is a particular stream available at this
> particular source is enough (i.e. RT membership NLRI on RR).
> Once other source/receivers (PEs) are connected to the RP(RR) and signal their
> desire to receive the stream (via RT membership NLRI).
> Then RP (RR) decides the desire receive the stream (RT membership NLRI) is
> worth spreading and disseminates the information to the interested
> sources/receivers (PEs).
> Once that happens sources/receivers (PEs) will start sending the mcast stream
> (e.g. VPN Prefixes) towards the RP (RR) which than reflects this mcase streams
> to all interested sources/receivers (PEs).

Nice analogy:) While in BGP IMO we also need to consider some existing rules (e.g. best path selection, loop detection), if we don't want to change them too much.

> I believe the concerns presented in draft-dong-idr-rtc-hierarchical-rr-00 could
> be resolved by add-path where at least one additional path is advertised along
> with the best-path.

As I mentioned in one previous mail, add-paths also need some rules to select the appropriate additional path(s), unless "add-paths all" is used. And if the RR knows which alternate path would work, just advertise that path would be enough.

> But still rule 3.2.i. in RFC4684 is very harmful in scenarios where RRs are clients
> of other RRs (same level or different level).

Agree that we may find some better way to achieve the same effect.:)

Best regards,
Jie

> > If RD is part of the RTC routes, this problem will not occur:) While
> > as analyzed in rfc 4684, this requires more RTC routes being exchanged
> > and kept in the local AS.
> >
> > I agree that add-path is a candidate solution to this problem, while
> > when enabling add-path on RTC AFI/SAFI, we still need to specify the
> > rules of which RTC routes should be advertised to solve this problem.
> > Or we just update the rules of RTC advertisement as the other RTC related
> new drafts?
> >
> > Many thanks,
> > Jie
> >
> > > routes :) (though I understand one could use RTC for non VPN routes
> > > as well so...) Anyways so than as you mentioned add-path on the RTC
> > > AF seems as the best option then.
> > >
> > > adam
> > > > >> > A new version of I-D,
> > > > >> > https://datatracker.ietf.org/doc/draft-dong-idr-rtc-hierarchi
> > > > >> > ca
> > > > >> > l-
> > > > >> > rr
> > > > >> > /
> > > > >> > Htmlized:
> > > > >> > http://tools.ietf.org/html/draft-dong-idr-rtc-hierarchical-rr
> > > > >> > -0
> > > > >> > 0
> > > > >> >
> > > > >> >
> > > > >> > Abstract:
> > > > >> > Please note that it may take a couple of minutes from the
> > > > >> > time of submission until the htmlized version and diff are
> > > > >> > available at
> > > > tools.ietf.org.
> > > > >> >
> > > > >> > The IETF Secretariat
> > > > >>

---

From: Vitkovský Adam <[email-masked]>
Date: Mon, 30 Jun 2014 15:33:40 +0000

Hi Jie,

Thank you for your reply.
Please see my comments inline. 

> > Why would I want the RR to learn the PE's prefixes if there's no one
> > interested in them (no other clients interested to receive NRLI for
> > given RTs) as PE1 is the only one that advertised the Route Target
> membership information to RR.
> 
> > Now there is another client (PE2) connected to the RR and it happens
> > to be interested in receiving NRLI for a given RT as designated by the
> > advertised Route Target membership information for the given RT.
> > Ok now that there are interested receivers for a given NRLI on both
> > sessions - RR should start learning the NRLIs from clients so that it
> > can disseminate the information accordingly.
> >
> > In order to start learning the NRLIs RR has to advertise a desire to
> > receive them and to do that we can use modified rule 3.2.ii.
> >
> > If you received RT membership NLRI from only one peer don't advertise
> > it until you have received the RT membership NLRI with the same RT
> > from another peer.
> > Then you can advertise the alternate (not the one that the peer sent
> > you) RT membership NLRI to a given peer (simple split horizon).
> 
> Partly agree. Advertising an alternate RT membership info route is also
> proposed in draft-dong-idr-rtc-hierarchical-rr, in addition the draft also
> specifies constraints on that "alternate path" considering the BGP loop
> detection mechanisms. When the RR has multiple alternates, there has to be
> some rule for choosing the alternate and advertising it to the sender of the
> best.
> 
Yes please I agree. 
However I think the new rule should be generalized for all iBGP sessions and should replace the two existing rules. 

> I'd agree that we may generalize the rule for the distribution of RT
> membership info if it works through all possible legitimate scenarios.
> 
I'm really interested to see a legitimate scenario where a simple "split-horizon-like" rule would not work. 

> > There's no need for the RP (RR) to receive the mcast stream (e.g. VPN
> > Prefixes) when there's just one Source/Receiver (PE) connected to it.
> > A simple state on the RP saying there is a particular stream available
> > at this particular source is enough (i.e. RT membership NLRI on RR).
> > Once other source/receivers (PEs) are connected to the RP(RR) and
> > signal their desire to receive the stream (via RT membership NLRI).
> > Then RP (RR) decides the desire receive the stream (RT membership
> > NLRI) is worth spreading and disseminates the information to the
> > interested sources/receivers (PEs).
> > Once that happens sources/receivers (PEs) will start sending the mcast
> > stream (e.g. VPN Prefixes) towards the RP (RR) which than reflects
> > this mcase streams to all interested sources/receivers (PEs).
> 
> Nice analogy:) While in BGP IMO we also need to consider some existing
> rules (e.g. best path selection, loop detection), if we don't want to change
> them too much.
> 
> > I believe the concerns presented in
> > draft-dong-idr-rtc-hierarchical-rr-00 could be resolved by add-path
> > where at least one additional path is advertised along with the best-path.
> 
> As I mentioned in one previous mail, add-paths also need some rules to
> select the appropriate additional path(s), unless "add-paths all" is used. And
> if the RR knows which alternate path would work, just advertise that path
> would be enough.

Yes please you have mentioned that there might be some tricky scenarios where add-path 2 could select two paths from which neither one will be accepted. 
Would you please share the scenarios as I'm really stuck with this one and I can't seem to find a topology where this could happen. 
The only one I came up with is a triangle of RR where single PE is connected to only one of the RRs. 
But in this topology there's no need to advertise the prefixes as I mentioned in my previous mail (no other interested receivers). 

> 
> > But still rule 3.2.i. in RFC4684 is very harmful in scenarios where
> > RRs are clients of other RRs (same level or different level).
> 

adam

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 30 Jun 2014 14:45:22 -0400

Adam,

> Now let's consider scenario b):

PE1 and PE2 advertise RT-C route R to RR.
RR selects one of R(PE1) or R(PE2).
It then would go to reflect that route back to the originator.

Using standard rules, even with modification, loop suppression occurs either
by the originator or via proxy at the reflector.

The rule accommodates this case.  The fact that it also incidentally serves
as an attractor to the RR can be perceived as a bug or as a feature.  Mostly
it tends to be viewed as a feature.

> But still rule 3.2.i. in RFC4684 is very harmful in scenarios where RRs are clients of other RRs (same level or different level). 

My current thinking is that add-path - within the RR hierarchy only - is
perhaps a sufficient solution to this problem.  This would avoid the need to
implement RT-C add-path at the PEs, an important detail for faster
deployment of a fix for this issue.  (I have my doubts about the
number of providers currently experiencing this issue.)

A rough version of a proposal using add-path only among reflectors would
look like the following:

Reflector clients are tagged as being route-reflectors or not via
configuration.
Among RR clients that are also RRs:
- Do not use 4684 3.2.i/ii
- RT-C paths are advertised using add-path with the ORIGINATOR attribute
  preserved as it would otherwise be attached by a RR not doing 3.2.i

The above procedure would permit routes received by the edge of a hierarchy
of RRs to pass along RT-C routes within the hierarchy and build a loop-free
graph following standard BGP distribution rules.

At a RR that has as candidates RT-C routes either non-hierarchy routes or
hierarchy routes, active route selection is performed as normal.  When
advertising to non-RR client routers, 3.2.i is performed.  This ensures that
an active path selected using best-path originating from that router may be
advertised back as per above.

(Similarly follow 3.2.ii, if necessary.)

I have that nagging feeling that I've missed an important detail, so the
above may not survive scrutiny.  (But that's how engineering and science is
supposed to work.)

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 30 Jun 2014 20:53:06 +0200

Jeff,

> My current thinking is that add-path - within the RR hierarchy only

Just for clarity this is what I proposed earlier. I never seen a need to
run RTC SAFI with add-paths to the PEs.

Thx,
R.

On Mon, Jun 30, 2014 at 8:45 PM, Jeffrey Haas <[email-masked]> wrote:

> interested in them (no other clients interested to receive NRLI for given
> RTs) as PE1 is the only one that advertised the Route Target membership
> information to RR.
> >
> draft-dong-idr-rtc-hierarchical-rr-00 could be resolved by add-path where
> at least one additional path is advertised along with the best-path.
> > But still rule 3.2.i. in RFC4684 is very harmful in scenarios where RRs
> are clients of other RRs (same level or different level).
>

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 30 Jun 2014 14:57:38 -0400

> run RTC SAFI with add-paths to the PEs.

Ah, that detail wasn't clear to me.  If you similarly made observations
about shutting off the games with path attributes in the hierarchy,
we're 100% in agreement. :-)

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 30 Jun 2014 21:04:10 +0200

> we're 100% in agreement. :-)

​Indeed - no games with attributes is desired.

Note I also made an observation that we could make this work with today's
shipping RTC alone by sending default RT from top RRs to lower RRs.

Someone claimed that it would need to be colored and suppressed from being
send to PEs however most PEs I know do send all VPN routes to RRs they are
peering with. So this would solve the issue we are facing too I believe.

r.

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 30 Jun 2014 17:00:27 -0400

On Mon, Jun 30, 2014 at 09:04:10PM +0200, Robert Raszuk wrote:
> ???Indeed - no games with attributes is desired.

Ok. In that case we have a proposal and just need to wait until the relevant
holes are poked in it.

> send to PEs

That would have been me.

> however most PEs I know do send all VPN routes to RRs they are

Depends on the deployment, especially for people who like "planes" of RRs.

What's sort of interesting is that if the presumption is bulit-in that
add-path must be part of the solution, an add-path default RT-C route with
no-advertise might help.

-- Jeff

---

From: Shyam Sethuram <[email-masked]>
Date: Mon, 30 Jun 2014 17:29:24 -0700

> If you received RT membership NLRI from only one peer don't advertise it
until you have received the RT membership NLRI with the same RT from
another peer.
> Then you can advertise the alternate (not the one that the peer sent you)
RT membership NLRI to a given peer (simple split horizon).

This sounds like best-external advertisement ?
(Section 5, draft-ietf-idr-best-external-05)

shyam

On Fri, Jun 27, 2014 at 4:52 AM, Vitkovský Adam <[email-masked]>
wrote:

> sessions - RR should start learning the NRLIs from clients so that it can
> disseminate the information accordingly.
>
> If you received RT membership NLRI from only one peer don't advertise it
> until you have received the RT membership NLRI with the same RT from
> another peer.
> Then you can advertise the alternate (not the one that the peer sent you)
> RT membership NLRI to a given peer (simple split horizon).
>
> A simple state on the RP saying there is a particular stream available at
> this particular source is enough (i.e. RT membership NLRI on RR).
> Once other source/receivers (PEs) are connected to the RP(RR) and signal
> their desire to receive the stream (via RT membership NLRI).
> stream (e.g. VPN Prefixes) towards the RP (RR) which than reflects this
> mcase streams to all interested sources/receivers (PEs).
>
>
> I believe the concerns presented in draft-dong-idr-rtc-hierarchical-rr-00
> could be resolved by add-path where at least one additional path is
> advertised along with the best-path.
> analyzed
> which