Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

From: Stewart Bryant <[email-masked]>
Date: Mon, 04 Aug 2014 18:31:51 +0100

Yimin

A question before I reply in detail

Does this work over both an LDP signaled MPLS layer
and an RSVP-TE signaled MPLS layer?

Stewart

---

From: Yimin Shen <[email-masked]>
Date: Mon, 4 Aug 2014 17:46:53 +0000

Hi Stewart,

Yes, it works with both LDP and RSVP-TE (or other signaling protocols). This is based on the PWE3 architecture where PWE3 layer and PSN layer are decoupled. If PSN is MPLS, all we need is an MPLS tunnel to the primary (or protected) PE, and an MPLS bypass tunnel from the PLR to the protector. The two tunnels may be LDP or RSVP-TE (or another protocol). The context ID defined in this draft can be used to facilitate the setup of these tunnels.

Thanks,

/Yimin

-----Original Message-----
From: Stewart Bryant [mailto:[email-masked]] 
Sent: Monday, August 04, 2014 1:32 PM
To: Yimin Shen; Alexander Vainshtein
Cc: Yakov Rekhter; [email-masked]
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin

Does this work over both an LDP signaled MPLS layer and an RSVP-TE signaled MPLS layer?

Stewart

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 5 Aug 2014 04:47:54 +0000

Yimin, Stewart and all,
I have serious doubts regarding the possibility to combine the mechanism defined in this draft with LDP-instantiated LSPs as tunnels for PWs.

Please consider the following use case:

- The PSN is an MPLS network with a full mesh of tunnel LSPs between all the PE that is instantiated between the PEs
- ECMP exists in this network so that actually there are several equal cost LSPs between PE1 and PE2, possibly with different penultimate hops, say, P1 and P2
- The user sets up a PW between PE1 and PE2 and defines PE3 as the "protector PE" for PE2 for this PW with some CE dual-homed to PE2 and PE3. 
 - The user defines the PW in question as a "fat"(a.k.a. flow-aware) PW. As a consequence some flows in this PW pass thru P1 and some other flows  - thru P2. However,   initially all these flows reach the CE via its link to PE2
- The link between P1 and PE2 fails. As a consequence the flows within the fat PW passing thru this link are redirected redirected to PE2. At the same time, the link between P2 and PE2 remains intact, so that the flows passing thru this link are not redirected
- The result is that some of the flows in the PW in question now reach the CE via its link to PE2 and some - via its link to PE3.  Since you've explained (see http://www.ietf.org/mail-archive/web/pwe3/current/msg13879.html) that "CE is dual-home to 2 PEs over 2 distinct ACs", the results could be discouraging: e.g., consider the case when you are running an fat Ethernet PW between two IP-callable devices, and, as the result of the split, IP traffic reaches the CE via one AC but ARP - via the other AC.

I also have doubts regarding ability of LDP to take into account the relationship between the initial destination PE and the protector PE. To the best of my understanding, the only thing LDP would recognize when setting up tunnel LSP is the ""Context ID IP address" - presumably an additional loopback address of the original destination PE. I do not see how, without suitable extensions it could identify Protector PE. 

I also suspect that LDP could recognize multiple Context ID addresses as belonging to the same PE router and as a consequence merge LSPs that use this IP address as a /32 IPv4 prefix FEC. If this happens, it would break the entire mechanism in the scenarios when the same destination PE participates in multiple contexts - quite a reasonable situation in the case of MS-PWs with S-PE protection IMO, but also possible in the case of a T-PE that use different Protectors for different PWs.

 Did I miss something substantial? 

Regards, 
Sasha
________________________________________
From: Yimin Shen <[email-masked]>
Sent: Monday, August 4, 2014 8:46 PM
To: [email-masked]; Alexander Vainshtein
Cc: Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Hi Stewart,

Thanks,

/Yimin

-----Original Message-----
From: Stewart Bryant [mailto:[email-masked]]
Sent: Monday, August 04, 2014 1:32 PM
To: Yimin Shen; Alexander Vainshtein
Cc: Yakov Rekhter; [email-masked]
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin

Stewart

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Tue, 5 Aug 2014 05:01:19 +0000

This solution relies on Ctxt label lookups which means PHP is not
supported. The Ctxt Label will determine on which table to lookup the PW
label and handle the appropriate action related to this PW.

Hope this clarifies

On 05/08/14 06:47, "Alexander Vainshtein"
<[email-masked]> wrote:

>Yimin, Stewart and all,
>I have serious doubts regarding the possibility to combine the mechanism
>defined in this draft with LDP-instantiated LSPs as tunnels for PWs.
>
>- The PSN is an MPLS network with a full mesh of tunnel LSPs between all
>the PE that is instantiated between the PEs
>- ECMP exists in this network so that actually there are several equal
>cost LSPs between PE1 and PE2, possibly with different penultimate hops,
>say, P1 and P2
>- The user sets up a PW between PE1 and PE2 and defines PE3 as the
>"protector PE" for PE2 for this PW with some CE dual-homed to PE2 and
>PE3. 
> - The user defines the PW in question as a "fat"(a.k.a. flow-aware) PW.
>As a consequence some flows in this PW pass thru P1 and some other flows
>- thru P2. However,   initially all these flows reach the CE via its link
>to PE2
>- The link between P1 and PE2 fails. As a consequence the flows within
>the fat PW passing thru this link are redirected redirected to PE2. At
>the same time, the link between P2 and PE2 remains intact, so that the
>flows passing thru this link are not redirected
>- The result is that some of the flows in the PW in question now reach
>the CE via its link to PE2 and some - via its link to PE3.  Since you've
>explained (see 
>http://www.ietf.org/mail-archive/web/pwe3/current/msg13879.html) that "CE
>is dual-home to 2 PEs over 2 distinct ACs", the results could be
>discouraging: e.g., consider the case when you are running an fat
>Ethernet PW between two IP-callable devices, and, as the result of the
>split, IP traffic reaches the CE via one AC but ARP - via the other AC.
>
>I also have doubts regarding ability of LDP to take into account the
>relationship between the initial destination PE and the protector PE. To
>the best of my understanding, the only thing LDP would recognize when
>setting up tunnel LSP is the ""Context ID IP address" - presumably an
>additional loopback address of the original destination PE. I do not see
>how, without suitable extensions it could identify Protector PE.
>
>I also suspect that LDP could recognize multiple Context ID addresses as
>belonging to the same PE router and as a consequence merge LSPs that use
>this IP address as a /32 IPv4 prefix FEC. If this happens, it would break
>the entire mechanism in the scenarios when the same destination PE
>participates in multiple contexts - quite a reasonable situation in the
>case of MS-PWs with S-PE protection IMO, but also possible in the case of
>a T-PE that use different Protectors for different PWs.
>
>Subject: RE: [PWE3] WG Last Call for
>draft-ietf-pwe3-endpoint-fast-protection-01
>
>Hi Stewart,
>
>Yes, it works with both LDP and RSVP-TE (or other signaling protocols).
>This is based on the PWE3 architecture where PWE3 layer and PSN layer are
>decoupled. If PSN is MPLS, all we need is an MPLS tunnel to the primary
>(or protected) PE, and an MPLS bypass tunnel from the PLR to the
>protector. The two tunnels may be LDP or RSVP-TE (or another protocol).
>The context ID defined in this draft can be used to facilitate the setup
>of these tunnels.
>
>
>Thanks,
>
>/Yimin
>
>
>
>Subject: Re: [PWE3] WG Last Call for
>draft-ietf-pwe3-endpoint-fast-protection-01
>
>Yimin
>
>_______________________________________________
>pwe3 mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/pwe3

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 5 Aug 2014 05:13:03 +0000

Yimin hi,
I apologize for missing your response when it has been sent.

I will try to address some of your comments without yet one more level of in-lining:-).

[Yimin] This draft assumes active-active PWs and bidirectional ACs.
[Sasha] It would be nice if this were explicitly stated in the draft.

[Yimin] I think you are talking about how to run Layer 3 between CEs with an overlay model, which I don't think is in scope of PWE3 architecture
[Sasha] To the best of my understanding PWE3 is about an emulation of a native service. It does not have to be perfect, but limitations are supposed to be explicitly presented in Applicability Statements.  On the practical level, if a PW mechanism is not good enough to carry L3 traffic between a pair of CEs, what is it good for?
And, last but not least, if CE1 and CE2 in my example were L2 switches running some flavor of STP, the situation would be no less disruptive.

[Yimin] In case of egress AC failure, CE2 will detect the failure and start to send traffic over PW2 (i.e. the backup PW). 
[Sasha] This is an assumption about CE2 which is definitely out of scope of the PWE3 architecture. Actually what you say is that CE2 should treat its failure to receive traffic over an AC as a bi-directional AC failure. 
[Yimin] CE1 should also start to send traffic over the backup PW2. This may be triggered by many events, e.g. [1] PE1's notification to CE1 based on the PW status you mentioned above (but it is not mandatory); 
[Sasha] This is possible of course, but, as you've said, not mandatory
[Yimin] or [2] CE1 receiving traffic from PE3. 
[Sasha] But you've said that ACs are active-active. So why receiving something from PE3 would be of any interest?
And of course, there is a case when only CE2 is dual-homed but CE1  is not... Is the draft relevant in this scenario?
[Yimin] In any case, it may take some time for CE1 to do so. Before this happens on CE1, the traffic is still going over PW1 (primary PW) which is being locally repaired by PE2.
[Sasha] I'd say that the draft relies on some "dual-homing protocol" between a dual-homed CE and its PEs to provide for a bi-directional switch, and that such a switch could take some time. I am not sure this counts for "fast protection".

Regards,
Sasha

________________________________________
From: Yimin Shen <[email-masked]>
Sent: Thursday, July 31, 2014 2:22 AM
To: Alexander Vainshtein
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Hi Sasha,

Thanks again for your comments. Please see inline...

Thanks,

/Yimin

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Wednesday, July 30, 2014 4:08 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,
Lots of thanks for a prompt response.

Unfortunately, from my point of view, the issues I've raised still remain unresolved. Please see additional questions inline below.

Regards,
       Sasha
Email: [email-masked]
Mobile: 054-9266302

> From: Yimin Shen [mailto:[email-masked]]
> Sent: Tuesday, July 29, 2014 7:56 PM
> To: Alexander Vainshtein; [email-masked]; Yakov Rekhter;
> [email-masked]
> Hi Sasha,
> Thanks very much for the review and comments. Please see my responses
> inline. Hope they are helpful.
>
>
> Thanks,
>
> /Yimin
>
>
>
> From: pwe3 [mailto:[email-masked]] On Behalf Of Alexander
> Vainshtein
> Sent: Tuesday, July 29, 2014 10:08 AM
> To: [email-masked]; Yakov Rekhter; [email-masked]
> Stewart, Yakov and all,
> I have started reading the draft, and it seems that it breaks quite a
> few common architectural assumptions.
>
> [yshen] This draft is fully based on the PWE3 architecture and the
> MS-PW architecture.
>
> The following text from Section 3.1 "Single-Segment PW" looks highly
> problematic to me:
> <quote>
>   At any given time, each CE sends traffic via only one AC and receives
>    traffic via only one AC.  The two ACs MAY or MAY NOT be the same.
>    The AC used to send traffic is determined by the CE, and MAY rely on
>    an end-to-end OAM mechanism between the CEs.  The AC used for the CE
>    to receive traffic is determined by the state of the network and the
>    protection mechanism in use, as described later in this document.
> <end quote>.
>
> To the best of my order for the pair of interconnected CEs to be able
> to operate in this way each of them should treat the pair of ACs shown
> in Figure
> 1 as a single logical entity.
> E.g., if we are speaking about Ethernet PWs, each CE could treat
> Ethernet links supporting these two ACs as a single Link Aggregation
> Group. However, the draft does not mention any such thing (or at least I did not find it).
>
> Do I miss something important here?
>
> [yshen] It is a logical view that the CE is dual-home to 2 PEs over 2
> distinct ACs. AC is a concept well defined in RFC 3985. In terms of
> the actual PE-CE connection in the case of Ethernet, the CE could have
> 2 distinct Ethernet links or 2 VLANs over a single LAG.
>
[[Sasha]] If I read your response correctly, each dual-homed CE treats each if its two ACs as an individual entity.
I see this is as highly problematic. E.g., consider the case when CE1 and CE2 are two CE routers running some link-state routing protocol between them.
-  In the normal situation, CE1 probably would see its two ACs as two parallel bi-directional links.
- With PW redundancy, one of these links (one that corresponds to the current Active PW) would be Up and the other one - Down
- With your proposal, the typical link-state routing protocol between CE1 and CE2 would not be able to form any adjacencies across any of these links, because this process requires bi-directional link connectivity; so regardless of whatever happens in the PW domain CE1 and CE2 would consider each other as unreachable across this domain.
Did I miss something? (I intentionally leave aside protocols like OLSR that can handle unidirectional links).

[yshen] I'm lost in the last 2 bullets. This draft assumes active-active PWs and bidirectional ACs. I think you are talking about how to run Layer 3 between CEs with an overlay model, which I don't think is in scope of PWE3 architecture.

> I also wonder whether, in the single segment PW scenario, the same
> level of protection could not be achieved with ICCP  (RFC 7275)
> running between each pair of PEs and, possibly with 4 (instead of 2)
> PWs.  For the reference, ICCP (or RFC 7275) is not mentioned in the
> draft, even if a possible alternative solution to some of the problems
> it solves. (
>
> [yshen] Thanks for pointing out ICCP. We will mention it in the next revision.
> In this draft we broadly refer to such kind of mechanism as "control
> plane repair". The purpose of this draft is to locally repair traffic
> at the failure point, in order to reduce traffic loss before ICCP or
> other control plane repair mechanism kicks in.

[[Sasha]] I am not sure ICCP in this case would mean slow protection. Its function here would be just to synchronize the state of PWs between PE1 and PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection State Coordination (PSC) protocol in RFC 6378 looks like an analog (with some stretch of course), and it is supposed to provide pretty fast protection switching.

[yshen] The draft doesn't comment specifically on ICCP performance. Generally speaking, local repair mechanism tends to have the least overhead when restoring traffic. This is because both failure detection and traffic restoration are done right in the dataplane. It doesn't rely on dataplane to control plane notification (for failure event) or control plane protocol. For this exact reason there have been various IP/MPLS fast-reroute mechanisms defined by IETF. Of course, in a given scenario where the delay in global repair or control plane repair is tolerable for users, they don't have to use local repair. Local repair is for the case where it is desirable to reduce traffic loss even further. Note that local repair is a temporary repair, and it doesn't prevent global repair or control plane repair to perform a permanent repair. I believe the draft has specifically mentioned this.

>
> Stewart did not specify in his email which MPLS architecture
> principles he thinks are broken by this draft.
>
> I think, however, that some PWE3 architectural principles are
> definitely are broken, and this includes:
> - Bi-directional nature of the PW - it seems that the draft allows
> each a PW be Up in one direction and Down in the other one
>
> [yshen] No, the draft does not allow that. It only says that we can
> send traffic of one direction over one PW, while sending traffic of
> the opposite direction over another PW (i.e. backup PW). Both PWs should be up bidirectionally.
>
>
[[Sasha]] Sorry but I do not understand that. The draft discusses something like  the "egress AC failure". If, say. PE1 detects that, I expect it would send a PW Status with "PW not Forwarding" and "Local AC Transmit Fault" bits set to PE2. Would this not mean that the PW between PE1 and PE2 is Down?

[yshen] In case of egress AC failure, CE2 will detect the failure and start to send traffic over PW2 (i.e. the backup PW). CE1 should also start to send traffic over the backup PW2. This may be triggered by many events, e.g. [1] PE1's notification to CE1 based on the PW status you mentioned above (but it is not mandatory); or [2] CE1 receiving traffic from PE3. In any case, it may take some time for CE1 to do so. Before this happens on CE1, the traffic is still going over PW1 (primary PW) which is being locally repaired by PE2.

> - Lack of dependencies between the PSN tunnels and the PWs they carry.
>       * Consider the network diagram shown in Figure 5 in Section 4.1
>       * The text suggests that the tunnel between PE1 and PE2 must
> recognize PE4 as the "protector" node for the PW it carries
>       * What is supposed to happen if the tunnel between PE1 and PE2
> carries, say, 10 PWs, each with its own backup PE PE4-1, ... PE4-10)?
>
> [yshen] In your example, the 10 PWs are protected by 10 different
> protectors. In this case, you would need 10 different tunnels,
> destined to 10 different context IDs (one context ID per <PE2,
> PE4-x>). If multiple PWs are protected by the same protector, they can be carried by the same tunnel.
>
[[Sasha]] This looks like a non-scalable solution to me.

[yshen] I'm not sure there is any scalability issue here. We basically need one tunnel per context ID, i.e. per <primary PE, protector>. In reality, the number of <primary PE, protector> should not be high. The draft also specifies the centralized model for reducing the number of context IDs.

>
> - Lack of dependencies between the mechanism used for the tunnel setup
> (LDP, RSVP-TE, manual configuration etc.) and the PWs this tunnel carries.
>       * Is the protection model described in Section 4.1 compatible with
> Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying the
> PWs using MPLS-in-GRE?
>
> [yshen] Yes, it is compatible. The draft is generic on transport
> layer, and it allows tunnels to be LDP, RSVP-TE, GRE, etc. These
> protocols already provide sufficient machineries for setting up transport tunnels and bypass tunnels.
> Therefore, the draft doesn't require new extension to these LSP
> signaling protocols.
>
[[Sasha]] Sorry but this does not match your response to my previous question/issue where you have explained that you effectively need a tunnel per PW between PE1 and PE2. RSVP-TE can provide that; but how is LDP supposed to do that?

[yshen] Again, a tunnel can carry multiple PWs, if these PWs are protected by the same protector. You could use 10 different protectors to protect 10 PWs, but in reality when SP design their network, they should ask why they have to do so. Regarding LDP, it is one FEC per context ID, and nothing special.

> Last but not least, I would like to understand better how the draft is
> protecting MS-PWs against S-PE failures.
> Specifically, I'd like to understand whether it is applicable to the
> case when each pair of T-PEs resides in its own AS, with two pairs of
> directly connected ASBRs between these two ASs acting as S-PEs.  There
> would be no need to set up any tunnel LSPs between each pair of S-PEs in this scenario.
> I would appreciate an analysis of this scenario regarding ability to
> protect about the ASBR/S-PE failure. IMO it is a relevant scenario
> because MS-PWs are one of the methods to provide inter-AS services.
>
> [yshen] The MS-PW topology in this draft is based on the Figure-4 of
> RFC 5659, which is slightly different than the scenario you mentioned above.
>
[[Sasha]] Figure 4 in RFC 5659 does not say anything about inter-AS.
>
> However, the mechanism in this draft does apply to this scenario as
> well. It should work equally if S-PEs are one-hop or multi-hops away
> from each other. Please see the picture below. In this case, SPE1 ~ SPE4 are ASBRs.
> When SPE4 detects the failure of SPE3, it will do a local repair and
> redirect PW traffic to SPE4 which is the protector. The bypass between
> SPE1 and SPE4 may be a one-hop tunnel or a multi-hop tunnel, depending
> on the actual topology between the 2 AS'.
 >
[[Sasha]] This approach can work if SPE1 is directly connected to SPE4 because in this case there would be no need for a tunnel LSP between them.
I do not see a simple way to set up an LSP between SPE1 (in AS-1) and SPE4(in AS-2) if such connectivity is not present; could you possibly expand on how this is supposed to happen?

[yshen] Sasha, if you don't have a bypass tunnel (due to specific topology or policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to work :) However it doesn't mean the mechanism is broken. In your case, I don't see any technique issue why you cannot set up a inter-AS bypass from SPE1-SPE2-SPE4.

>
 >  In any case, SPE4 will send PW traffic over a
> tunnel to TPE4.
>
>                   |<--------------- PW1 ----------->|
>
>              - TPE1 -------- SPE1 - SPE3 --------- TPE2 -
>             /                PLR \                       \
>            /                      \                       \
>         CE1                  bypass\                        CE2
>            \                        \                     /
>             \                        \                   /
>              - TPE3 -------- SPE2 - SPE4 --------- TPE4 -
>                                   protector
>
>                   |<--------------- PW2 ----------->|
>
> My 2c,
>        Sasha
> Email: [email-masked]
> Mobile: 054-9266302
>
> > From: pwe3 [mailto:[email-masked]] On Behalf Of Stewart
> > Bryant
> > Sent: Tuesday, July 29, 2014 3:35 PM
> > To: Yakov Rekhter; [email-masked]
> > On 25/07/2014 16:24, Yakov Rekhter wrote:
> > > Hi,
> > >
> > >> This email begins a second two week working group last call for
> > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-01 .
> > >>
> > >> Please review the draft and post any comments to the PWE3 list.
> > >>
> > >> If you have read the draft, even if you don't have any comments,
> > >> but would like to see the draft progressed,  please can you
> > >> indicate this to the
> > list.
> > >>
> > >> This working group last call will end on Friday August 8th, 2014.
> > > I would like to see the draft progressed.
> > >
> > > Yakov.
> > >
> > Yakov
> > Doesn't this draft implicitly make a number of extensions to the
> > MPLS architecture?
> >
> > Stewart
> >

---

From: Yimin Shen <[email-masked]>
Date: Tue, 5 Aug 2014 18:55:23 +0000

Hi Sasha,

All ACs and PWs in this draft are active at individual level. One set of AC-PW-AC is used as backup to protect the other set of AC-PW-AC (i.e. primary).

I think we both agree that CE behavior is out of the scope of PWE3. If you want to run L3 between CEs, you could bundle the two ACs on CE as a single logical L3 interface. I'm sure there are other approaches depending on the type of CE. But again, this is out scope of PWE3.

The "fast protection" refers to the ability to locally detect failure, restore traffic, and reduce loss. CEs and PEs may go through any negotiation or coordination to move traffic from the repaired PW to anther PW, at any pace. Before that happens, traffic has already been repaired by this mechanism.

Thanks,

/Yimin

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]] 
Sent: Tuesday, August 05, 2014 1:13 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

[Yimin] I think you are talking about how to run Layer 3 between CEs with an overlay model, which I don't think is in scope of PWE3 architecture [Sasha] To the best of my understanding PWE3 is about an emulation of a native service. It does not have to be perfect, but limitations are supposed to be explicitly presented in Applicability Statements.  On the practical level, if a PW mechanism is not good enough to carry L3 traffic between a pair of CEs, what is it good for?
And, last but not least, if CE1 and CE2 in my example were L2 switches running some flavor of STP, the situation would be no less disruptive.

[Yimin] CE1 should also start to send traffic over the backup PW2. This may be triggered by many events, e.g. [1] PE1's notification to CE1 based on the PW status you mentioned above (but it is not mandatory); [Sasha] This is possible of course, but, as you've said, not mandatory [Yimin] or [2] CE1 receiving traffic from PE3. 

Regards,
Sasha

Hi Sasha,

Thanks,

/Yimin

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Wednesday, July 30, 2014 4:08 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,
       Sasha

>       * Is the protection model described in Section 4.1 compatible 
> with Tunnel LSPs being set up by LDP? With, say, GRE tunnels carrying 
> the PWs using MPLS-in-GRE?
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 6 Aug 2014 05:55:09 +0000

Yimin hi!

Please see a couple of comments inline below (marked [[Sasha]].

Regards,
Sasha
Sent: Tuesday, August 5, 2014 9:55 PM

Hi Sasha,

I think we both agree that CE behavior is out of the scope of PWE3. 

[[Sasha]] Yes - provided that PWE3 matches reasonable CE expectations. If it is not so, it becomes very much in scope IMO.

If you want to run L3 between CEs, you could bundle the two ACs on CE as a single logical L3 interface. I'm sure there are other approaches depending on the type of CE. But again, this is out scope of PWE3.

[[Sasha]] Well, not really IMO. E.g., Section 7.2 of RFC 7225 specifically defines a PWE3 solution for PW redundancy with multi-Chassis LAG. This solution addresses the case when a dual-homed CE perceives a pair of Ethernet links between itself and a pair of redundant PEs as a single Link Aggregation Group, and effectively uses LACP as the dual-homing protocol. I'd say that if you decide that CE should use some standard (or widely deployed) mechanism between itself and the PEs in order for PWE3 to provide a meaningful service, PWE3 has both to specify its expectations regarding the CE behavior and the methods it uses to accommodate this behavior. 
An example that looks highly relevant to me with regard to your draft is like following:

The draft explicitly differentiates between "ingress" and "egress" AC failures and states that "Today, fast protection against ingress AC failure and ingress (T-)PE  failure is achievable by using a multi-homed CE and redundant PWs.". Suppose that CE and PE employ some native mechanism (out of scope of PWE3, of course) that would revert any unidirectional AC failure to a bi-directional one in a very fast manner. (One example that comes to mind is the LF/RF exchange mechanism oft 10-Gigabit Ethernet.) Would such a mechanism eliminate the need  to protect against egress AC failures which seems to be one of the major drives for the draft?

Thanks,

/Yimin

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Tuesday, August 05, 2014 1:13 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,
Sasha

Hi Sasha,

Thanks,

/Yimin

-----Original Message-----
From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Wednesday, July 30, 2014 4:08 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,
       Sasha

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 6 Aug 2014 09:11:18 +0000

Yimin and all,

I would like to summarize the issues that I have with this draft. Digging for them in a very long email thread with multiple levels of in-lining is not very effective IMO.

1.       Protection against unidirectional AC failures looks problematic to me:

a.       The draft explicitly differentiates between "ingress" and "egress" AC failures

b.      The outcome can be using a different AC for each direction of CE-to-CE traffic with potentially destructive effect that cannot be mitigated at the PWE3 level

c.       IMHO and FWIW a viable alternative could be to employ native AC mechanisms converting unidirectional AC failures into bi-directional ones. This would eliminate the differentiation between ingress and egress AC failures and facilitate using the PW redundancy mechanism for bi-directional protection

2.       Dependency upon specific mechanisms for setting up PSN tunnels seems to break some common PWE3 architectural principles:

a.       I am pretty well sure that the mechanisms specified in the draft cannot be used with the PSN tunnel LSPs set up by LDP in DU mode

b.      If the authors, refer to LDP in the DoD mode augmented with some CR-LDP-like mechanisms (e.g., for exclusion of specific links when setting up the bypass LSP between the PLR and the Protector), this should, at the very least, be discussed with and approved by the MPLS WG

3.       Scalability of the proposed solution should be, at least, explicitly presented. In particular, the draft assumes that a dedicated LSP would be used between PE1  PE2 for each combination of PE2 with a specific Protector LSR.

4.       The Protector LSR could be either a kind of S-PE (if different from the secondary T-PE) or the secondary T-PE itself. In both cases it should be capable of handling labels from specific context spaces as if they were PW labels. This requires (at the very least) an update of RFC 4447 and should be explicitly presented and discussed.

Hopefully this summary will be useful.

Regards,

       Sasha

Mobile: 054-9266302

> From: Alexander Vainshtein

> Sent: Wednesday, August 06, 2014 8:55 AM

> Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> protection-01

> From: Yimin Shen <[email-masked]<mailto:[email-masked]>>

> Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> protection-01

> All ACs and PWs in this draft are active at individual level. One set of AC-PW-

> AC is used as backup to protect the other set of AC-PW-AC (i.e. primary).

> [[Sasha]] Yes - provided that PWE3 matches reasonable CE expectations. If it

> is not so, it becomes very much in scope IMO.

> If you want to run L3 between CEs, you could bundle the two ACs on CE as a

> single logical L3 interface. I'm sure there are other approaches depending on

> the type of CE. But again, this is out scope of PWE3.

> [[Sasha]] Well, not really IMO. E.g., Section 7.2 of RFC 7225 specifically

> defines a PWE3 solution for PW redundancy with multi-Chassis LAG. This

> solution addresses the case when a dual-homed CE perceives a pair of

> Ethernet links between itself and a pair of redundant PEs as a single Link

> Aggregation Group, and effectively uses LACP as the dual-homing protocol.

> I'd say that if you decide that CE should use some standard (or widely

> deployed) mechanism between itself and the PEs in order for PWE3 to

> provide a meaningful service, PWE3 has both to specify its expectations

> regarding the CE behavior and the methods it uses to accommodate this

> behavior.

> An example that looks highly relevant to me with regard to your draft is like

> following:

> The draft explicitly differentiates between "ingress" and "egress" AC failures

> and states that "Today, fast protection against ingress AC failure and ingress

> (T-)PE  failure is achievable by using a multi-homed CE and redundant PWs.".

> Suppose that CE and PE employ some native mechanism (out of scope of

> PWE3, of course) that would revert any unidirectional AC failure to a bi-

> directional one in a very fast manner. (One example that comes to mind is

> the LF/RF exchange mechanism oft 10-Gigabit Ethernet.) Would such a

> mechanism eliminate the need  to protect against egress AC failures which

> seems to be one of the major drives for the draft?

> The "fast protection" refers to the ability to locally detect failure, restore

> traffic, and reduce loss. CEs and PEs may go through any negotiation or

> coordination to move traffic from the repaired PW to anther PW, at any pace.

> Before that happens, traffic has already been repaired by this mechanism.

> Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> protection-01

> I will try to address some of your comments without yet one more level of in-

> lining:-).

> [Yimin] I think you are talking about how to run Layer 3 between CEs with an

> overlay model, which I don't think is in scope of PWE3 architecture [Sasha] To

> the best of my understanding PWE3 is about an emulation of a native service.

> It does not have to be perfect, but limitations are supposed to be explicitly

> presented in Applicability Statements.  On the practical level, if a PW

> mechanism is not good enough to carry L3 traffic between a pair of CEs, what

> is it good for?

> And, last but not least, if CE1 and CE2 in my example were L2 switches

> running some flavor of STP, the situation would be no less disruptive.

> [Yimin] In case of egress AC failure, CE2 will detect the failure and start to

> send traffic over PW2 (i.e. the backup PW).

> [Sasha] This is an assumption about CE2 which is definitely out of scope of

> the PWE3 architecture. Actually what you say is that CE2 should treat its

> failure to receive traffic over an AC as a bi-directional AC failure.

> [Yimin] CE1 should also start to send traffic over the backup PW2. This may be

> triggered by many events, e.g. [1] PE1's notification to CE1 based on the PW

> status you mentioned above (but it is not mandatory); [Sasha] This is possible

> of course, but, as you've said, not mandatory [Yimin] or [2] CE1 receiving

> traffic from PE3.

> [Sasha] But you've said that ACs are active-active. So why receiving

> something from PE3 would be of any interest?

> And of course, there is a case when only CE2 is dual-homed but CE1  is not...

> Is the draft relevant in this scenario?

> [Yimin] In any case, it may take some time for CE1 to do so. Before this

> happens on CE1, the traffic is still going over PW1 (primary PW) which is being

> locally repaired by PE2.

> [Sasha] I'd say that the draft relies on some "dual-homing protocol" between

> a dual-homed CE and its PEs to provide for a bi-directional switch, and that

> such a switch could take some time. I am not sure this counts for "fast

> protection".

> From: Yimin Shen <[email-masked]<mailto:[email-masked]>>

> Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> protection-01

> Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> protection-01

> Unfortunately, from my point of view, the issues I've raised still remain

> unresolved. Please see additional questions inline below.

> Email: [email-masked]<mailto:[email-masked]>

> > To: Alexander Vainshtein; [email-masked]<mailto:[email-masked]>; Yakov Rekhter;

> > [email-masked]<mailto:[email-masked]>

> > Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> > protection-01

> > To: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> > protection-01

> >    an end-to-end OAM mechanism between the CEs.  The AC used for the

> CE

> > Group. However, the draft does not mention any such thing (or at least I

> did not find it).

> [[Sasha]] If I read your response correctly, each dual-homed CE treats each if

> its two ACs as an individual entity.

> I see this is as highly problematic. E.g., consider the case when CE1 and CE2

> are two CE routers running some link-state routing protocol between them.

> -  In the normal situation, CE1 probably would see its two ACs as two parallel

> bi-directional links.

> - With PW redundancy, one of these links (one that corresponds to the

> current Active PW) would be Up and the other one - Down

> - With your proposal, the typical link-state routing protocol between CE1 and

> CE2 would not be able to form any adjacencies across any of these links,

> because this process requires bi-directional link connectivity; so regardless of

> whatever happens in the PW domain CE1 and CE2 would consider each other

> as unreachable across this domain.

> Did I miss something? (I intentionally leave aside protocols like OLSR that can

> handle unidirectional links).

> [yshen] I'm lost in the last 2 bullets. This draft assumes active-active PWs and

> bidirectional ACs. I think you are talking about how to run Layer 3 between

> CEs with an overlay model, which I don't think is in scope of PWE3

> architecture.

> > [yshen] Thanks for pointing out ICCP. We will mention it in the next

> revision.

> [[Sasha]] I am not sure ICCP in this case would mean slow protection. Its

> function here would be just to synchronize the state of PWs between PE1

> and PE3/PE2 and PE4, and this can be reasonably fast IMO. The Protection

> State Coordination (PSC) protocol in RFC 6378 looks like an analog (with some

> stretch of course), and it is supposed to provide pretty fast protection

> switching.

> [yshen] The draft doesn't comment specifically on ICCP performance.

> Generally speaking, local repair mechanism tends to have the least overhead

> when restoring traffic. This is because both failure detection and traffic

> restoration are done right in the dataplane. It doesn't rely on dataplane to

> control plane notification (for failure event) or control plane protocol. For this

> exact reason there have been various IP/MPLS fast-reroute mechanisms

> defined by IETF. Of course, in a given scenario where the delay in global

> repair or control plane repair is tolerable for users, they don't have to use

> local repair. Local repair is for the case where it is desirable to reduce traffic

> loss even further. Note that local repair is a temporary repair, and it doesn't

> prevent global repair or control plane repair to perform a permanent repair. I

> believe the draft has specifically mentioned this.

> > the opposite direction over another PW (i.e. backup PW). Both PWs should

> be up bidirectionally.

> [[Sasha]] Sorry but I do not understand that. The draft discusses something

> like  the "egress AC failure". If, say. PE1 detects that, I expect it would send a

> PW Status with "PW not Forwarding" and "Local AC Transmit Fault" bits set to

> PE2. Would this not mean that the PW between PE1 and PE2 is Down?

> [yshen] In case of egress AC failure, CE2 will detect the failure and start to

> send traffic over PW2 (i.e. the backup PW). CE1 should also start to send

> traffic over the backup PW2. This may be triggered by many events, e.g. [1]

> PE1's notification to CE1 based on the PW status you mentioned above (but it

> is not mandatory); or [2] CE1 receiving traffic from PE3. In any case, it may

> take some time for CE1 to do so. Before this happens on CE1, the traffic is still

> going over PW1 (primary PW) which is being locally repaired by PE2.

> > PE4-x>). If multiple PWs are protected by the same protector, they can be

> carried by the same tunnel.

> [yshen] I'm not sure there is any scalability issue here. We basically need one

> tunnel per context ID, i.e. per <primary PE, protector>. In reality, the number

> of <primary PE, protector> should not be high. The draft also specifies the

> centralized model for reducing the number of context IDs.

> > protocols already provide sufficient machineries for setting up transport

> tunnels and bypass tunnels.

> [[Sasha]] Sorry but this does not match your response to my previous

> question/issue where you have explained that you effectively need a tunnel

> per PW between PE1 and PE2. RSVP-TE can provide that; but how is LDP

> supposed to do that?

> [yshen] Again, a tunnel can carry multiple PWs, if these PWs are protected by

> the same protector. You could use 10 different protectors to protect 10 PWs,

> but in reality when SP design their network, they should ask why they have

> to do so. Regarding LDP, it is one FEC per context ID, and nothing special.

> > would be no need to set up any tunnel LSPs between each pair of S-PEs in

> this scenario.

> > RFC 5659, which is slightly different than the scenario you mentioned

> above.

> > from each other. Please see the picture below. In this case, SPE1 ~ SPE4 are

> ASBRs.

> [[Sasha]] This approach can work if SPE1 is directly connected to SPE4

> because in this case there would be no need for a tunnel LSP between them.

> I do not see a simple way to set up an LSP between SPE1 (in AS-1) and

> SPE4(in AS-2) if such connectivity is not present; could you possibly expand

> on how this is supposed to happen?

> [yshen] Sasha, if you don't have a bypass tunnel (due to specific topology or

> policy or whatever), you won't get any IP/MPLS fast-reroute mechanism to

> work :) However it doesn't mean the mechanism is broken. In your case, I

> don't see any technique issue why you cannot set up a inter-AS bypass from

> SPE1-SPE2-SPE4.

> > Email: [email-masked]<mailto:[email-masked]>

> > > To: Yakov Rekhter; [email-masked]<mailto:[email-masked]>

> > > Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-

> > > protection-01

> > > >> http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-<http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-01>

> 01<http://tools.ietf.org/html/draft-ietf-pwe3-endpoint-fast-protection-01> .

> > > > [email-masked]<mailto:[email-masked]>

> > > [email-masked]<mailto:[email-masked]>

> > [email-masked]<mailto:[email-masked]>

---

From: Yimin Shen <[email-masked]>
Date: Thu, 7 Aug 2014 02:33:37 +0000

Hi Sasha,

Thanks again for the comments. Please see the following 4 items in response to the 4 comments in your email.

1. Today when an egress PE detects an egress AC failure, it will simply have to drop packets, until the remote PE stops sending traffic over the PW. This draft is intended to improve this behavior. By no means the local repair on egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note that bi-directional AC failure handling must involve the CE, but the CE may detect the failure at a slower pace, or it may not be able to detect it in a timely manner, depending on the actual situation. E.g. there may be other L2 devices between the PE and CE; the AC may itself be a logical wire across another network; the type and location of the failure may make it easier for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-directionally), the egress traffic has been locally repaired by the egress PE.

2. As mentioned before, this draft doesn't have a dependency on a specific PSN tunnel technology.

3. The draft already talks about scalability when describing the centralized protector model vs the co-located protector model. We can improve the text to explicitly mention that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary PE basis can potentially increase the number of LSPs in a network. This is a tradeoff. As mentioned before, this does not necessarily cause a scaling issue.

4. We will leave this to PWE3 WG to decide whether this is really necessary.

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Wednesday, August 06, 2014 5:11 AM
To: Yimin Shen
Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 7 Aug 2014 14:36:39 +0000

Yimin hi,
Please see some comments inline below.

Regards,
       Sasha

From: Yimin Shen [mailto:[email-masked]]
Sent: Thursday, August 07, 2014 5:34 AM
To: Alexander Vainshtein
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Hi Sasha,

1. Today when an egress PE detects an egress AC failure

[[Sasha]] Could you please explain how the egress PE detects an egress AC failure? I have always thought that egress AC failure has to be detected first by corresponding CE which would then notify the egress PE using some suitable "Remote Failure"/BDI/RDI mechanism. Specific examples would really help.

it will simply have to drop packets, until the remote PE stops sending traffic over the PW. This draft is intended to improve this behavior. By no means the local repair on egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note that bi-directional AC failure handling must involve the CE, but the CE may detect the failure at a slower pace, or it may not be able to detect it in a timely manner, depending on the actual situation.
[[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its "Receive Failure", the PE would not never know about the "Transmit Failure.

E.g. there may be other L2 devices between the PE and CE; the AC may itself be a logical wire across another network; the type and location of the failure may make it easier for the PE to detect it; and so on. In any case, before the AC failure is handled more gracefully (or bi-directionally), the egress traffic has been locally repaired by the egress PE.

2. As mentioned before, this draft doesn't have a dependency on a specific PSN tunnel technology.
[[Sasha]] I have provided my comments on this in my response to Eric. From my point of view some LDP-specific issues still require clarification, but my bottom line is that operating the draft over an MPLS network that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possible but would always be tricky. This does not match my understanding of "no dependency on a specific PSN tunnel technology", but this may be a matter of taste.

3. The draft already talks about scalability when describing the centralized protector model vs the co-located protector model. We can improve the text to explicitly mention that the requirement for setting up LSPs on a per <primary PE, protector> basis rather than per primary PE basis can potentially increase the number of LSPs in a network. This is a tradeoff. As mentioned before, this does not necessarily cause a scaling issue.
[[Sasha]] Yes, I have found the corresponding text in the draft.  From my point of view this issue is resolved.

4. We will leave this to PWE3 WG to decide whether this is really necessary.[[Sasha]] I tend to agree with Eric that usage of context-specific labels in the draft does not mean update to 4447. From my point of view this issue is resolved as well.

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Yimin Shen <[email-masked]>
Date: Fri, 8 Aug 2014 14:34:19 +0000

Hi Sasha,

Thanks again for the comments and questions!

Regarding AC failure detection on PE, in the typical cases, if there is a fiber cut, or a hardware failure affecting the port on the PE, or a hardware failure affecting the port on the CE (assume the CE is directed attached to the PE), the PE should be able to directly and independently detect the failure at physical or link layer.

Regarding LDP, I agree with you that LDP by its nature is not as straight-forward as RSVP-TE in this scenario. Since a context ID are advertised by both primary PE and protector, we should avoid creating an ECMP scenario. That's why section 4.2.2 provides a suggestion to advertise the context ID as a proxy node (i.e. a virtual node attached to the two routers), and make its reachability via the primary PE more favorable than that via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As you can see, there may be many different approaches, and there may new approaches coming in the future. In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this mechanism is achievable with LDP, and LDP as a tunnel signaling protocol really doesn't need any new extension.

Hope this clarifies it.

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Thursday, August 07, 2014 10:37 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Alexander Vainshtein <[email-masked]>
Date: Sun, 10 Aug 2014 05:15:28 +0000

Yimin hi,

I must admit that I have missed the proposal in Section 4.2.2 to treat "context ID" IP address as a proxy node that is adjacent to both the Primary PE and the Protector NEs. (It also has not been mentioned in Eriç's clarification email). I think that this approach would require certain manipulation of both the IGP and LDP, but this requires more thought on my side. One of the issues that come to mind, is the need for LDP in both these nodes to allocate a locally terminated label for the corresponding FEC while not advertising context IP address as one of the IP addresses it owns.

I think that your statement about "avoiding ECMP  scenario" is problematic, because, from my point of view,  ECMP is one of the important characteristics of LDP-based MPLS PSN.  And in any case the draft does not say anything about that (or, at least neither ECMP nor "equal cost multipath" is mentioned in the text).  If you see ECMP avoidance as a pre-condition for using the mechanism defined in draft with LDP LSPs, this should be quite explicit and detailed IMO.

I still fail to understand  what constitutes an egress AC failure in the draft and how it is different from the ingress AC failure. Unfortunately the examples you've provided in your response do not help much: e.g., a fiber cut close to  would be most probably recognized by the PE as a bi-directional AC failure, while a fiber cut that is close to CE but remote from PE would most probably not detected by the PE. (Of course if PE and CE are terminating the same fiber span, the fiber cut would be recognized by both).

My guess (FWIW) that the draft actually discusses protection of traffic in the PE --> CE direction triggered by a failure detected by the PE while leaving protection of traffic in the CE --> PE direction some other mechanism that would be triggered by CE detecting the failure. This mechanism presumably would be different from the PW redundancy mechanism because it would require that an active-active scheme for Primary and Backup  PWs.

If this guess is correct (or close to reality), I would expect some explicit text explaining it. In particular, this text should explain that "redundant PWs" mentioned in the draft are not related to "PW redundancy". The current text makes a mix of these two notions, and this definitely does not help the reader (or at least this reader :-).

Hopefully these comments would be useful.

Regards,

Sasha

________________________________
From: Yimin Shen <[email-masked]>
Sent: Friday, August 8, 2014 5:34 PM
To: Alexander Vainshtein
Cc: [email-masked]; Yakov Rekhter; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Hi Sasha,

Regarding LDP, I agree with you that LDP by its nature is not as straight-forward as RSVP-TE in this scenario. Since a context ID are advertised by both primary PE and protector, we should avoid creating an ECMP scenario. That’s why section 4.2.2 provides a suggestion to advertise the context ID as a proxy node (i.e. a virtual node attached to the two routers), and make its reachability via the primary PE more favorable than that via the protector. MRT and other mechanisms could be used to help LDP improve protection coverage. Also, nothing would prevent LDP from using RSVP bypass tunnels. As you can see, there may be many different approaches, and there may new approaches coming in the future. In this PWE3 document, we just want to stay at PW level and be generic with PSN level. In any case, this mechanism is achievable with LDP, and LDP as a tunnel signaling protocol really doesn’t need any new extension.

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

[[Sasha]] Could you please explain how the egress PE detects an egress AC failure? I have always thought that egress AC failure has to be detected first by corresponding CE which would then notify the egress PE using some suitable “Remote Failure”/BDI/RDI mechanism. Specific examples would really help.

it will simply have to drop packets, until the remote PE stops sending traffic over the PW. This draft is intended to improve this behavior. By no means the local repair on egress PE will prevent the AC failure from being converted to and handled as a bi-directional AC failure. Note that bi-directional AC failure handling must involve the CE, but the CE may detect the failure at a slower pace, or it may not be able to detect it in a timely manner, depending on the actual situation.
[[Sasha]] Please see above. IMHO and FWIW if the CE cannot detect its “Receive Failure”, the PE would not never know about the “Transmit Failure.

2. As mentioned before, this draft doesn’t have a dependency on a specific PSN tunnel technology.
[[Sasha]] I have provided my comments on this in my response to Eric. From my point of view some LDP-specific issues still require clarification, but my bottom line is that operating the draft over an MPLS network that uses LDP in DU mode for setting up tunnel LSPs may sometimes be possible but would always be tricky. This does not match my understanding of “no dependency on a specific PSN tunnel technology”, but this may be a matter of taste.

Thanks,

/Yimin

Yimin and all,

a.       The draft explicitly differentiates between “ingress” and “egress” AC failures

Regards,

       Sasha

Mobile: 054-9266302

---

From: "Stewart Bryant (stbryant)" <[email-masked]>
Date: Sun, 10 Aug 2014 09:24:03 +0000

ECMP paths to xPEs naturally form and disappear in LDP PSNs (they are a fundamental property of that PSN type) and it is entirely normal  to find an ECMP to an xPE.

So, if you have a PW that is load balancing (again quite normal) and one of the attachments to the xPE fails, you can legitimately end  up with some traffic on one egress AC and some on another.

please you confirm that this design works correctly when a PE gets its traffic from both ACs?

Stewart

---

From: Alexander Vainshtein <[email-masked]>
Date: Sun, 10 Aug 2014 10:22:50 +0000

Stewart,

My reading of your email is like following:

1.       Within the scope of the draft the PLR is always the penultimate P LSR on the LSP terminated by the Primary PE. I believe that the draft states that explicitly

2.       The PLR cannot differentiate between the failure of the link connecting it to the Primary PE and failure of the Primary PE in a timely manner; hence every time such a link fails, the PLR assumes failure of the Primary PE to be on the safe side. T

3.       With ECMP in the PSN, there typically would be several LSPs terminated by the Primary PE, each with its own PLR. So if the link between one of these PLRs and the Primary PE fails, it would apply the mechanisms defined in the draft, while the rest of the PLRs would not do anything.

4.       As a consequence, the dual-homed PE would receive some flows of the traffic crossing a flow-aware pw from one AC and some - from the other AC.  The draft does not specify that this is possible and that the CE should be able to cope with this situation.

Is my interpretation correct?

Regards,
       Sasha

From: Stewart Bryant (stbryant) [mailto:[email-masked]]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; [email-masked]; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Stewart

---

From: "Stewart Bryant (stbryant)" <[email-masked]>
Date: Sun, 10 Aug 2014 13:38:26 +0000

That is my interpretation.

Now they could specify no PW ecmp but that restricts the scope of the draft and must be both stated and tested against the requirements.

Stewart

On 10 Aug 2014, at 11:23, "Alexander Vainshtein" <[email-masked]<mailto:[email-masked]>> wrote:

Stewart,

4.       As a consequence, the dual-homed PE would receive some flows of the traffic crossing a flow-aware pw from one AC and some – from the other AC.  The draft does not specify that this is possible and that the CE should be able to cope with this situation.

Regards,
       Sasha

From: Stewart Bryant (stbryant) [mailto:[email-masked]]
Sent: Sunday, August 10, 2014 12:24 PM
To: Alexander Vainshtein
Cc: Yimin Shen; Yakov Rekhter; [email-masked]<mailto:[email-masked]>; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Stewart

---

From: Yimin Shen <[email-masked]>
Date: Mon, 11 Aug 2014 20:15:38 +0000

Hi Sasha,

By "avoiding ECMP", I was strictly talking about the context ID FEC from the PLR's perspective, rather than avoiding ECMP broadly for other FECs on the PLR or any FECs on other routers.

Specifically, if the PLR has a set of ECMP to the context ID FEC, all the ECMP paths must traverse the primary PE only. This will ensure that all PWs be carried to the primary PE. And this can be achieved by using the proxy node approach suggested in this draft. The path(s) traversing the protector to reach the context ID FEC can be used as bypass LSP(s).

We can improve the text on this to make it clearer.

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: "Stewart Bryant (stbryant)" <[email-masked]>
Date: Mon, 11 Aug 2014 20:34:09 +0000

Yimin

How does that address the case I described?

ECMPs may form further out than the PLR, and not all May route through the PLR.

Stewart

Sent from my iPad

On 11 Aug 2014, at 22:16, "Yimin Shen" <[email-masked]<mailto:[email-masked]>> wrote:

Hi Sasha,

By “avoiding ECMP”, I was strictly talking about the context ID FEC from the PLR’s perspective, rather than avoiding ECMP broadly for other FECs on the PLR or any FECs on other routers.

We can improve the text on this to make it clearer
Thanksf,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Sunday, August 10, 2014 1:15 AM
To: Yimin Shen
Cc: [email-masked]<mailto:[email-masked]>; Yakov Rekhter; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Yimin Shen <[email-masked]>
Date: Mon, 11 Aug 2014 20:53:51 +0000

Hi Stewart, Sasha,

In LDP, a transit router may have ECMP to a give FEC (of egress PE), as you mentioned. If this LSP carries multiple PWs, the hash algorithm on the router should map a given PW to one and only one path, in order to preserve packet order on the egress PE.

Let's look at an example in the context of this draft. Suppose this transit router has 3 ECMP to the primary egress PE. On path_1 and path_2, the router is the penultimate hop, most likely because this router has 2 parallel links to the primary PE. On path_3, it is not the penultimate hop, because there are other downstream routers. Also, suppose there are 3 PWs to the egress PE, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, and PW_3 to path_3. Hence, this router serves as PLR for PW_1 and PW_2, but not for PW_3.

The PLR will set up a bypass tunnel to protect PW_1 and PW_2.  (Theoretically it could also set up 2 bypass tunnels to protect PW_1 and PW_2 separately, but it doesn't really matter here). The PLR will then attach the bypass as a backup nexthop to Path_1, and a backup nexthop to Path_2.

Now, the PLR detects a link failure over Path_1. In the dataplane, the PLR reacts to this event by moving PW_1's packets to the bypass. It will not do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and PW_3 are not affected. From CE's perspective, the CE will be receiving PW_1's traffic over the backup AC, while continuing to receive PW_2 and PW_3's traffic over the primary AC.

Likewise, if the router detects a failure of PW_2 or PW_3, the other 2 PWs will not be affected.

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Sunday, August 10, 2014 6:23 AM
To: Stewart Bryant (stbryant)
Cc: Yimin Shen; Yakov Rekhter; [email-masked]; Eric Rosen (erosen)
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Stewart,

Regards,
       Sasha

Stewart

---

From: "Stewart Bryant (stbryant)" <[email-masked]>
Date: Mon, 11 Aug 2014 21:28:13 +0000

Sent from my iPad

On 11 Aug 2014, at 22:53, "Yimin Shen" <[email-masked]<mailto:[email-masked]>> wrote:

Hi Stewart, Sasha,

Only when

A) the CW is present

and

B) there is no FAT label, or I suppose no EL if you chose to do it that way.

The point is that with some PEs you only care about flow order not absolute order.

If you want to put them out of scope then fine, but you have to either explicitly state the restriction, or fix the problem, and without taking either of those approaches you get traffic split over both ACs.

Stewart

Let’s look at an example in the context of this draft. Suppose this transit router has 3 ECMP to the primary egress PE. On path_1 and path_2, the router is the penultimate hop, most likely because this router has 2 parallel links to the primary PE. On path_3, it is not the penultimate hop, because there are other downstream routers. Also, suppose there are 3 PWs to the egress PE, and the hash algorithm has mapped PW_1 to path_1, PW_2 to path_2, and PW_3 to path_3. Hence, this router serves as PLR for PW_1 and PW_2, but not for PW_3.

The PLR will set up a bypass tunnel to protect PW_1 and PW_2.  (Theoretically it could also set up 2 bypass tunnels to protect PW_1 and PW_2 separately, but it doesn’t really matter here). The PLR will then attach the bypass as a backup nexthop to Path_1, and a backup nexthop to Path_2.

Now, the PLR detects a link failure over Path_1. In the dataplane, the PLR reacts to this event by moving PW_1’s packets to the bypass. It will not do anything for PW_2 or PW_3, because their paths are still up. IOW, PW_2 and PW_3 are not affected. From CE’s perspective, the CE will be receiving PW_1’s traffic over the backup AC, while continuing to receive PW_2 and PW_3’s traffic over the primary AC.

Thanks,

/Yimin

Stewart,

Regards,
       Sasha

Stewart

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 12 Aug 2014 04:15:08 +0000

Yimin,

My concerns related to ECMP have been mainly about multiple potential PLRs on equal cost LSPs between the ingress PE" and "Primary egress PE".

I do not see how this can be easily avoided and my reading of your response is that you do not consider it either.

One of the aspects of multiple potential PLRs has been raised by Stewart in his email: if the link between one of these and the Primary PE fails while Primary PE remains OK, the traffic would be split.

Another aspect of this scenario is that, in the case of the Primary PE failure, each of these multiple PLRs would select the Protector as its best Next Hop to the Context ID.

Regards,

Sasha

Sent: Monday, August 11, 2014 11:15 PM

Hi Sasha,

Thanks,

/Yimin

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Yimin Shen <[email-masked]>
Date: Tue, 12 Aug 2014 15:24:27 +0000

Hi Stewart,

Thanks for pointing this out. We will explicitly describe these cases in the draft. Even if traffic may arrive at CE on both ACs, it should not cause an issue for the CE, because either the CE doesn't care about packet order (case A), or packet order of sub-flow associated with a FAT label is still preserved (case B).

Thanks,

/Yimin

From: Stewart Bryant (stbryant) [mailto:[email-masked]]
Sent: Monday, August 11, 2014 5:28 PM
To: Yimin Shen
Cc: Alexander Vainshtein; Yakov Rekhter; [email-masked]; Eric Rosen (erosen)
Subject: Re: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Sent from my iPad

Only when

A) the CW is present

and

Stewart

Thanks,

/Yimin

Stewart,

Regards,
       Sasha

Stewart

---

From: Yimin Shen <[email-masked]>
Date: Tue, 12 Aug 2014 15:25:51 +0000

Hi Sasha,

Please see inline...

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Tuesday, August 12, 2014 12:15 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin,

[yshen] This draft doesn't preclude or avoid it, as I explained below.

[yshen] Packets of "one given PW" may be load-balanced over ECMP to the primary PE, due to no-CW or FAT label. In the above failure case, we only move the affected "branch of flow" over to a bypass tunnel. This shouldn't cause any issue for the CE, because either [1] the CE doesn't care about packet order (in the case where the PW has no CW), or [2] packet order of the sub-flow associated with a FAT label is still preserved.

[yshen] Yes, and the PLR should move packets from the bypass tunnel back to this LSP, which is now going over the new "best next hop" to the protector.  From the protector's perspective, this LSP will actually be the same as the bypass tunnel, because it's associated with the context ID FEC and hence its label still points to the context label table where PW label should be looked up. We can improve the text of the draft to explicitly clarify this.

Regards,

Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 12 Aug 2014 15:42:40 +0000

Yimin hi!

The problem with splitting the PW traffic is not limited to re0rdering.

In the simplest case, consider the case when the PW s have been set between two IEEE802.1 bridges that perform MAC learning.
Split traffic could easily result in frequent re-learning of a certain MAC address...

The bottom line is that if split traffic cannot be precluded, the dual-homed CE must be aware of this fact and behave accordingly (e.g., by grouping a pair of ports that support its connections to Primary PE and the Protector PE as a LAG:)).

>From my point of view these dependencies should be explicitly defined (say, in the Applicability Statement). To the best of my recollection, this has been a long-established PWE3 WG policy from its early days.
Regards,
       Sasha

From: Yimin Shen [mailto:[email-masked]]
Sent: Tuesday, August 12, 2014 6:26 PM
To: Alexander Vainshtein
Cc: [email-masked]; Yakov Rekhter; [email-masked]; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Hi Sasha,

Please see inline...

Thanks,

/Yimin

Yimin,

Regards,

Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302

---

From: Yimin Shen <[email-masked]>
Date: Wed, 13 Aug 2014 21:09:44 +0000

Hi Sasha,

Thanks for these comments! I agree with your points on these scenarios. Will describe and clarify them in the next rev.

Thanks,

/Yimin

From: Alexander Vainshtein [mailto:[email-masked]]
Sent: Tuesday, August 12, 2014 11:43 AM
To: Yimin Shen
Cc: [email-masked]; Yakov Rekhter; [email-masked]; [email-masked]
Subject: RE: [PWE3] WG Last Call for draft-ietf-pwe3-endpoint-fast-protection-01

Yimin hi!

Regards,
       Sasha

Hi Sasha,

Please see inline...

Thanks,

/Yimin

Yimin,

Regards,

Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin hi,

Regards,

Sasha

Hi Sasha,

Hope this clarifies it.

Thanks,

/Yimin

Yimin hi,

Regards,
       Sasha

Hi Sasha,

Thanks,

/Yimin

Yimin and all,

Regards,

       Sasha

Mobile: 054-9266302