Subject: Re: [PWE3] Questions on draft-martini-pwe3-iccp-01.txt

From: Luca Martini <[email-masked]>
Date: Tue, 04 Aug 2009 10:02:17 -0600

Olen,

the bottom line is that the problem you are describing below has no 
protocol solution.
If one has to meet the SP requirement of being in control of which link 
is active one has no choice but to accept that a "split-brain" problem 
is possible. Also if the two PE are connected to the network they must 
be able to reach each other through the network.
Simply the SP MUST make sure that there is enough redundancy in the 
connection between the two PEs so that the "split-brain" situation 
cannot possibly happen.

More comments below.
Olen Stokes wrote:
>
> Please consider the following network based on Figure 1 in the draft:
>
> +===============+ +======================
>
> | | |
>
> | +-----+ | | +-----+
>
> | | PE1 ||<----|---PW---|-->|| PE3 ||<-- PW -->
>
> -|--| | | | | ||<-- PW -->
>
> / | +-----+ | | +-----+
>
> / | || | | ^
>
> / | || ICCP | | | PW
>
> +---------+ / | || | | V
>
> +----+ | | / | +-----+ | | +-----+
>
> | CE | | Access |/ | | PE2 ||<----|---PW---|-->|| PE4 ||<-- PW -->
>
> | +---+ Network |-------|--| | | | | ||<-- PW -->
>
> +----+ | | | +-----+ | | +-----+
>
> | | | | |
>
> +---------+ | Redundancy | | H-VPLS
>
> ^ | Group | | Full-Mesh Core
>
> | +===============+ +======================
>
> |
>
> Multi-homed Network
>
> PE1 and PE2 provide redundant access to an H-VPLS network. They 
> effectively combine to provide the function of a MTU-s as shown in 
> Figure 3 of RFC 4762. They attach to different PEs to avoid a single 
> point of failure in the H-VPLS core.
>
> The access network is a generic Ethernet L2 network. A packet destined 
> to an unknown unicast MAC address sent from the CE will be delivered 
> to both PE1 and PE2.
>
> ICCP is used between PE1 and PE2 to determine the active PW to the 
> H-VPLS core.
>
> From the responses below, the above network configuration seems to be 
> a valid use of ICCP.
>
> While it may be typical that PE1 and PE2 are directly connected and 
> co-located, this is not always the case. It is definitely possible in 
> some customer installations that the links between them may fail 
> without PE1 or PE2 getting a “loss of signal”.
>
> Given the above type of network, the question becomes what happens 
> when all of the links between PE1 and PE2 fail? Will PE1 and PE2 both 
> assume that the other node has failed since the BFD session between 
> them will go down? If so, is there not a loop when the PWs from PE1 
> and PE2 are both active at the same time?
>
assuming that for some strange reason the PE cannot communicate with 
each other through the network, you would have a loop.
>
> I apologize, but I did not completely understand the statement:
>
> “yes. the split brain problem cannot be completely avoided. This is 
> why it is very important that only if all possible connectivity is 
> lost, the ICCP/BFD session goes down. If a PE is isolated , no looping 
> is possible, however the service might no longer work.”
>
> In the above scenario, the two PEs are isolated from each other, but 
> they are not isolated from the network.
>
> I apologize again, but I do not see how to distinguish with complete 
> certainty in all network environments between:
>
> 1) The node with the active PW has crashed and therefore the secondary 
> PW should be activated, and
>
> 2) All of the links between the RG nodes have failed and therefore the 
> secondary PW should not be activated
>
The assumption is that connectivity between the 2 PE cannot be lost by 
design , unless either a PE has gone completely offline or is completely 
isolated.
>
> My concern is that in order to apply ICCP as it is defined today to 
> generic Ethernet access in an H-VPLS network, it has to be guaranteed 
> that the ICCP session cannot go down due to link failure while the RG 
> nodes (MTU-s) still have connectivity to the H-VPLS core network. I am 
> not sure how to make that guarantee in the networks in which some 
> customers want to deploy H-VPLS.
>
The network has to be designed to guarantee that a split-brain problem 
cannot occur. This is not something you can do in the protocol given the 
requirements.

Thanks.
Luca

> I understand that the Ethernet section of the draft has not completed 
> and that more insight into this discussion may appear at that time.
>
> Cheers,
>
> Olen
>
> *From:* Luca Martini [mailto:[email-masked]]
> *Sent:* Friday, July 24, 2009 8:58 AM
> *To:* Olen Stokes
> *Cc:* [email-masked]
> *Subject:* Re: [PWE3] Questions on draft-martini-pwe3-iccp-01.txt
>
> Olen,
> Sorry for the delay in response , somehow i missed this e-mail.
>
> More below.
>
> Olen Stokes wrote:
>
> I have the following questions on draft-martini-pwe3-iccp-01.txt:
>  
> 1)      In Section 4.1, it states that "the redundancy mechanism employed towards the access node/network can be one of a multitude of technologies".  Are there any restrictions on the access network?  In particular, are there any restrictions about a given packet reaching both PE1 and PE2 from the access network in Figure 1?  If not, is it true that a packet received on a PW at PE1 and sent toward the access network may reach PE2 via the access network?
>  
>   
>
> How the packet is sent to the PE on the AC is not dependent on ICCP. 
> THe protocol used i nthe access network will dictate how we choose the 
> active link, and how we forward the packet.
>
>
>
>
>
>
> 2)      In Section 5.2, it states, "The RG will not become operational until the corresponding RG Connect Message has been received."  If a single node comes up and does not receive any ICCP capable announcements, does it become operational?  What if it has been configured that a second node is supposed to be present?  If so, what happens when a second node later advertises that it is ICCP capable?  Is the RG operational between the second capability announcement and the subsequent connect message?
>  
>   
>
> ok, It takes at least 2 nodes to have an RG become operational. So if 
> one node comes up and there are no other PEs advertising that 
> particular RG, the RG never becomes operational.
> The capability in LDP is only used to for an LSR to be allowed to send 
> an RG connect message. The RG becomes operational only when both PEs 
> have sent the RG connect messages. Sec 4.2 in 
> draft-ietf-pwe3-iccp-01.txt is quite clear on the procedures.
>
>
>
> 3)      In Section 6, it states the following:
>  
> - BFD
>        Run a BFD session [BFD] between the PEs that are members of a
>        given RG, and use that to detect PE node failure. This assumes
>        that resiliency mechanisms are in place to protect connectivity
>        to the remote PE nodes, and hence loss of BFD periodic messages
>        from a given PE node can only mean that the node itself has
>        failed.
>  
> How is this guaranteed?  There can be n-links between two interconnected PEs in a Redundancy Group.  If the BFD session between the two PEs goes down, all that is really known is that the two PEs cannot communicate over those n-links.  How does this guarantee that the remote PE is not forwarding access network data to the core network over other links?
>  
>   
>
> There are no guarantees against multiple failure events. We are 
> protecting against a single failure , and all we are saying here is 
> that the PEs MUST have really good connectivity between each other to 
> almost exclude the possibility to loose communication between the PEs.
>
> Take a Redundancy Group as shown in Figure 1 where PE1 and PE2 are H-VPLS MTU-s switches with connections to the core.  Assume that they have dedicated interconnection with n-links.  If somehow all n-interconnect links fail, then will not both nodes assume that the remote node has failed?  If so, then it appears that a loop is very possible.  Losing connection to the core after a large number of failures would seem to be a better alternative than any risk of a loop.
>  
>   
>
> yes. the split brain problem cannot be completely avoided. This is why 
> it is very important that only if all possible connectivity is lost, 
> the ICCP/BFD session goes down. If a PE is isolated , no looping is 
> possible, however the service might no longer work.
> In a typical application the two PEs are directly connected and 
> co-located.
>
> 4)      Section 8.1.5 - It states that a PW Id TLV is "used to communicate the configuration of PWs for VPWS."  Is there some reason that this cannot be used with a (H)VPLS?  For example, there could be redundant MTUs that are connecting to a core group of PEs as discussed above.
>  
>   
>
> no - this is only terminology. We meant to say "For PW" . Of course 
> H-VPLS with PW access is a valid application.
>
> 5)      Section 8.1.5 - There is a Peer ID field included in the TLV.  What is the intent of the Peer ID?
>   
>
> this is defined as "Four octet LDP Router ID of the peer at the far 
> end of the PW"
> This is necessary to uniquely identify the PW when using FEC 128.
>
>
>  
> 6)      Are there any provisions for when an access network might segment with part of the access network still having connectivity to one PE while the remainder of the network has connectivity to the other PE?  I realize that how this condition is recognized may be out of the scope of this document, but how the PEs communicate this condition could potentially be included.
>  
>   
>
> this would have to be handled by the access network protocol. It is 
> out of scope of ICCP.
>
> Thanks.
> Luca
>
>
> Thanks for your attention.
>  
> Cheers,
> Olen
>  
>  
>  
>  
>  
>  
>  
>   
>  
> ------------------------------------------------------------------------
>
>
>   
>  
> _______________________________________________
> pwe3 mailing list
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/pwe3
>   
>

---

From: Luca Martini <[email-masked]>
Date: Thu, 13 Aug 2009 18:07:18 -0600

Yuanlong,

I asked stewart if I could make the changes that youy proposed, and he 
agreed that all the editorial changes can be made now.
More comments Below.
Thanks.
Luca

Jiang Yuan-long wrote:
> Hi Luca,
> I have some comments on your draft-ietf-pwe3-segmented-pw-12.
> Technical comments:
> 1. In Sec 7.4.1, the description of the 5th sub-TLV is not only "The
> FEC of last PW segment traversed", but also "The Attachment
> Identifier of the last PW segment traversed" in the same section.
> IMO, neither is the exact description.
The FEC contains the AI.

To be very accurate this should have said "The FEC element of last PW 
segment traversed"
I will correct this.

> As defined in RFC 4447, FEC element 129 consists of <AGI,
> SAII, TAII>, and AI consists of <AGI, AII>, so the first is closer
> to the actual format in this ID. But FEC element 129 also includes
> a sub-TLV header which is removed here, therefore I think a more
> accurate description is needed. How about "The neat content of the
> FEC of last PW segment traversed" or something new?
> BTW, this description should also align with the sub-TLV type 5
> in Sec 13.5 (in the next to the last line):
> " 0x05 variable AI of last PW segment traversed"
I also fixed this to match the text above.
> 2. Also in Sec 7.4.1, the last sub-TLV is very confusing:
> " - L2 PW address of PW Switching Point (recommended format).
> This sub-TLV type contains a L2 PW address of PW Switching Point in
> the format described in [RFC5003]. This includes the AII type field ,
> and length, as well as the L2 PW address."
> From the above paragraph, it seems that 'L2 PW address' is defined in
> RFC 5003, but in fact there is no such a definition. And the actual
> components of L2 PW address is also misleading.
> I think a more clear definition is needed, how about the following:
> " - L2 PW address of PW Switching Point (recommended format).
> This sub-TLV type contains a L2 PW address of PW Switching Point,
> which includes a Global ID field and a Prefix field as described
> in the AII type 2 construct defined in [RFC5003]."
> 3. In Sec 13.5, in the last line, a sub-TLV type 6 is defined:
> 0x06 10 L2 PW address of PW Switching Point
> But the length should be 8 rather than 10, as both Global ID and Prefix
> are 4 octets fields. This mismatch may root in the original ID of RFC 
> 5003,
> which defined Global ID as 6 octets in length, but changes it to 4 in the
> final RFC.
Yes, this needs to be more accurate. It was meant to specify the prefix 
allocated to the PE without an AC ID.

> BTW, the same problem also exists in draft-ietf-pwe3-dynamic-ms-pw-09.
Can you give me more details about the problem in 
draft-ietf-pwe3-dynamic-ms-pw-09 ?
( maybe in a separate e-mail , so i can fix that document as well )

> Some editoral comments:
> In Sec 7.4, s/the the/to the/
fixed.
> In Sec 8.1, in the 4th line, "VC Label" should be replaced with "PW 
> label".
fixed.
> In Sec 9.3, in the first sentence,
> s/interface parameter field/Interface Parameter sub-TLV field/
this is actually correct. The FEC 128 interface parameter filed is 
called out correctly.

> s/sub-TLV interface parameter/Interface Parameters sub-TLV/
fixed.
> In Sec 9.4.2, in the first line, s/form/from/
fixed.
> More than one occurence of "no packets MUST be sent..." found in this ID,
> maybe it would be better to say "packets MUST NOT be sent..."
yes, fixed.
> In Sec 10.1,
> s/an an/an/
> s/MH-PW/MS-PW/
> s/more then/more than/
fixed.
> In Sec 13.1, s/ne/new/
> In Sec 13.2, inserts 'uses' behind "This document"
fixed.
> In Sec 14 and 15, there are some minor copy & paste errors:
> - The title of [RFC4447] should be "Pseudowire Setup and Maintenance
> Using the Label Distribution Protocol (LDP)", rather than "Transport of
> Layer 2 Frames Over MPLS". The latter is the title of a historic RFC
> (RFC 4906).
Yes :-) i cut and pasted an old quote.
Fixed.
> - The title of [RFC3438] should be "Layer Two Tunneling Protocol (L2TP)
> Internet Assigned Numbers Authority (IANA) Considerations Update"
fixed
> - Both [RFC3985] and [RFC4379] appear twice in the list of References.
> Please delete one copy for each.
fixed.

> I find the following nomenclatures are used for the same TLV across 
> this ID:
> "Switching Point PE TLV，Switching Point TLV，S-PE TLV，
> PW sw TLV, PW Switching Point TLV, Pseudowire Switching TLV"
> If we could simplify them to two (one full name and another 
> abbreviation),
> it would be a little easier for us to read this document ^&^
I changed all PW sw TLV to S-PE TLV
changed PW Switching Point TLV to Pseudowire Switching Point PE TLV Type.
changed Pseudowire Switching TLV to Pseudowire Switching Point PE TLV

> Best Regards
> Yuanlong Jiang

---

From: Jiang Yuan-long <[email-masked]>
Date: Fri, 14 Aug 2009 10:33:06 +0800

Luca, 

Please see my comments inline.

  Thanks
Yuanlong Jiang

----- Original Message ----- 
From: "Luca Martini" <[email-masked]>
To: "Jiang Yuan-long" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 14, 2009 8:07 AM
Subject: Re: Comments on draft "Segmented Pseudowire"

> allocated to the PE without an AC ID.

[J] I still can not understand why the Lengh of  L2 PW address of PW 
Switching Point remains 10 in the new draft 
draft-ietf-pwe3-segmented-pw-13.txt. IMO, If AC ID is included, then
the Length should be 12, otherwise it should be 8.
And this sub TLV should also be aligned with draft-ietf-pwe3-dynamic-ms-pw.

> ( maybe in a separate e-mail , so i can fix that document as well )

[J] I've sent another email before, if you have not received it, I can send you
another copy:)

---

From: Luca Martini <[email-masked]>
Date: Fri, 14 Aug 2009 12:08:23 -0600

Jiang Yuan-long wrote:
> From: "Luca Martini" <[email-masked] <mailto:[email-masked]>>
> To: "Jiang Yuan-long" <[email-masked] <mailto:[email-masked]>>
> Cc: <[email-masked] <mailto:[email-masked]>>
ok now I see what the problem is ...
the length in the IANA action section is indeed wrong.
it should be 12.
Note that the length field in the IANA allocation listing is just for 
informational purposes. The code should always use the length in the packet.
The reason I even included a length in the IANA registry is to make sure 
people do no create huge TLVs that do not belong in the protocol.

I fixed in both document sources, this will appear in the next revision.
I'll send this as a correction to the rfc editor is we do not need 
another revision.

Thanks!
Luca

> [J] I've sent another email before, if you have not received it, I can 
If the problem was just the wrong length, I already fixed that .

---

From: Luca Martini <[email-masked]>
Date: Fri, 14 Aug 2009 12:07:00 -0600

> draft-ietf-pwe3-dynamic-ms-pw.
ok now U see what the problem is ...

Thanks!
Luca