Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

From: John E Drake <[email-masked]>
Date: Fri, 3 Apr 2015 13:38:59 +0000

Weiguo,

I have been marginally paying attention to this email thread and I thought I should update the points we have established.  In addition to the points listed in my email, below, I think we have established that;

1)  Your proposal is even more broken than I I previously indicated because you have no way to identify to which EVI your DF election handshake applies

2)  Your DF election handshake only applies to single-active MHN, if it applies at all.

3)  Your DF election handshake creates a window in which no PE is the DF which leads to the black-holing of traffic.

4)  Because the EVPN routes are not time-stamped, you entire discussion of NTP and PTP is completely irrelevant.

Yours Irrespectively,

John

From: John E Drake
Sent: Thursday, March 26, 2015 6:00 PM
To: 'Haoweiguo'; Patrice Brissette (pbrisset)
Cc: Ali Sajassi (sajassi); [email-masked]
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

To recap,

We have established that your proposal is untenable because of its control plane load.

We have established that your proposal is based upon a flawed understanding of the DF election in RFC 7432.

What you are now arguing is that your draft prevents two or more PEs from being DF simultaneously.  This is clearly nonsense.

Furthermore, we have established that having two or more DFs for what even you admit is a brief transient leads to duplicate traffic, which is acceptable, but not loops, your assertion to the contrary.

Yours Irrespectively,

John

From: Haoweiguo [mailto:[email-masked]]
Sent: Thursday, March 26, 2015 5:37 PM
To: John E Drake; Patrice Brissette (pbrisset)
Cc: Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

John,

As your understanding of the EVPN draft, the DF election mechanism has more serious side effect, it will have short time traffic loop,i.e., dual DF PEs will exist for a short time. I think dual DF PEs is absolutely not tolerated, because native ethernet header has no TTL, up to several hundred ms traffic loop normally not tolerated in commertial networks.

As your understanding, the PEs should do as following:

1. Accurate timer sync. NTP accuracy is bad, 1588v2 is good but have rarely deployment.

Assuming PE1,PE2 and PE3 have consistent timer clock, when PE3 joins ESI and trigger DF re-election. When reception timer expires:

PE1 upgrades to DF PE.

After reception timer+ ES route transmission timer:

PE2 downloads to non-DF PE.

So in timer clock sync case, dual DF PEs will exist at least transmission timer.

If NTP is used for timer sync, because it has bad accuracy, dual DF PEs will exist more longer timer.

So as your understanding for DF election, the drawback is more clear.

Thanks,

weiguo

________________________________
From: John E Drake [[email-masked]]
Sent: Friday, March 27, 2015 4:41
To: Haoweiguo; Patrice Brissette (pbrisset)
Cc: Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm
Weiguo,

We have already established that your proposal is untenable because of its control plane load.

What we are now discussing is that your proposal is based upon a misunderstanding of the algorithm in RFC 7432.  You are assuming that PE1 will advertise an ES route and then wait for the configured interval before performing the DF election while PE2 and PE3 will perform the DF election as soon as they receive the ES route from PE1.  This is not what RFC 7432 says.

Rather, what is says is that the advertisement of the ES route by PE1 and its receipt by PE2 and PE3 causes all three PEs to start the configured interval timer  - "3. When the timer expires, each PE builds an ordered list of the IP addresses of all the PE nodes connected to the Ethernet segment (including itself), in increasing numeric value."

Yours Irrespectively,

John

From: Haoweiguo [mailto:[email-masked]]
Sent: Thursday, March 26, 2015 3:26 PM
To: John E Drake; Patrice Brissette (pbrisset)
Cc: Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Pls read my detail replies to Satya. If you still can't catch it, pls read my draft and EVPN base protocol,  thanks

weiguo

________________________________
From: John E Drake [[email-masked]]
Sent: Friday, March 27, 2015 1:28
To: Patrice Brissette (pbrisset)
Cc: Haoweiguo; Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm
I think Patrice is correct.  Your proposal doesn't solve the problem and it does so at huge cost.

Sent from my iPhone

On Mar 26, 2015, at 10:34 AM, Patrice Brissette (pbrisset) <[email-masked]<mailto:[email-masked]>> wrote:
Weiguo,

I'm not sure I'm following here.
Don't you have the same issue with your handshaking mechanism?
If you don't know your peer, how can you handshake?

Regards,

Patrice
   [Image removed by sender.]

Patrice Brissette
TECHNICAL LEADER.ENGINEERING

[email-masked]<mailto:[email-masked]>
Phone: +1 613 254 3336

Cisco Systems Canada Co. / Les Systemes Cisco Canada CIE
Canada
Cisco.com<http://www.cisco.com/global/CA/>

[Image removed by sender.]Think before you print.

This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.

Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.

From: Haoweiguo <[email-masked]<mailto:[email-masked]>>
Date: Thursday, March 26, 2015 at 10:24 AM
To: Patrice Brissette <[email-masked]<mailto:[email-masked]>>, John E Drake <[email-masked]<mailto:[email-masked]>>, Ali Sajassi <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Hi Patrice,

Up to reception timer traffic disruption in transient phase is one of the issues.

Thanks,

weiguo

________________________________
From: Patrice Brissette (pbrisset) [[email-masked]<mailto:[email-masked]>]
Sent: Thursday, March 26, 2015 20:54
To: Haoweiguo; John E Drake; Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm
Weiguo,

You mention "But if your draft have not solved all issues",
Can you explain what Satya's draft is not solving?

Regards,

Patrice
   [Image removed by sender.]

Patrice Brissette
TECHNICAL LEADER.ENGINEERING

[email-masked]<mailto:[email-masked]>
Phone: +1 613 254 3336

Cisco Systems Canada Co. / Les Systemes Cisco Canada CIE
Canada
Cisco.com<http://www.cisco.com/global/CA/>

[Image removed by sender.]Think before you print.

This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.

Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for Company Registration Information.

From: Haoweiguo <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, March 25, 2015 at 10:38 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>, Ali Sajassi <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Hi John,

 Firstly i think EVPN community should reach consensus on the issues of current DF election mechanism. All these issues should be resolved in a single new DF election draft,rather than in multiple separate drafts. If your draft can solve all these issues and stable, i have no question for its progressing. But if your draft have not solved all issues, i think it had better combine with other drafts to provide a comprehensive solution. I think the issues listed in draft-rabadan-bess-evpn-ac-df-01 and draft-hao-bess-evpn-df-handshaking-00 is valid, it should be resolved. So i think although your new Hash algorithm for DF election is good, it only includes partial enhancements, maybe it still needs some time for consensus.

Thanks,

weiguo

________________________________
From: John E Drake [[email-masked]<mailto:[email-masked]>]
Sent: Thursday, March 26, 2015 7:16
To: Haoweiguo; Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm
Weiguo,

Your proposal introduces a control plane processing load that is O(#EVIs * PEs) per DF election and given that there can be 4K EVIs per ES, this looks like a *substantial* load.  Furthermore,
you can't  use the ES route to co-ordinate DF election because you would need to carry your new extended community for each EVI and they would not all fit.  You also can't use the Per EVI Ethernet AD route because that is processed by all PEs in the EVI.

I think that from a practical perspective the new DF election proposed in Satya's draft is sufficiently stable that it renders your draft moot, even if it could be made to work.
Yours Irrespectively,

John

From: BESS [mailto:[email-masked]] On Behalf Of Haoweiguo
Sent: Wednesday, March 25, 2015 5:34 PM
To: Ali Sajassi (sajassi); [email-masked]<mailto:[email-masked]>
Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Hi Ali,

Thanks for your information. I scanned through this draft, it really introduces inter-chassis message for DF election handshaking, the requirements in this draft is to eliminate transiet Loop and traffic duplication. Current EVPN DF election mechanism already eliminated loop and traffic duplication by configuring long reception timer on each multi-homed PE, but up to reception timer traffic disruption issue still exist. EVPN for DCI is an important use case for EVPN, up to reception timer traffic disruption can't be tolerated for service providers, it should be improved.

Also for accuracy, i think handshaking state machine on each multi-homed PE is also needed. From solution perspective, in my draft, no inter-chassis message is introduced, only one new extended community is introduced, i think the process is comparatively simple than your following draft.

Current EVPN DF election has some drawbacks, so there are three new drafts about DF election emerged. I think BESS WG can consider these three drafts in global view,  a single,comprehensive new DF election draft is hoped.

thanks.

Thanks,

weiguo

________________________________
From: BESS [[email-masked]<mailto:[email-masked]>] on behalf of Ali Sajassi (sajassi) [[email-masked]<mailto:[email-masked]>]
Sent: Thursday, March 26, 2015 0:21
To: [email-masked]<mailto:[email-masked]>
Subject: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

FYI- First published July 4, 2011

https://datatracker.ietf.org/doc/draft-sajassi-l2vpn-evpn-segment-route/

-Ali

---

From: Haoweiguo <[email-masked]>
Date: Tue, 7 Apr 2015 02:03:22 +0000

Hi John,

Your summary is good. Pls see my further reply inline with [weiguo].

Thanks,

weiguo

Sent: Friday, April 03, 2015 21:38

Weiguo,

1)  Your proposal is even more broken than I I previously indicated because you have no way to identify to which EVI your DF election handshake applies
[weiguo]: Firstly we had better reach consensus on the problem(Loop and MAC flip-flop in transient peirod for MHN) that we are trying to resolve, then we can search for the suitable solution. The current proposed solution is to use A-D route for handshaking, the A-D route can identify EVI.
2)  Your DF election handshake only applies to single-active MHN, if it applies at all.
[weiguo]: Yes, currently our DF election handshake can be used to avoid loop and mac flip-flop for single-active MHN in transiet state. For MHD, if CE receives BUM traffic from PE1, the CE won't send the traffic back to another multi-homed PE2, so the loop won't be formed.
3)  Your DF election handshake creates a window in which no PE is the DF which leads to the black-holing of traffic.
[weiguo]:The window is  BGP message transmission timer, all handshake mechanism has this issue and i think it is more acceptable than loop and mac flip-flop.
4)  Because the EVPN routes are not time-stamped, you entire discussion of NTP and PTP is completely irrelevant.
[weiguo]: OK.
Yours Irrespectively,

John

From: John E Drake

To recap,

Yours Irrespectively,

John

John,

PE1 upgrades to DF PE.

Thanks,

weiguo

Rather, what is says is that the advertisement of the ES route by PE1 and its receipt by PE2 and PE3 causes all three PEs to start the configured interval timer  - “3. When the timer expires, each PE builds an ordered list of the IP addresses of all the PE nodes connected to the Ethernet segment (including itself), in increasing numeric value.”

Yours Irrespectively,

John

weiguo

Sent from my iPhone

I’m not sure I’m following here.
Don’t you have the same issue with your handshaking mechanism?
If you don’t know your peer, how can you handshake?

Regards,

Patrice

Patrice Brissette

Hi Patrice,

Thanks,

weiguo

You mention "But if your draft have not solved all issues”,
Can you explain what Satya’s draft is not solving?

Regards,

Patrice

Patrice Brissette

Hi John,

Thanks,

weiguo

Your proposal introduces a control plane processing load that is O(#EVIs * PEs) per DF election and given that there can be 4K EVIs per ES, this looks like a *substantial* load.  Furthermore,
you can’t  use the ES route to co-ordinate DF election because you would need to carry your new extended community for each EVI and they would not all fit.  You also can’t use the Per EVI Ethernet AD route because that is processed by all PEs in the EVI.

I think that from a practical perspective the new DF election proposed in Satya’s draft is sufficiently stable that it renders your draft moot, even if it could be made to work.
Yours Irrespectively,

John

Hi Ali,

thanks.

Thanks,

weiguo

-Ali

---

From: John E Drake <[email-masked]>
Date: Tue, 7 Apr 2015 13:53:59 +0000

Weiguo,

Snipped, comments inline.

Yours Irrespectively,

John

Sent: Monday, April 06, 2015 10:03 PM

Hi John,

Thanks,

weiguo

[JD]  I don't think there is a problem to be solved, so the first order of business is for you to write a draft clearly articulating the problems you perceive and their ramifications.  Please keep in mind that single-active, and more specifically single-active MHN, is not where I think we should focus our resources.  DCI, for example, uses all-active.

Further, as I had previously pointed out, we can't use an EVPN route that goes to all PEs so the Per EVI Ethernet AD route can't be used.

---

From: "Satya Mohanty (satyamoh)" <[email-masked]>
Date: Wed, 8 Apr 2015 19:00:44 +0000

Also, as in the private communication, I don’t really see any mac flip-flop issue as we are considering the same ESI.
It is "NOT" equivalent to the mac-mobility.

Thanks,
--Satya

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 7, 2015 6:53 AM
To: Haoweiguo <[email-masked]<mailto:[email-masked]>>, "Patrice Brissette (pbrisset)" <[email-masked]<mailto:[email-masked]>>
Cc: "Ali Sajassi (sajassi)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Weiguo,

Yours Irrespectively,

John

Hi John,

Thanks,

weiguo

[JD]  I don’t think there is a problem to be solved, so the first order of business is for you to write a draft clearly articulating the problems you perceive and their ramifications.  Please keep in mind that single-active, and more specifically single-active MHN, is not where I think we should focus our resources.  DCI, for example, uses all-active.

Further, as I had previously pointed out, we can’t use an EVPN route that goes to all PEs so the Per EVI Ethernet AD route can’t be used.

---

From: "Ali Sajassi (sajassi)" <[email-masked]>
Date: Thu, 9 Apr 2015 01:32:25 +0000

It seems like this is a solution looking for a problem. The draft is written for all-active redundancy mode and it talks about loop scenarios in figure-1. During the BESS session, I went to the microphone and stated that there can NOT be a loop for figure-1 because of SH label for both All-Active and Single-Active redundancy mode. It seems like after a long email thread, it is concluded that there is no loop during a transient state for a Ethernet Segment operating in either All-Active or Single-Active redundancy mode.

In order to prevent loop, RFC 7432 has triple mechanism:

1) It requires a PE to start a timer upon Ethernet Segment auto-discovery and ES route advertisements, to ensure that other PEs have the same view of the redundancy group –i.e., how many PEs participating in a given ES and thus be able to perform DF election with the right info. This procedure is described in section 8.5 of the draft. So, I don’t see how we can have a transient loop when a PE discovers a new ES. Also, with HRW algorithm, I don’t see how we can have a transient loop during a failure either.

2) EVPN uses SH label to prevent a packet coming from an ES to be sent to the same ES.

3) In case of mis-configuration and backdoor connections among customer sites (which may be only single-homed), the RFC describes how to catch such scenarios. These scenarios goes beyond multi-homing!!

So in summary, there is a consensus that there is no loop for the described scenario and if there was a loop for a TBD scenario, the above triple mechanism can handle it well.

Cheers,
Ali

From: Haoweiguo <[email-masked]<mailto:[email-masked]>>
Date: Monday, April 6, 2015 at 7:03 PM
To: John E Drake <[email-masked]<mailto:[email-masked]>>, "Patrice Brissette (pbrisset)" <[email-masked]<mailto:[email-masked]>>
Cc: Cisco Employee <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [bess] Handshaking among PEs in an EVPN ES based on Paxos algorithm

Hi John,

Thanks,

weiguo

Weiguo,

John

From: John E Drake

To recap,

Yours Irrespectively,

John

John,

PE1 upgrades to DF PE.

Thanks,

weiguo

Yours Irrespectively,

John

weiguo

Sent from my iPhone

Regards,

Patrice

Patrice Brissette

Hi Patrice,

Thanks,

weiguo

Regards,

Patrice

Patrice Brissette

Hi John,

Thanks,

weiguo

John

Hi Ali,

thanks.

Thanks,

weiguo

-Ali