Subject: Re: [pim] Some questions on RFC 6754

From: "Heidi Ou (hou)" <[email-masked]>
Date: Thu, 18 Jul 2013 23:23:10 +0000

Hi Bharat,

Please find reply inline, search for [heidi]

>________________________________________
>From: Bharat Joshi
>Sent: Monday, July 08, 2013 12:17 PM
>To: [email-masked]
>Subject: Some questions on RFC 6754
>
>Hi,
>
>       I was reading up RFC 6754 to understand the requirement and
>solution proposed for ECMP routes handling and I have some basic
>questions. It would be great if someone can asnwer them for me.
>
>Regards,
>Bharat
>
>€ In section 1, in sentence:
>
>This document introduces the ECMP Redirect, a mechanism to improve the
>RPF procedure over ECMP.
>
>      I did not get what does the term 'RPF procedure over ECMP'
>actually means? Can somebody please elaborate the same?

[heidi] "RPF procedure" means the algorithm a router runs to find the
[heidi] RPF neighbor for a source or RP. The steps include routing table
[heidi] lookups and tie break evaluation.

>
>€ In section 1, in sentence:
>
>While implementations supporting ECMP have been deployed widely, the
>existing RPF selection methods have weaknesses
>
>      Should 'RPF selection' be 'Upstream PIM neighbor selection'?

[heidi] The same meaning.

>
>€ In section 1, in sentence
>
>For example, there is no straightforward way to tell two downstream
>routers to select either the same or different RPF neighbor routers for
>the same traffic flows.
>
>      What are the conditions where downstream routers would want to
>choose two different RPF neighbors for the same traffic flows? I was
>thinking that we need some method like a hash to make sure that all
>downstream router uses the same upstream neighbor so that only one
>router forwards traffic for a given channel. This will also avoid need
>for assert messages in such scenarios.

[heidi] Please refer to the IETF 80 proceedings for more background
information. 
[heidi] But in short, due to different implementations deployed, different
tie-break 
[heidi] algorithms used and/or policies configured, two routers might very
well choose 
[heidi] different RPF neighbors. It is not common but it happens.

>
>€ In Overview, in sentence
>
>When the RPF neighbor router receives the Join message and finds that
>the receiving interface is one of the ECMP interfaces, it will check if
>the same flow is already being forwarded out of another ECMP interface.
>
>      How does a upstream router figures out that the receiving
>interface is one of the ECMP interfaces for the downstream router?

[heidi] If the router sees Join on an interface, and the interface is one
[heidi] of those in the ECMP bundle, the interface is one of the ECMP
interface.

>
>      Also, it is not clear how the upstream route can check if the same
>flow is being forwarded out of another ECMP interfaces? Are these
>interfaces pointing to the same LAN?
>
>       Let me try to put a figure and ask this.
>
>        +------+          +-----+
>         |  R1   |            |  R2 |
>        +------+          +-----+
>             |                     |
>   +-----------------------------------+
>             |                     |
>        +------+          +-----+
>         |  R3   |            |  R4 |
>        +------+          +-----+

[heidi] The ECMP bundles contain interfaces on different LANs. In the
digram
[heidi] you drew above, all routers are connected to the same LAN, regular
PIM 
[heidi] Assert will take place to determine RPF path.

Thanks
- Heidi

>
>In above configuration, how R1 would come to know if R3 has an ECMP
>route for a source/RP through both R1 and R2? Also, if there is a flow
>already exist through R2, how will R1 come that the flow is because of
>the ECMP route in R3 and R4?
>
>Am I missing something here?
>
>€ Its not clear how will the routers (R1 to R4) create ECMP bundle? Is
>it that in this particular case, there will only be one interface in
>that ECMP bundle?
>
>€ It would have been great if a figure was used to describe the
>operations. I know its my limitation as I am not able to understand the
>text correctly.
>
>
>**************** CAUTION - Disclaimer *****************
>This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
>solely
>for the use of the addressee(s). If you are not the intended recipient,
>please
>notify the sender by e-mail and delete the original message. Further,
>you are not
>to copy, disclose, or distribute this e-mail or its contents to any
>other person and
>any such actions are unlawful. This e-mail may contain viruses. Infosys
>has taken
>every reasonable precaution to minimize this risk, but is not liable for
>any damage
>you may sustain as a result of any virus in this e-mail. You should
>carry out your
>own virus checks before opening the e-mail or attachment. Infosys
>reserves the
>right to monitor and review the content of all messages sent to or from
>this e-mail
>address. Messages sent to or from this e-mail address may be stored on the
>Infosys e-mail system.
>***INFOSYS******** End of Disclaimer ********INFOSYS***
>_______________________________________________
>pim mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/pim
>
>

---

From: Bharat Joshi <[email-masked]>
Date: Fri, 19 Jul 2013 04:28:22 +0000

Hi Heidi,

        Please see my response in line.

Regards,
Bharat

> [heidi] lookups and tie break evaluation.

I understand 'RPF procedure' but your explanation above is still not enough to explain what does 'RPF procedure over ECMP' means?

My thinking was that this RFC has proposed a mechanism (ECMP redirect) to suggest downstream neighbors to select another upstream neighbor for its PIM join. I am not sure how it "improces" the RPF procedure over ECMP. It bascially direct the downstream neighbor to use another neighbor. Right?

> [heidi] The same meaning.

IMHO, we are selecting upstream PIM neighbor. RPF is a method to do that. Right?

> [heidi] different RPF neighbors. It is not common but it happens.

I perfectly understand the reason for selecting two different upstream neighbors but that issue can be resolved by using a simple hash algorithm instead of adding this new PIM ECMP redirect message. In fact, I was looking for such a solution and when I found this RFC solves a different issue, I have submitted a very small draft to solve this. Please do review the draft at http://tools.ietf.org/html/draft-joshi-pim-ecmp-neighbor-select-00‎

> [heidi] of those in the ECMP bundle, the interface is one of the ECMP

Please bear with me but I still do not understand. How the upstream router will be able to find out if downstream router has an ECMP route for a specific source/RP?

I sincerely feel that RFC is not clear enough on how this is done? In fact, I was later looking at AD and IESG comments and I see that Stewart had the same doubt. Please find his question submitted on '2012-06-20' at http://datatracker.ietf.org/doc/rfc6754/history/ Unfortunately, I could not find the author's response for his questions and so I asked here.

> [heidi] Assert will take place to determine RPF path.

It is not clear from the text in RFC that we are talking about two different LANs. In fact, when I looked through the mailing list, I found an e-mail from Adrain who has a similar issue. E-mail is at http://www.ietf.org/mail-archive/web/pim/current/msg02396.html.

Overall, IMHO I feel that the text in this RFC needs improvement to clearly define the network configuration it is aimed for. A figure explainging the same will be very helpful. I am not sure if it will be good to file an Errata to cleear this confusion or not. I will leave this to Working-Group Chair to decide.

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely 
for the use of the addressee(s). If you are not the intended recipient, please 
notify the sender by e-mail and delete the original message. Further, you are not 
to copy, disclose, or distribute this e-mail or its contents to any other person and 
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken 
every reasonable precaution to minimize this risk, but is not liable for any damage 
you may sustain as a result of any virus in this e-mail. You should carry out your 
own virus checks before opening the e-mail or attachment. Infosys reserves the 
right to monitor and review the content of all messages sent to or from this e-mail 
address. Messages sent to or from this e-mail address may be stored on the 
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

---

From: "Heidi Ou (hou)" <[email-masked]>
Date: Mon, 22 Jul 2013 19:14:24 +0000

Hi Bharat,

Please search for [heidi2] for reply. I will also reply to your comments
on draft-ietf-pim-drlb. But
let's close this discussion first.

On 7/18/13 9:28 PM, "Bharat Joshi" <[email-masked]> wrote:

>I understand 'RPF procedure' but your explanation above is still not
>enough to explain what does 'RPF procedure over ECMP' means?
>
>My thinking was that this RFC has proposed a mechanism (ECMP redirect) to
>suggest downstream neighbors to select another upstream neighbor for its
>PIM join. I am not sure how it "improces" the RPF procedure over ECMP. It
>bascially direct the downstream neighbor to use another neighbor. Right?

[heidi2] It solves a problem that current mechanism, namely PIM Assert,
can't 
[heidi2] handle well. The potential use cases were discussed at the IETF
meetings
[heidi2] and capture in the proceedings and meeting notes

>IMHO, we are selecting upstream PIM neighbor. RPF is a method to do that.

[heidi2] We use "RPF procedure" to determine RPF neighbors. These are
[heidi2] fairly common and well accepted terminologies.

>I perfectly understand the reason for selecting two different upstream
>neighbors but that issue can be resolved by using a simple hash algorithm
>instead of adding this new PIM ECMP redirect message. In fact, I was
>looking for such a solution and when I found this RFC solves a different
>issue, I have submitted a very small draft to solve this. Please do
>review the draft at
>http://tools.ietf.org/html/draft-joshi-pim-ecmp-neighbor-select-00‎

[heidi2] We believe there are legitimate cases where downstream routers
[heidi2] would select different upstream routers, no matter what hash
[heidi2] algorithm they use. And that is why we believe the solution
[heidi2] should come from upstream router, not from downstream router.
[heidi2] And the implementation has been deployed.

[heidi2] I will comment on your draft once I study it carefully.

>Please bear with me but I still do not understand. How the upstream
>router will be able to find out if downstream router has an ECMP route
>for a specific source/RP?

[heidi2] The upstream router never needs to find out if a downstream
router 
[heidi2] has an ECMP route for a specific source/RP or not. This is not a
[heidi2] requirement. All the upstream router needs to know is that it
[heidi2] receives a PIM Join from an interface A that is part of an ECMP
[heidi2] bundle, and within that bundle, there is another interface B who
[heidi2] has previously received a PIM Join for the same flow.

>
>I sincerely feel that RFC is not clear enough on how this is done? In
>fact, I was later looking at AD and IESG comments and I see that Stewart
>had the same doubt. Please find his question submitted on '2012-06-20' at
>http://datatracker.ietf.org/doc/rfc6754/history/ Unfortunately, I could
>not find the author's response for his questions and so I asked here.

[heidi2] That discussion took place during the IESG review. We did
[heidi2] address the concerns and the reviewers were satisfied by the
[heidi2] answers. But I’m not sure how to retrieve the discussion
[heidi2] from IETF archives. Perhaps the chairs can help here.
[heidi2] Below, I’m including some emails that I happen to keep.
[heidi2] Hope it helps.

>>>
>>>
>>>
>>>===
>>>
>>> Section 2.1: From the statement "a PIM ECMP Redirect message is
>>>sent by
>>> an upstream router to notify downstream routers to redirect PIM
>>>Joins
>>> to the new RPF neighbor via a different interface. . When the
>>> downstream routers receive this message, they SHOULD trigger PIM
>>> Joins toward the new RPF neighbor specified in the packet."
>>>
>>>
>>> It is unclear how the upstream neighbor (preferred upstream router)
>>>determines the alternate neighbor.
>>>
>>[Yiqun Cai]
>>We can actually look at it from two ways.
>>
>>The draft doesn't intend to describe all the possible use cases and
>>what kind of decision a router must make in those situations before
>>sending out an ECMP Redirect. It is impractical to do the first and the
>>information will likely be obsoleted over time. And it does not make
>>much sense to do the second because 1) it is mostly proprietary and 2)
>>whether one follows the same logic or not, interoperability is not
>>affected. So we focus on the description of "bits and bytes", what they
>>mean and how a PIM router would react to that to make sure interoperable
>>implementations are possible.
>>
>>Having said that, the draft does hint a couple of examples how an
>>upstream router might use ECMP Redirect. For example, if the router is
>>forwarding out to ethernet1, it will "redirect" downstream routers
>>joining ethernet2 to "rejoin" ethernet1. It is not a very complicated
>>decision process in this case.
>OK, though it might be best if you explained that in the draft to stop
>others puzzling over the problem. It will only take a few lines.

Right. This was clearly explained in the Overview section, 4th
paragraph,

"When the RPF neighbor router receives the Join message and finds that
the receiving interface is one of the ECMP interfaces, it will check
if the same flow is already being forwarded out of another ECMP
interface. If so, this RPF neighbor router will send a PIM ECMP
Redirect message onto the interface the Join was received on. The
PIM ECMP Redirect message contains the address of the desired RPF
neighbor, an interface ID [RFC6395], along with other parameters used
as tie breakers."

>
>>
>>
>>>Please describe how the proposed algorithm ensures an upstream
>>>neighbor
>>>will be determined if the selection rules aren't consistent or known
>>>among
>>>PIM routers.
>>[Yiqun Cai]
>>This is determined by "preference" and "metric". The process is very
>>similar to the Assert process described by RFC4601. We can highlight
>>that in the draft if you think it will help.
>
>Please do that.

Will add one clarification to the end of sub-section "5.2 Receiving ECMP
Redirect",

"The tie break procedure is the same as that used in PIM Assert processing
described by [RFC4601]."

>It is not clear from the text in RFC that we are talking about two
>different LANs. In fact, when I looked through the mailing list, I found
>an e-mail from Adrain who has a similar issue. E-mail is at
>http://www.ietf.org/mail-archive/web/pim/current/msg02396.html.

[heidi2] We also replied that and stated it clearly in Overview

When the RPF neighbor router receives the Join message and finds that
the receiving interface is one of the ECMP interfaces, it will check
if the same flow is already being forwarded out of another ECMP
interface. 

[heidi2] The concept of ECMP bundle is also defined in the Terminology
section.

>
>Overall, IMHO I feel that the text in this RFC needs improvement to
>clearly define the network configuration it is aimed for. A figure
>explainging the same will be very helpful. I am not sure if it will be
>good to file an Errata to cleear this confusion or not. I will leave this
>to Working-Group Chair to decide.

[heidi2] Yes, it is up to the chair. But so far, I don't see any need yet.

[heidi2] Regards,
[heidi2] - Heidi

---

From: Bharat Joshi <[email-masked]>
Date: Tue, 23 Jul 2013 05:45:42 +0000

Hi Heidi,

      Thanks for your response. Please see few replies inline.

      Please bear with me as I am still trying to understand this RFC.

Regards,
Bharat

> let's close this discussion first.

Ok. Not a problem.
> [heidi2] and capture in the proceedings and meeting notes

I am not sure how it solves the PIM Assert problem. In fact, If my understanding is correct, I think the PIM assert problem is not there at all. Let us assume that there are two LANs. One downstream router sends PIM join on one LAN while another downstream router sends it on another LAN to the same upstream router. With this, there will be packets on both the LAN. But as they are two different LANs, PIM assert won't trigger. Right? Am I missing something here?

It would have been really good if we would have added some of the use cases and discussions in the RFC to explain various rationale.

> >IMHO, we are selecting upstream PIM neighbor. RPF is a method to do
> that.
> >Right?
> 

Ok.

> [heidi2] And the implementation has been deployed.

I think PIM assert was introduced to solve this problem. In a steady state when routing has converged, there are small chances that downstream routers will pick up different upstream routers. But this problem becomes acute when a ECMP route exists and there is no standard way to decide which upstream router to pick.

When a PIM join is received by upstream router, it cannot tell whether it was chosen based on a single-gateway route or a ECMP route. So in my opinion, it is difficult for upstream router to decide whether it should ask the downstream router to choose another upstream router or not.

So in my opinion downstream routers should figure out the same upstream neighbor when using a ECMP route using a standard algorithm. This does not solve the problem completely. The problem still remains because another downstream router can choose a different upstream router because of errorneous implementation, not-using-standard-algorith, not-having-ECMP-route etc.. This brings us back to the need of PIM assert.

If we want to solve this problem completely, we need changes in PIM protocol itself.

> [heidi2] I will comment on your draft once I study it carefully.

Its a pretty small draft. Please do read and send your comments.

> if

I am sorry that I assumed few things here. I went back to the RFC and read the definition of ECMP bundle once more.

It clearly says that it is configured by the administrator and it must have the same route metric. Now I understand how upstream router can figure this out.

Thanks for your patience.

> [heidi2] Below, I’m including some emails that I happen to keep.

Ok. Thanks for posting the discussions here.

> if the same flow is already being forwarded out of another ECMP

IMHO, it is not clear enough. 'ECMP interfaces' does not give the indication that we are talking about those interfaces that are connecting the same devices in two different broadcast domain and has the same metric.

> [heidi2] The concept of ECMP bundle is also defined in the Terminology

Yes. I again read through the same and understand what you are suggesting.
 
> [heidi2] Yes, it is up to the chair. But so far, I don't see any need

Please note that I am not saying there is anything wrong here. What I am proposing is that if there is a way to make it more readable and lucid for people to follow. It may be something to do with me also.

Once again thanks for your replies.

Regards,
Bharat
 
> you
> >are not
> other
> >person and
> carry
> >out your

---

From: "Heidi Ou (hou)" <[email-masked]>
Date: Wed, 31 Jul 2013 21:35:23 +0000

Hi Bharat,

My suggestions are at [heidi3]

On 7/22/13 10:45 PM, "Bharat Joshi" <[email-masked]> wrote:

>I am not sure how it solves the PIM Assert problem. In fact, If my
>understanding is correct, I think the PIM assert problem is not there at
>all. Let us assume that there are two LANs. One downstream router sends
>PIM join on one LAN while another downstream router sends it on another
>LAN to the same upstream router. With this, there will be packets on both
>the LAN. But as they are two different LANs, PIM assert won't trigger.
>Right? Am I missing something here?

[heidi3] You are not missing anything here.  This is the problem we want
to 
[heidi3] solve. Since PIM Assert is only used within a single LAN, it
[heidi3] cannot be used.  A different mechanism is needed.

>
>It would have been really good if we would have added some of the use
>cases and discussions in the RFC to explain various rationale.

[heidi3] We wanted to have the initial internet-draft and the
[heidi3] RFC focused on the protocol specification to help produce
[heidi3] interoperable implementations. So we made an attempt to
[heidi3] avoid getting into detailed description of use cases. We
[heidi3] believed that how people would use the protocol improvement
[heidi3] to build networking functions is out of the scope. This is
[heidi3] consistent with other protocol specifications. For example,
[heidi3] RFC4601 doesn't describe how one would use PIM-SM.

>I think PIM assert was introduced to solve this problem. In a steady
>state when routing has converged, there are small chances that downstream
>routers will pick up different upstream routers. But this problem becomes
>acute when a ECMP route exists and there is no standard way to decide
>which upstream router to pick.
>
>When a PIM join is received by upstream router, it cannot tell whether it
>was chosen based on a single-gateway route or a ECMP route. So in my
>opinion, it is difficult for upstream router to decide whether it should
>ask the downstream router to choose another upstream router or not.
>
>So in my opinion downstream routers should figure out the same upstream
>neighbor when using a ECMP route using a standard algorithm. This does
>not solve the problem completely. The problem still remains because
>another downstream router can choose a different upstream router because
>of errorneous implementation, not-using-standard-algorith,
>not-having-ECMP-route etc.. This brings us back to the need of PIM assert.
>

[heidi3] We believe incremental modification is possible.

>I am sorry that I assumed few things here. I went back to the RFC and
>read the definition of ECMP bundle once more.
>
>It clearly says that it is configured by the administrator and it must
>have the same route metric. Now I understand how upstream router can
>figure this out.
>
>IMHO, it is not clear enough. 'ECMP interfaces' does not give the
>indication that we are talking about those interfaces that are connecting
>the same devices in two different broadcast domain and has the same
>metric.
>Please note that I am not saying there is anything wrong here. What I am
>proposing is that if there is a way to make it more readable and lucid
>for people to follow. It may be something to do with me also.
>

[heidi3] thank you for your comment. I hope I have provided
[heidi3] sufficient explanation to clarify any doubt you have.

thanks
- Heidi

---

From: Bharat Joshi <[email-masked]>
Date: Thu, 1 Aug 2013 05:07:27 +0000

Hi Heidi,

      Thanks for your reply. Please see few responses in line.

Regards,
Bharat

> >> >My thinking was that this RFC has proposed a mechanism (ECMP
> redirect)
> >> to
> >> >PIM join. I am not sure how it "improces" the RPF procedure over
> ECMP.
> >> It

I am not sure if I am able to convey my thinking correctly or not. What I was trying to say is that it would have been easier to follow if a simple use case with some figure would have been available in the RFC.

But now that it is already a RFC, I am not sure if you can do much here even if you agree with me.

> >> >>>routers to select either the same or different RPF neighbor
> routers
> >> for
> >> >>>the same traffic flows.
> >> >>>
> >> >>[heidi] algorithms used and/or policies configured, two routers
> might
> >> >>very
> >> >>well choose

Ok.
 
> >> >>[heidi] If the router sees Join on an interface, and the interface
> is
> >> >>one
> >> [heidi2] has an ECMP route for a specific source/RP or not. This is
> not
> >> a
> >> >had the same doubt. Please find his question submitted on
> '2012-06-20'
> >> at
> >> >>>      Also, it is not clear how the upstream route can check if
> the
> >> >>same

Yes. Your explanation have surely helped in clearing my doubts. Thanks a lot.