Subject: [Ipoverib] IPoIB multicast

From: Vivek Kashyap <[email-masked]>
Date: Thu, 18 Apr 2002 11:15:32 -0700 (PDT)

Aditya,

Below is the overall algorithm since you had asked me to submit it in
one writeup. However, in IBTA we are working on some ideas (I
participated in one such meeting today morning) that will make all
this a lot simpler.

The following is assuming IB multicast as defined at present.
There is no notification in IB that can be sent to IB nodes on group
creation or deletion. As per Mike's mail the IB group gets deleted
when the last member leaves.

Note: In this discussion I am referring to IP routers, receivers and
senders. 

The idea below is to let the router(an IP router) know that a group
has been created (by the receivers) or have the routers create the
groups when they need to forward. This is accomplished by the routers
acting on the IP group creation reports(IGMP/MLD) or the routing
protocol updates.

A sender can send at any time. It therefore attempts to join the group
when it has some packet ot send and retries if it cannot join the
first time.

The receivers leave the IB groups when they are not interested in the
associated IP group. The routers leave when they leave the associated
IP group or determine that they no longer have to forward the
associated IP group packets.

Senders leave the groups when the group has not been used for some
time. 

	Receiver:

        IP receivers must join a specific local interface and send a
        report to the router.

		IP_ADD_MEMBERSHIP called
			check for corresponding IB group
			if IB group exists 
				IB_join it
				IB_send report to group
			else
				IB_create IB group
                                IB_send report to all-routers MGID or
                                could send it to all-hosts group

		IP_DROP_MEMBERSHIP called
			(Send leave report to router )
			IB_leave the IB group

	Router:
		always receive on broadcast/all-hosts and all-routers
		if IGMP/MLD indicates new group
			IB_join corresponding IB group
		if routing protocol indicates new group
			IB_create corresponding IB group 
		if report informs of leave
			IB_leave corresponding IB group
		if routing protocol indicates no need to forward
			IB_leave corresponding IB Group

	Sender:
                IP senders can send at any time. They don't have to
                join at an interface or report to an IP router.

		got IP multicast packet to send
			map to the corresponding IB group
			if already 'IB_joined' the IB_group
				IB_send to it
				reset timer

			else if IB group exists
				IB_join the group
				start a timer
				IB_send to it
			else
				
				IB_send to all-routers MGID ....(c)

		if timer goes off
			IB_leave the IB group

        We can modify (c) above to 'send to all-hosts MGID'. That way
        the timer can be set to a longer value and the sender
        algorithm can become:

		got IP multicast packet to send
			map to the coresponding IB group
			if already IB_joined
				IB_send to it
				reset timer
			else 
				IB_send to all-hosts MGID
				if no timer associated
					create a timer

		if  timer goes off
			if IB_joined
				IB_leave the IB group
			else
				try to IB_join
				restart timer
			
Vivek
				
-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "Aditya Dube" <[email-masked]>
Date: Fri, 19 Apr 2002 10:51:32 -0700

Vivek

Thanks for sending this out. This definitely makes it much easier for me
to analyze the proposed solution.

The Router algorithm has a problem:

The First leave will cause the Router to leave the group regardless of
whether there are other nodes that have done an IB_Join.

The other thing that I have recently learnt is that Profile B hardware
requires a node to support a minimum of 16 multicast groups. Each join
is therefore a valuable resource.

Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:[email-masked]] 
Sent: Thursday, April 18, 2002 11:16 AM
To: [email-masked]
Subject: [Ipoverib] IPoIB multicast

Aditya,

	Receiver:

		IP_ADD_MEMBERSHIP called

	Router:

	Sender:

			else if IB group exists
				IB_join the group
				start a timer
				IB_send to it
			else

		if timer goes off
			IB_leave the IB group

		if  timer goes off
			if IB_joined
				IB_leave the IB group
			else
				try to IB_join
				restart timer
			
Vivek

---

From: "Vivek Kashyap" <[email-masked]>
Date: Fri, 19 Apr 2002 11:35:59 -0700

Vivek

<VK>
Yes, I was careless in stating this. I certainly didn't mean that one
leaves at the first report but was listing conditionals for leaving. I am
not looking into it anymore because of the changes being contemplated in
IBTA; and it also the reason for the delete case not being fully fleshed
out. I guess if we allow the router to leave the IB group (due to no
receivers and no need to forward) we also need to ensure that the senders
leave the groups. It works ok with my earlier comment on check for group
before every send(that could be quite inefficient though). Now one would
need to check for group existence periodically or have the router set a
timer when there are no receivers or need to forward on an MGID and leave
after some idle time.

The algorithms provide an outline that we can revisit (along with the
corrections as discussed above) and see how the conditions are met with the
IBA changes.

<VK>

<VK>
      Isn't profile B just a list of characteristics that are needed. As
far as I know it doesn't reflect the actual products in the market. Do you
know what is the number advertised by various vendors ?

      If it is number of MLIDs - one can always overload them. If it is
number of MGIDs at the CA then we need to consider the issue - it is not
limited to IPoIB multicast though. It is an IB multicast issue. We might
have to just say, use adequately provisioned CAs just like we state that
one must use multicast aware CAs or CAs which support a minimum of 2K MTU
etc.

Vivek
<VK>

Aditya Dube

-----Original Message-----
From: Vivek Kashyap [mailto:[email-masked]]
Sent: Thursday, April 18, 2002 11:16 AM
To: [email-masked]
Subject: [Ipoverib] IPoIB multicast

Aditya,

             Receiver:

                         IP_ADD_MEMBERSHIP called

             Router:
                         always receive on broadcast/all-hosts and
all-routers

             Sender:

                                     else if IB group exists
                                                 IB_join the group
                                                 start a timer
                                                 IB_send to it
                                     else

                                                 IB_send to all-routers
MGID ....(c)

                         if timer goes off
                                     IB_leave the IB group

                         if  timer goes off
                                     if IB_joined
                                                 IB_leave the IB group
                                     else
                                                 try to IB_join
                                                 restart timer

Vivek

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Mon, 22 Apr 2002 11:39:32 -0700 (PDT)

>At 01:45 AM 4/22/2002 -0700, H.K. Jerry Chu wrote:
>>The proposal I made in the link-multicast draft when the IB MC
>>group corresponding to the target IP MC address doesn't already
>>exist, is to have the interface driver forward the pkt to the all-
>>router MC group. If the MC group comes into existence later
>>(i.e. someone joins as a member), the driver needs to be notified
>>so it can forward pkts back to the right MC group. This requires
>>some support from SA. I believe a proposal has been made recently
>>in IBTA to add this support to SM/SA.
>
>To make it quite clear - one cannot send on IB without providing a MLID 
>that has been configured into switches.  It does not matter whether this is
>for an all routers or a specific multicast group - this level of 
>configuration must be performed.

I'm confused. What part of my proposal implies sending to a non-existing
MGID/MLID? The all-router MC group is required by IP and thus will
always exist (with a valid MLID).

>Hence, trying to target an all routers 
>multicast group provides zero benefit since the same work needs to be 
>performed for both types so why not do it right the first time.

I think you're assuming the only way a MLID will get configured correctly
for a sender to use is to require the sender to also be a member. In that
case yes the above algorithm degenerates itself into broadcast if every
potential MC sender must also become a member of the all-router MC group.
It defeats the purpose of the latter.

But I have also heard different opinions regarding the join-to-send
issue. My original proposal was based on the current 1.0 spec. that
doesn't require a MC sender to join before send.

Has IBTA come to a consensus on the join-to-send issue yet?

>This is 
>why I am strongly opposed to the use of such a technique and view this as 
>unnecessary complexity.
>
>I've stated the simplest algorithm:
>
>- Join the IB multicast group
>- Get the MLID configured
>- Send the packet
>- If any one is listening, they will receive it
>- If they are not listening at the time of the send, then they will get a 
>subsequent send but no attempt should be made to turn this into a reliable 
>service as suggested in some e-mails.

Agreed, as long as whatever race condition there may be doesn't turn
into a persistent one.

>- Group changes are tracked by the SA
>- The SA will inform a group when it is no longer available
>
>The new IBTA method will be a form of async notification service that will 
>allow endnodes to detect when groups are added / deleted which is something 
>a couple of us have proposed for quite some time and is now finding 
>traction.  If the workgroup focuses on the join process and recognizes that 
>a notification method will exist (which is entirely an IB issue with an 
>implementation's LLP masking any of this complexity - this is clearly not 
>an IP over IB issue), then we can complete this set of specifications.

If IBA can successfully hide the complexity of IB MC group management
from the IP stack so that to the latter, every link multicast group
exist and can be readily used (sorta like Ethernet), then I agree we don't
have to muck with IP stack and IPoIB spec. But this is a big IF. I don't
believe this is the case in the 1.0a of IBA spec. We need a clear
statement from IBTA on the direction it will take.

>
>>2. Ethernet NICs can be programmed into promiscuous multicast
>>mode when the NIC will send a copy of every MC pkt to IP.
>>IGMP/MLD uses this feature conveniently to pick up a copy of
>>every MC pkt to track MC membership, and to route MC pkts
>>correctly to other subnets.
>>
>>Since IB doesn't support a promiscuous multicast mode. The
>>workaround I proposed is to have the interface driver make a copy
>>of every MC pkt and send it to the all-router MC group. This is
>>pretty straightfoward with a small performance penalty. But is
>>still much more efficient than resorting to broadcast. There is
>>a small problem in case the router itself also joins a MC group
>>and will then receive two copies. This case can be adressed with
>>some more check in the interface driver of a router but since
>>the case is rare, I'm not sure it's worth the effort and additional
>>complexity.
>
>IB can be configured to see all multicast packets within a given partition 
>- this requires one to join all multicast groups.  This is a policy 
>decision.  The fact that it is different than Ethernet is irrelevant quite 
>frankly.  This is an IP over IB not IP over Ethernet workgroup.

If one can use the existing IBA spec. to emulate promisucous MC mode
from any CA attachment point, then we've been wasting time solving
a non-existing problem. I don't think that has been the case.

Like I said in my previous mail, like it or not, much of the original
IP design was done on Ethernet so was influenced by it. Both IGMP and
MLD are supposed to be link agnostic. But both assume a promisuous MC
mode, which exists on Ethernet but not on many other link types. This
is just reality.

>What you 
>are describing is in many cases really no different than the class of 
>problems associated with LANE.
>
>>Other alternative includes Vivek's first proposal to snoop IGMP/MLD
>>pkts in the interface driver. This requires driver to understand
>>IGMP/MLD, and will make it very difficult to maintain (with newer
>>version of IGMPv1,2,3.../MLD).
>>
>>Vivek's recent proposal fixed the above problem by moving the work
>>from the driver to IGMP/MLD themselves. Unfortunately IMO the
>>chance of requesting link specific changes in a link-independent
>>module like IGMP is not very good.
>
>There is no reason to add this complexity at all which is why I continue to 
>oppose this line of logic.

Agreed.

>
>>Perhaps if IBTA can supply us an equivalent of a promiscuous
>>multicast mode without bending IBA backwards too much that would
>>be ideal.  I've heard some discussion along that line is going on.
>>Let's see what comes out of it.
>
>One has an promiscuous solution by employing the notification method and 
>performing a join operation for all multicast groups associated with a 
>particular partition.

Good. I'll be looking forward to a concrete solution by IBTA then.

Jerry

>Mike