Subject: RE: [Ipoverib] IPoIB multicast forwarding

From: "Larsen, Roy K" <[email-masked]>
Date: Mon, 15 Apr 2002 15:26:54 -0700

> From: Vivek Kashyap [mailto:[email-masked]]
> Sent: Monday, April 15, 2002 1:25 PM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> 
> 
> > 
> > Hi Vivek,
> > 
> > After reading through your response, I realized that there 
> is one more
> > case to consider: 
> > 
> > Sender does a check for IB Mcast Group...check succeeds because a
> > receiver has created the group.
> > 
> > Now, the receiver decides to leave the group and the IB MGroup gets
> > deleted.
> > 
> > The Sender needs detect this and revert back to sending the 
> multicast
> > packets to the All-router Address.
> > 
> > I think in the scheme we are discussing, the sender will 
> always have to
> > check for the existence of the IB Group just because of the 
> edge cases.
> 
>         I have made some assumptions on IB deletion. Let us assume
>         that the sender has joined the group (I am in the group that
>         believes for IB to work right all have to join - no-join-send
>         does not work universally).
> 
>         The receiver will not delete the group just leave. The
>         sender's 'check' really is 'join'. The sender is a member of
>         the group - the group cannot really be deleted until all
>         members leave. The sender can age its entries of the groups it
>         has joined and after certain idle time 'leave' the group.
> 
>         Also, if the receiver goes through the process, it will first
>         have sent a report to the router. THe router is also a member
>         of the group.
> 
>         We can also have it that the last guy leaving the group delete
>         it. IF someone joins it just as we are deleting it then the
>         deletion should fail if the membership count is greater than
>         one.

Vivek,

What mechanism do you propose to use that will let a node know, when it
leaves, that it is the last one in the group and thus responsible for
deleting it?

Roy

> 
>         So, if the sender has joined the group, I don't think that it
>         will have to check the group everytime it sends a packet.
> 
> 
> Vivek
> 
> > 
> > Aditya Dube
> > 
> > -----Original Message-----
> > From: Vivek Kashyap [mailto:[email-masked]] 
> > Sent: Monday, April 15, 2002 11:39 AM
> > To: Aditya Dube
> > Cc: [email-masked]
> > Subject: Re: [Ipoverib] IPoIB multicast forwarding
> > 
> > > 
> > > 
> > > Vivek
> > > 
> > > As the sender will only send packets to the all-router MGID after
> > doing
> > > the check, how will a receiver who creates the MGroup on the same
> > subnet
> > > as the sender ever receive the multicast packets that it 
> should get?
> > 
> > 
> >         The suggested solution is as follows. The basic idea is
> >         certainly workable. Hopefully, after some discussion we can
> >         refine it to cover all the issues that crop up.
> > 
> > 
> > 	Sender:
> > 		got IP multicast packet to send
> > 			determine the corresponding IB group
> > 			if already 'IB_joined' or 'recorded'
> > 				IB_send to it
> > 
> > 			else if IB group exists
> > 				IB_join/record MGID locally
> > 				IB_send to it
> > 			else
> > 				
> > 				IB_send to all-routers MGID
> > 
> > 	
> > 	Receiver:
> > 		want to receive IP multicast beloging to an IP group 
> > 			check for corresponding IB group
> > 			if IB group exists 
> > 				IB_join it
> > 				IB_send report to group
> > 			else
> > 				IB_create IB group
> > 				IB_send report to all-routers MGID
> > 				(IB_send report to group)
> > 
> > 	
> > 	Router:
> > 		always receive on broadcast/all-hosts and all-routers
> > 		if IGMP/MLD indicates new group
> > 			IB_join corresponding IB group
> > 		if routing protocol indicates new group
> > 			IB_create corresponding IB group 
> > 
> > 
> >         Now, a sender must check for the IB group everytime it is
> > sending
> >         to an IB group it doesn't know about (no prior 
> IB_join/record
> >         with it). But once the group exists and the sender has a
> >         packet to send, it will detect the presence and join the
> >         group. From then on it will always send to that MGID. The
> >         receiver will get the packets since it (or another receiver)
> >         created the IB group.
> > 
> >         In other situation where the router determines that 
> there are
> >         external receivers it creates the group too. The sender will
> >         therefore find this and join the group.
> > 
> >         The sender needs to continuously check for the IB group only
> >         in case is where the router doesn't need to forward 
> the packet
> >         and there are no receivers in the local subnet. I 
> don't think
> >         this will be too often. Is this case of concern ?
> > 
> > 
> > 
> > Vivek
> > 
> > <snip>
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> [email-masked] 
> [email-masked] 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vivek Kashyap <[email-masked]>
Date: Mon, 15 Apr 2002 16:31:33 -0700 (PDT)

> leaves, that it is the last one in the group and thus responsible for

        There are a few possible solutions; all requiring IB
        clarifications. Any alternate solutions are welcome.

	a) No delete is done by the IPoIB interfaces, it is an SM issue

        This I took from the IB architecture document in chapter 3. It
        talks of the SM deleting unused groups. ( Now, there are a few
        who say that this is not going to be the case -- in which case
        a different solution is needed.)

        The receiver leaves when it no longer wants to receive. The
        router when there are no more receivers. The sender after some
        idle time.

        b) Any entity wanting to leave an IB group attempts to delete
        it. If there are members in the group then the deletion should
        fail. The last one will automatically delete the group.

	[Depends on delete semantics]

        c) Leave is fine. No need to bother about delete.
	
        let the group entries stay around in the database. They don't
        bother anyone. The admin can periodically, once in a few
        months may be, delete unused ones. The entries will get reused
        if others join. The multicast tree is affected by joins and
        leaves and not by deletes.

	d) {Don't know about this }

        The setup is such that if the last member leaves the group the
        group is automatically removed.

Vivek

---

From: Michael Krause <[email-masked]>
Date: Mon, 15 Apr 2002 17:57:34 -0700

>         router when there are no more receivers. The sender after some

The specification states that a multicast management entity not the SM is 
responsible for multicast group management.  This entity implement IB 
multicast group management and could also be responsible for resolving IP 
multicast addressing to IB multicast groups.  It may also be responsible 
for informing / updating participating IB routers of any changes (and vice 
versa for the IB routers informing the entity of changes within its view of 
the router cloud).  This entity could also be used to provide a similar 
service for CAs that act as multi-protocol router endnodes.

>         [Depends on delete semantics]

If an endnode wants to leave an IB multicast group, it informs the 
management entity of this event and the management entity working with the 
SM removes this endnode from the IB multicast tree.  If the entity 
determines that there are no longer any endnodes participating in the 
multicast group, it may delete the entire group from the fabric.  This does 
not require the endnode to delete the group and deal with any failure 
semantics.  An endnode should simply delete itself and let the management 
entity deal with the problem.

>         leaves and not by deletes.

Delete should be performed automatically by the management entity; this is 
a policy decision that has no bearing on the IP over IB specification.

>         group is automatically removed.

This is how IB specifies it should work.

Mike

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 16 Apr 2002 10:58:43 -0700 (PDT)

Great! So the deletion of the group is not a difficult issue at all.
The IPoIB nodes only need to leave the group.

        The receiver leaves when it doesn't need to receive any more.
        Router when there are no receivers and it knows it doesn't
        need to forward. sender after some idle time i.e. no activity
        on the group. My suggestion is to have a minimum timeout of 2
        minutes - avoids group join/leave happening too often.

Vivek

---

From: Bill Strahm <[email-masked]>
Date: Tue, 16 Apr 2002 16:02:19 -0700

I don't think it is that easy... How will the router know to leave ? It can't
be when there are no receivers, because it has to forward any traffic from
the senders to other locations on the network...  This is especially true for
flood/prune algorithms (see PIM-DM)

I'm also not sure that this would work in an intermitant sender environment.

I may have a sender that only wants to multicast trigger events, lets say
an SNMP TRAP message that happens once in a blue moon...  This means that
there is the overhead of group creating and deletion with every send (and
you still have the problem with the router not being on the line)

I am very tempted to do some deep thinking about having ALL multicast packets
go to a default multicast traffic IB address, and have the IP portion echo
the packets back to the IB subnet on the correct IB MC Group if there are 
konwn subscribers (through IGMP etc) and forward on to other ports depending
on the IP multicast forwarding tree...

This means that all locally sent MC traffic goes onto the fabric twice, once
to get to the router on a "All multicast" traffic channel, and once to the
correct multicast channel...

Does IB provide a mechanism to allow the Router to detect multicast group
creations... if it does, then we can use the existance of the group rather
than needing IGMP, I don't think this has been defined in IBTA fully however

Bill

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 16 Apr 2002 16:39:25 -0700 (PDT)

> flood/prune algorithms (see PIM-DM)

        That is not a problem. As I noted earlier a router will join
        the groups on getting a notification that there are receivers.
        When no receivers are present it can choose to leave. 
	
        If it is forwarding the packets then it creates the IB group
        and when it thinks it no longer needs to forward the packets,
        it leaves the group.

> you still have the problem with the router not being on the line)

        If it is once in a blue-moon the overhead also occurs once in
        a blue-moon; that should not be too much of trouble.

        In general you join the group when you need to send the snmp
        trap. If you join the group and you know that you need to stay
        with this group you can choose not to leave the IB group at
        all. I have suggested the sender leaving the group after some
        idle time - the delay is suggested to be a minimum of 2
        minutes to avoid the create/delete operation to occur too
        often. You can always set the timer to be longer.

> correct multicast channel...

        So you are also asking the router to join the IB group based
        on IGMP information. So, in that case why not just let the
        router join the group, and have the sender send directly to
        the IB group? I don't think the load on the sender is all that
        much that the router(which router? if there are multiple)
        needs to be loaded with the task of resending the packet.

        Deliberately sending duplicate multicast packets on the fabric
        does not seem like a good idea.

> than needing IGMP, I don't think this has been defined in IBTA fully however

        Believe me - I would have used an existing IB method if I
        could find one. There is not one and hence I have suggested
        the solution I have.

        I have already received a request, and it is a very valid one,
        to put the overall algorithm for senders/routers/receivers
        together in one draft and post it here. I'll do that in the
        next couple of days. That way we can evaluate the overall
        solution in one go and see what is missing.

Vivek

---

From: Michael Krause <[email-masked]>
Date: Tue, 16 Apr 2002 13:23:11 -0700

>         minutes - avoids group join/leave happening too often.

I don't believe this is an IP over IB issue - the policy is implemented 
within the management entity which is an IB focused agent not an IP 
agent.  A CA acting as a multiprotocol router could continue to forward 
packets to the IB switches which would simply drop the packets as there is 
no place to route.  An agent providing performance monitoring would note 
the drops and determine whether they should be terminate or not.  It would 
contact the multicast entity who would inform the CA that the following IB 
multicast group no longer exists and that CA-specific software would then 
update its "routing" tables to no longer forward packets into the IB subnet.

Mike