Subject: RE: [Ipoverib] IPoIB multicast forwarding

From: "Culley, Paul" <[email-masked]>
Date: Tue, 9 Apr 2002 08:53:57 -0500

I suspect that the problem can be made simpler; IP routers should always IB_join ALL IPoIB Multicast groups for a given interface.  The router will simply ask to join any groups in the same partition(s) it is assigned to.  Whether the IP router forwards the multicast packets will then depend on the IP routing protocol.

IB routers have the same problem, and may use the same solution...

Paul R. Culley
Compaq Sr. Fellow
281-514-5543

-----Original Message-----
From: Vivek Kashyap [mailto:[email-masked]]
Sent: Tuesday, April 09, 2002 3:31 AM
To: [email-masked]
Cc: [email-masked]
Subject: [Ipoverib] IPoIB multicast forwarding

Folks,

        We have defined a mapping from the IP addresses to the IB
        multicast groups. Use of different groups causes minimal
        replication at the switches and less load at the receivers.

        But there is a quirk in the way the groups are setup in
        InfiniBand - there is no way for any of the nodes to know if
        another has created an InfiniBand group. This is easy to fix
        in most cases - one checks with the SM. But it doesn't work
        out well for IPoIB packet forwarding.

        The router needs to forward the multicast packets beyond the
        local IP subnet if it knows there are recipients for it or in
        the case of some routing protocols it must do that unless it
        gets an update stating not to forward. Similarly, it needs to
        forward external packets to the local receivers if there are
        any receiver present. 

        Thus if there are IB multicast groups associated with an IP
        multicast group the router must know about them or the router
        must receive all multicast packets in the subnet via some
        other construct.

        Below are the three alternatives that I mentioned at the
        53rd IETF meet. 
	
        i) Define only one IB multicast group i.e. IB broadcast MGID.
        All IP multicast packets use this group.

        ii) The multicast IP packet is sent to both the associated IB
        multicast group and the IB mutlicast group corresponding to
        the all-IP routers IP group.

        iii) Sender always sends to corresonding IB group if it exists
        else it sends to the broadcast group.

             IP receivers create IB groups and send the IGMP/MLD
             groups on the broadcast MGID.

             Routers join relevant groups based on the IGMP report
             and routing protocol updates.

        I also put this problem to Bill Fenner (AD of routing area)
        and asked his advice on the alternatives. Of the three options
        he favored option (iii) above.

        Option (i) doesn't take advantage fo IB multicast at all and
        so should be looked at as a last resort solution only.

        Option (ii) requires the sender to send two copies of the same
        packet and 'deliberately' increases the chance of duplicates
        spreading across the network.

Suggested solution  in detail:

        1. Pure senders - at IB level check for the relevant IB group.
           If it exists then send to the IB group else send to the
           'all-routers' IB group. If there is no IB group
           corresponding to a particular IP group then it implies that
           there are no IP receivers on the subnet for that IP
           multicast group. 

           Note: a) Pure senders don't create IB groups. This is
           because there is no existing mechanism for the senders to
           let the routers know of the new groups.

        2. IP receivers: As per IGMP/MLD the first packet sent by a
           host on joining a multicast group for receiving
           (IP_ADD_MEMBERSHIP) is the multicast report.

	   The hosts, as a result of IP_ADD_MEMBERSHIP, do the following:

	   	check if the corresponding IB group exists. 

		if (IB group exists)
			IB_join the group and send the IGMP/MLD report

		else
			IB_Create the group
			IB_join the group
                        Send IGMP/MLD report to IB group corresponding
                        to all-routers (all-hosts or broadcast) IP
                        group (Note: only done when creating the IB
                        group) 

	3. IP routers IB_join MGIDs on:

		a) Finding a new IP group on the subnet in the IGMP/MLD report
                b) Determining a need to forward IP multicast packets
                   through a routing protocol

Comments ?

Vivek
			
-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Bill Strahm <[email-masked]>
Date: Tue, 9 Apr 2002 11:05:47 -0700

Ok, I am going to express a scary amount of ignorance here...

I know how multicast works on receive, I know how IGMP works to build the forwarding tree,
I know how flood/prune, RP Multicast Routing protocols work...

What I would like a pointer to is a simple this is how a first hop multicast router 
behaves.  An example on Ethernet would be acceptable, a generic example would be better.

The question I have is as a Multicast sender, I send a multicast packet to the first
hop router that is multicast aware.  Now, do I send that packet to its unicast MAC 
address, or do I send that to the mapped layer 2 broadcast address and let the IP 
layer rebroadcast to the broadcast address if it is needed on the local subnet.

Once I have that, everything else I understand

Bill
PS Bill Fenner, the IPoIB list is subscriber posting only.  If you haven't subscribed, please do 
so we can get the benefit of your wisdom...

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 9 Apr 2002 11:05:43 -0700 (PDT)

> I suspect that the problem can be made simpler; IP routers should always IB_join ALL IPoIB Multicast groups for a given interface.  The router will simply ask to join any groups in the same partiti…

That does not seem workable. How does the router know which groups to
join ? Note that the groups are not static - new ones can always be
created and the old ones deleted.

In my solution the IGMP/MLD report which tells the router about IP
joins is used to trigger the corresponding IB join. The same is the
case with routing protocols.

Vivek

---

From: Bill Strahm <[email-masked]>
Date: Tue, 9 Apr 2002 11:40:37 -0700

Question then...

As an IP Multicast sender, I am not required to join 
the Multicast group... So now there has not been an
IGMP message telling the first hop router to join the
group either.  So the router has not told its layer
2 to join the broadcast address for the multicast group.

How does the first hop Multicast router know to join
this group and forward the traffic on to the multicast
tree (either RP, or Prune/Flood) if it can't receive
the initial packet...

This is the magic that I was wanting to know, I know how
I would make it work, but I am hoping Bill F. can tell me
the correct way to do it <G>

Bill Strahm

On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> Bill,
>   When you send in the Multicast packet (as a sender),
> at layer 3 you send it to the multicast IP address
> that the sender decided on. Your layer 2 module will
> decide on the next hop for that Multicast IP address.
> And a similar functioning will be build up as a chain.
> Note in this pacth while the senders original packet
> is moving to its destination which is the IP multicast
> address, the intermediate routers (which are multicast
> aware) will check to see if they have any IGMP joins
> they have received from any other subnets. If they
> have any such mapping they have built due to receive
> of IGMP joins then they will oblige that join by
> copying the packet to that joined destination. That's
> how the copy of the packet is enroute to its rightful
> receiver.
> 
> SG
> 
> P.S :: Dr Fenner is an authority on this and can
> provide some special comments 
> 
> 
> --- Bill Strahm <[email-masked]> wrote:
> > behaves.  An example on Ethernet would be
> > acceptable, a generic example would be better.
> > 
> > PS Bill Fenner, the IPoIB list is subscriber posting
> > only.  If you haven't subscribed, please do 
> > > I suspect that the problem can be made simpler; IP
> > routers should always IB_join ALL IPoIB Multicast
> > groups for a given interface.  The router will
> > simply ask to join any groups in the same
> > partition(s) it is assigned to.  Whether the IP
> > router forwards the multicast packets will then
> > depend on the IP routing protocol.
> > > 
> === message truncated ===
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 9 Apr 2002 12:02:39 -0700 (PDT)

> 
> The problem IP multicast joining/leaving is performed
> optimally becuase the routers implement the IGMP.
> For IPoIB have an optimal implementation of multicast,
> the IB node intending to join the multicast needs to
> express the IB multicast group ID to the appropriate
> IP multicast group ID.

        In this WG we have defined a mapping from the IP multicast
        addresses to IB multicast addresses. When I say
        'corresponding' or 'relevant' IB group I mean the IB mutlciast
        GID mapped to from the IP multicast address.

	Eg. 224.0.0.1 is mapped to FF12:<401B>:<P_Key>::1

>   On a similar account the originator of the IB
> multicast data should create a mapping of IB Multicast
> groupID to IP multicast groupID. Otherwise if a
> blanket link would flood the IP network with unwanted
> packets, since the forwarder would not know better and
> hence would take the most pessimitic approach an
> forward any and every multicast data.

        The reason the originator of the data is not being asked to
        create the group is because it cannot tell the IP router about
        the corresponding IB group the router needs to join. We cannot
        have the router parsing every mutlicast packet received to
        determine if it is a new group to IB_join.

        In any case if there are no IP receivers for the IP group then
        the packet will be either dropped by the router (after
        receiving it on the 'MGID corresponding to the all-routers IP
        address') or forwarded further. The IP router may receive
        pruning updates to remove the IP group and in that case it
        will drop any future packets on that IP address.

	Thus there is no flooding as you fear. 

        Things at the IP layer work exactly as in any other media. For
        IB the only aspect that is new is:

                IP receivers create corresponding IB groups. They also
                ensure a copy of the MLD/IGMP report is sent out on
                the *IB MGID corresponding to all-hosts IP address*

                IP routers join the IB group corresponding to the IP
                groups listed in the MLD/IGMP report. They may
                optionally choose to create and join IB groups based
                on IP groups they expect to forward out of the local
                subnet. 

> 
>   In order to not have any major infrastucture rework
> to get IPoIB and its multicast to work this needs to
> be done.
>   
>    Vivek, I read your three proposals and saw the
> commanility of sticking to onely one IB multicast
> group ID. Whay is that ? That means simultaneous IB
> multicast of data cannot be facilitated. Did I get
> that right?

         No I am not suggesting sticking to one IB group ID. That is
         option (i) only - and I don't support that option.

Vivek

> 
> SG
> 
> 
> --- Vivek Kashyap <[email-masked]> wrote:
> > >

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 9 Apr 2002 12:37:12 -0700 (PDT)

Bill,
        The multicast router does not 'join' any group at the IP
        level. The router sets the ethernet interface in all-multicast
        promiscuous mode. Thus the router gets all multicast packets.

        It filters or forwards the packets based on the multicast
        routing protocol updates.

Vivek

---

From: Sukanta ganguly <[email-masked]>
Date: Tue, 9 Apr 2002 11:30:02 -0700 (PDT)

SG

---

From: Sukanta ganguly <[email-masked]>
Date: Tue, 9 Apr 2002 11:30:19 -0700 (PDT)

SG

---

From: Sukanta ganguly <[email-masked]>
Date: Tue, 9 Apr 2002 11:36:22 -0700 (PDT)

Bill,

SG