Subject: RE: [Ipoverib] IPoIB multicast forwarding

From: "Larsen, Roy K" <[email-masked]>
Date: Wed, 10 Apr 2002 09:58:22 -0700

/rkl
embedded
rkl/

-----Original Message-----
From: Vivek Kashyap [mailto:[email-masked]]
Sent: Tuesday, April 09, 2002 1:20 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [Ipoverib] IPoIB multicast forwarding

> 
> Ahh, that clarifies it for me.  However, from an implementation point of
> view, the IPoIB link driver can not indicate that it supports promiscuous
> mode, so one would presume that the OS network infrastructure will not use
> that interface for multicast routing unless the OS network stack were
> modified.  If you need to modify the network stack, why not simply alter
it
> so that it IB creates/joins all multicast groups it has received updates
> for?  In other words, keep all the special processing and "fixes" local to
> the router implementation and out of the common host IPoIB driver.

Aren't you saying the same thing as the solution I have presented?

/rkl
I am utterly confused so I don't know.  I was trying to remove the need of a
multicast sender to broadcast a frame if the multicast group did not exist.
Your response to Bill said that multicast router does not join any group at
the IP level.  However, below you state that it will IB_join based on
IGMP/MLD reports.  Isn't that at the IP level?  If not, what is going to
prompt the IPoIB driver to do the IB_join?

If I understood Yaron's proposals correctly, in one he suggests simply
broadcast all multicast frames on IB fabrics.  When considering the overhead
and relative complexity of IB multicast management, this is very compelling
to me.  But I obviously need to get much smarter on IGMP before contributing
anything meaningful to this thread.

The other point I was trying to make was one of implementation.  An IPoIB
driver should not indicate that it supports promiscuous/all-multicast
filtering unless it truly can.  So, if routers using IGMP protocols require
this capability of a network interface, there are going to be some software
integration issues.

Roy
rkl/

        The router knows the groups to forward to based on IGMP/MLD
        reports and multicast routing protocol updates. It therefore
        IB_joins (or creates in some cases) the corresponding IB
        MGIDs. 

For the above to occur the MLD/IGMP report must reach the router. So,
the host must send it to the router.

        The host network stack is not effected. The solution is
        limited to the host's interface driver. As I had mentioned in
        the presentation at the IETF meet: the IP_ADD_MEMBERSHIP call
        causes the host IP stack to tell the interface
        (ethernet/IPoIB/FDDI/Token ring etc.) of the IP multicast
        group. In the case of ethernet the ethernet driver updates the
        device/internal tables to include the corresponding ethernet
        mutlicast group. In the case of IPoIB the driver lets the
        SM/SA know (create if group doesn't exist; join if it
        exists). 

So, the solution is limited to a) routers' IPoIB module b) in the
IPoIB interface driver of the host.

Vivek
> 
> Roy
> 
> -----Original Message-----
> From: Vivek Kashyap [mailto:[email-masked]]
> Sent: Tuesday, April 09, 2002 12:37 PM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Ipoverib] IPoIB multicast forwarding
> 
> 
> Bill,
>         The multicast router does not 'join' any group at the IP
>         level. The router sets the ethernet interface in all-multicast
>         promiscuous mode. Thus the router gets all multicast packets.
> 
>         It filters or forwards the packets based on the multicast
>         routing protocol updates.
> 
> Vivek
> 
> 
> > 
> > Question then...
> > 
> > As an IP Multicast sender, I am not required to join 
> > the Multicast group... So now there has not been an
> > IGMP message telling the first hop router to join the
> > group either.  So the router has not told its layer
> > 2 to join the broadcast address for the multicast group.
> > 
> > How does the first hop Multicast router know to join
> > this group and forward the traffic on to the multicast
> > tree (either RP, or Prune/Flood) if it can't receive
> > the initial packet...
> > 
> > This is the magic that I was wanting to know, I know how
> > I would make it work, but I am hoping Bill F. can tell me
> > the correct way to do it <G>
> > 
> > Bill Strahm
> > 
> > On Tue, Apr 09, 2002 at 11:36:22AM -0700, Sukanta ganguly wrote:
> > > Bill,
> > >   When you send in the Multicast packet (as a sender),
> > > at layer 3 you send it to the multicast IP address
> > > that the sender decided on. Your layer 2 module will
> > > decide on the next hop for that Multicast IP address.
> > > And a similar functioning will be build up as a chain.
> > > Note in this pacth while the senders original packet
> > > is moving to its destination which is the IP multicast
> > > address, the intermediate routers (which are multicast
> > > aware) will check to see if they have any IGMP joins
> > > they have received from any other subnets. If they
> > > have any such mapping they have built due to receive
> > > of IGMP joins then they will oblige that join by
> > > copying the packet to that joined destination. That's
> > > how the copy of the packet is enroute to its rightful
> > > receiver.
> > > 
> > > SG
> > > 
> > > P.S :: Dr Fenner is an authority on this and can
> > > provide some special comments 
> > > 
> > > 
> > > --- Bill Strahm <[email-masked]> wrote:
> > > > Ok, I am going to express a scary amount of
> > > > ignorance here...
> > > > 
> > > > I know how multicast works on receive, I know how
> > > > IGMP works to build the forwarding tree,
> > > > I know how flood/prune, RP Multicast Routing
> > > > protocols work...
> > > > 
> > > > What I would like a pointer to is a simple this is
> > > > how a first hop multicast router 
> > > > behaves.  An example on Ethernet would be
> > > > acceptable, a generic example would be better.
> > > > 
> > > > The question I have is as a Multicast sender, I send
> > > > a multicast packet to the first
> > > > hop router that is multicast aware.  Now, do I send
> > > > that packet to its unicast MAC 
> > > > address, or do I send that to the mapped layer 2
> > > > broadcast address and let the IP 
> > > > layer rebroadcast to the broadcast address if it is
> > > > needed on the local subnet.
> > > > 
> > > > Once I have that, everything else I understand
> > > > 
> > > > Bill
> > > > PS Bill Fenner, the IPoIB list is subscriber posting
> > > > only.  If you haven't subscribed, please do 
> > > > so we can get the benefit of your wisdom...
> > > > 
> > > > On Tue, Apr 09, 2002 at 08:53:57AM -0500, Culley,
> > > > Paul wrote:
> > > > > I suspect that the problem can be made simpler; IP
> > > > routers should always IB_join ALL IPoIB Multicast
> > > > groups for a given interface.  The router will
> > > > simply ask to join any groups in the same
> > > > partition(s) it is assigned to.  Whether the IP
> > > > router forwards the multicast packets will then
> > > > depend on the IP routing protocol.
> > > > > 
> > > > > IB routers have the same problem, and may use the
> > > > same solution...
> > > > > 
> > > > > Paul R. Culley
> > > > > Compaq Sr. Fellow
> > > > > 281-514-5543
> > > > > 
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Vivek Kashyap [mailto:[email-masked]]
> > > > > Sent: Tuesday, April 09, 2002 3:31 AM
> > > > > To: [email-masked]
> > > > > Cc: [email-masked]
> > > > > Subject: [Ipoverib] IPoIB multicast forwarding
> > > > > 
> > > > > 
> > > > > 
> > > > > Folks,
> > > > > 
> > > > >         We have defined a mapping from the IP
> > > > addresses to the IB
> > > > >         multicast groups. Use of different groups
> > > > causes minimal
> > > > >         replication at the switches and less load
> > > > at the receivers.
> > > > > 
> > > > >         But there is a quirk in the way the groups
> > > > are setup in
> > > > >         InfiniBand - there is no way for any of
> > > > the nodes to know if
> > > > >         another has created an InfiniBand group.
> > > > This is easy to fix
> > > > >         in most cases - one checks with the SM.
> > > > But it doesn't work
> > > > >         out well for IPoIB packet forwarding.
> > > > > 
> > > > >         The router needs to forward the multicast
> > > > packets beyond the
> > > > >         local IP subnet if it knows there are
> > > > recipients for it or in
> > > > >         the case of some routing protocols it must
> > > > do that unless it
> > > > >         gets an update stating not to forward.
> > > > Similarly, it needs to
> > > > >         forward external packets to the local
> > > > receivers if there are
> > > > >         any receiver present. 
> > > > > 
> > > > >         Thus if there are IB multicast groups
> > > > associated with an IP
> > > > >         multicast group the router must know about
> > > > them or the router
> > > > >         must receive all multicast packets in the
> > > > subnet via some
> > > > >         other construct.
> > > > > 
> > > > >         Below are the three alternatives that I
> > > > mentioned at the
> > > > >         53rd IETF meet. 
> > > > > 	
> > > > >         i) Define only one IB multicast group i.e.
> > > > IB broadcast MGID.
> > > > >         All IP multicast packets use this group.
> > > > > 
> > > > >         ii) The multicast IP packet is sent to
> > > > both the associated IB
> > > > >         multicast group and the IB mutlicast group
> > > > corresponding to
> > > > >         the all-IP routers IP group.
> > > > > 
> > > > >         iii) Sender always sends to corresonding
> > > > IB group if it exists
> > > > >         else it sends to the broadcast group.
> > > > > 
> > > > >              IP receivers create IB groups and
> > > > send the IGMP/MLD
> > > > >              groups on the broadcast MGID.
> > > > > 
> > > > >              Routers join relevant groups based on
> > > > the IGMP report
> > > > >              and routing protocol updates.
> > > > > 
> > > > > 
> > > > >         I also put this problem to Bill Fenner (AD
> > > > of routing area)
> > > > >         and asked his advice on the alternatives.
> > > > Of the three options
> > > > >         he favored option (iii) above.
> > > > > 
> > > > >         Option (i) doesn't take advantage fo IB
> > > > multicast at all and
> > > > >         so should be looked at as a last resort
> > > > solution only.
> > > > > 
> > > > >         Option (ii) requires the sender to send
> > > > two copies of the same
> > > > >         packet and 'deliberately' increases the
> > > > chance of duplicates
> > > > >         spreading across the network.
> > > > > 
> > > > > 
> > > > > Suggested solution  in detail:
> > > > > 
> > > > > 
> > > > >         1. Pure senders - at IB level check for
> > > > the relevant IB group.
> > > > >            If it exists then send to the IB group
> > > > else send to the
> > > > >            'all-routers' IB group. If there is no
> > > > IB group
> > > > >            corresponding to a particular IP group
> > > > then it implies that
> > > > >            there are no IP receivers on the subnet
> > > > for that IP
> > > > >            multicast group. 
> > > > > 
> > > > >            Note: a) Pure senders don't create IB
> > > > groups. This is
> > > > >            because there is no existing mechanism
> > > > for the senders to
> > > > >            let the routers know of the new groups.
> > > > > 
> > > > >         2. IP receivers: As per IGMP/MLD the first
> > > > packet sent by a
> > > > >            host on joining a multicast group for
> > > > receiving
> > > > >            (IP_ADD_MEMBERSHIP) is the multicast
> > > > report.
> > > > > 
> > > > > 	   The hosts, as a result of IP_ADD_MEMBERSHIP,
> > > > do the following:
> > > > > 
> > > > > 	   	check if the corresponding IB group exists. 
> > > > > 
> > > > > 		if (IB group exists)
> > > > > 			IB_join the group and send the IGMP/MLD
> report
> > > > > 
> > > > > 		else
> > > > > 			IB_Create the group
> > > > > 			IB_join the group
> > > > >                         Send IGMP/MLD report to IB
> > > > group corresponding
> > > > >                         to all-routers (all-hosts
> > > > or broadcast) IP
> > > > >                         group (Note: only done
> > > > when creating the IB
> > > > >                         group) 
> > > > > 
> > > > > 	3. IP routers IB_join MGIDs on:
> > > > > 
> > > > > 		a) Finding a new IP group on the subnet in the
> > > > IGMP/MLD report
> > > > >                 b) Determining a need to forward
> > > > IP multicast packets
> > > > >                    through a routing protocol
> > > > > 
> > > > > 
> > > > > 
> > > > > Comments ?
> > > > > 
> > > > > 
> > > > > Vivek
> > > > 
> > > === message truncated ===
> > > 
> > > 
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > 
> > _______________________________________________
> > IPoverIB mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/ipoverib
> > 
> 
> 
> -- 
> Vivek Kashyap
> Linux Technology Center, IBM
> [email-masked] 
> [email-masked] 
> 503 578 3422 (o)
> 
> _______________________________________________
> IPoverIB mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipoverib
> 

-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 10 Apr 2002 11:45:00 -0700 (PDT)

> multicast sender to broadcast a frame if the multicast group did not exist.

        Actually, it is not broadcast it is sent to all-routers MGID.
        So it is received by routers only. Since there are no
        receivers there is no need to send it on the broadcast MGID.

> prompt the IPoIB driver to do the IB_join?

        The IGMP/MLD report lets the router know the IP multicast
        groups that exist on an interface. I am suggesting that the
        router then give a command to the interface to add the
        corresponding IB group to its list. This command, at the IPoIB
        driver translates into an IB_join for the corresponding IB
        group. 

> to me.  

        It is compelling to you to broadcast everything and yet you
        are bothered that a sender will broadcast(though it won't) its
        packets if it can't IB_join the relevant group ??? 

But I obviously need to get much smarter on IGMP before contributing
> this capability of a network interface, there are going to be some software

        This does not arise with respect to the solution I have
        indicated. Neither the router nor the end hosts are being
        asked to support such a mode. 
	
        The solution hinges on the IP multicast senders sending to
        groups that exist - IB group corresponding to the IP group or
        the all-routers MGID. 
	
        The IP multicast receivers create IB groups if they don't
        exist. When they create a group they must ensure that the
        IGMP/MLD report is received by the routers. And so instead of
        sending it to the group they just created they send it to the
        MGID corresponding to the all-routers IP address.

        The IP routers receive packets only on MGIDs they are members
        of - this includes the broadcast/all-routers MGIDs and the
        MGIDs they have joined. When they parse an IGMP/MLD report and
        find a new IP group they IB_join the corrresponding IB group.

Vivek