Subject: [Ipoverib] Comments on draft-ietf-ipoib-link-multicast-04.txt

From: Roy Brabson <[email-masked]>
Date: Fri, 2 May 2003 10:52:52 -0400

I've just started to look at IPoIB, so please forgive me if some of the 
comments below have been discussed previously.  I did try to read through 
the mailing list archive, but may have missed some of the relevant posts.

- The draft contains a reference to draft-ietf-ipoib-architecture-01.txt. 
  This is an expired draft which hasn't been updated in almost 18 months. 
  Should this reference be deleted, or is referenced draft updated and 
  published as an RFC?

- In Section 9.0, am I correct in assuming that all routers MUST join the 
  all-routers multicast group if it exists?  And, if one does not exist, 
  shouldn't the router create the group when it first connects to the IB 
  subnet as a "FullMember"?  This section also makes it sound as though 
  the IB administrator must create the all-router multicast group, which 
  doesn't seem to be necessary.  And it doesn't describe if/when/how a 
  router is expected to join the all-router multicast group.

- Section 9.0 indicates that "The packet SHOULD be forwarded to locally 
  connected routers.  [snip] The specific mechanism for a sender to 
  forward packets to routers are left to implementations".  At a 
  minimum, I think the exact implementation needs to be specified.  The 
  text suggests using all-router multicast group or the broadcast group. 
  If a router is required to create the all-router multicast group (see 
  the previous comment), then wouldn't it be sufficient to check for the 
  existence of the all-router multicast group and, if it exists, send to 
  that?  If it doesn't exist, then there can't be any multicast routers 
  on the link, and since there aren't any local listeners there are no 
  on-link listeners there isn't any need to send the packet.

  And shouldn't the IPoIB sender subscribe to the IB multicast group 
  creation events using a wildcard MGID so that it can 
  "SendOnlyNonMember" join the group if a listener appears on the local 
  IB subnet, or "SendOnlyNonMember" join the all-routers group if a 
  router appears on-link and creates the all-routers multicast group? 
  And doesn't it need to subscribe to the IB multicast group deletion 
  events so that it knows when to switch back to sending to the 
  all-router multicast group and/or stop sending?

  But I'm wondering why a node needs to do this at all.  Wouldn't it 
  make more sense for the IPoIB node instead create the multicast group 
  and join the group so that IPoIB multicast routers can join the group? 
  Yes, the IPoIB sender will need create the group as a "FullMember", 
  but this seems to provide a simpler mapping of IP multicast to IB 
  multicast.

- Section 9.0 also indicates that "A multicast sender must join the 
  target multicast group as a "SendOnlyNonMember" before outgoing 
  messages from it can be successfully routed." This text should be 
  clarified to indicate this is only true if the node has not already 
  joined the group as either a "FullMember" or "SendOnlyNonMember".  The 
  If-Then-Else logic below correctly shows this behavior, but this 
  paragraph does not reflect that behavior.

- Section 9.0 states that the MGID cache may need to be validated 
  periodically.  I think it would be useful to indicate why this might 
  be necessary, how a node should go about doing so, and what action 
  should be taken if the MGID is found to be invalid.  I'm assuming that 
  this would be done by checking with the SA on the outbound link for a 
  "MCMemberRecord" that matches the MGID and, if not found, creating the 
  multicast group and rejoining.

- Section 9.0 indicates that "For the rest of attributes, it is 
  recommended the same values from the all-node multicast/broadcast 
  group be used." Should this instead say "For the rest of the 
  attributes, the same values from the all-node multicast/broadcast 
  group SHOULD be used." Regardless of the wording, this text seems to 
  contradict the Section 7.0 which states a node must "use the rest of 
  the link attributes associated with the group for all future 
  communication on the link.  If Section 7.0 is correct, should this be 
  a MUST instead of a SHOULD?  And, if not, under what circumstances 
  would a node choose to not use the same values?

Roy
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Mon, 5 May 2003 20:50:35 -0700 (PDT)

Roy,

>To: [email-masked]
>MIME-Version: 1.0
>From: Roy Brabson <[email-masked]>
>
>  all-routers multicast group if it exists?

Yes, but this is not a new requirement imposed by IPoIB, but rather
an existing requirement from the IP layer.

>  And, if one does not exist, 
>  doesn't seem to be necessary.

You are right. In prior versions of the draft the all-router mulitcast
group played a more prominent role. Now it looks like the all-router
MC group can be treated just like many other multicast groups, and gets
created by demand when a router joins.

I'll change the "SHOULD" to "may" regarding the creation of the all-
router multicast group in the paragraph before section 8.0.

>  And it doesn't describe if/when/how a 

Isn't this determined by the IP software independent of IPoIB?

>  minimum, I think the exact implementation needs to be specified.

Why? (The draft strives to avoid dictating implementation details that
do not affect interoperability.)

>  The 

Don't the code sinppet following 9.B match pretty much what you described
above?

>  all-router multicast group and/or stop sending?

Aren't you saying the same thing (more or less) as what is described in
the section following the code snippet?

>  but this seems to provide a simpler mapping of IP multicast to IB 

While what you suggested may simplify the procedure, it essentially
removes the difference between a multicast sender and a multicast receiver,
which is wrong. In the IP multicast model a multicast sender doesn't need
to be a member of a multicast group to send. The draft has tried to
keep the same distinction.

>  paragraph does not reflect that behavior.

Although your comment is correct, I'd argue this kind of details can
be found in the IBA spec. It has nothing to do with IPoIB in particular,
So the question becomes how much IBA info we need to repeat in an IPoIB RFC.

Since the code sippet already covers this correctly, if you still believe
this needs to be clarified i can certain add more text to it.

>  multicast group and rejoining.

Good catch. The text comes from the old drafts before the MC group
creation/deletion traps were invented. Now with the trap support the
periodic cache flush is not needed anymore.

>  would a node choose to not use the same values?

7.0 describes the attributes used by a node when conducting unicast
communication. 9.0 describes the attributes used to create a multicast
group, i.e., for multicast communication. So the question is:
Must path attributes used for unicast match with those for multicast?

The flexibility allowed in the multicast case in the current draft is
deliberate. I don't know in reality if this is useful, or even good to
have. (Comments are welcome.) My design principle has been to not to
make something a requirement unless it is needed for interoperability.

Jerry

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 6 May 2003 00:40:51 -0700

My comments in <VK> below.

Vivek
--
Vivek Kashyap
Linux Technology Center, IBM
[email-masked]
[email-masked]
Ph: 503 578 3422 T/L: 775 3422

                      Roy                                                                                                    
                      Brabson/Raleigh/I        To:       [email-masked]                                    
                      BM@IBMUS                 cc:       [email-masked]                                                   
                      Sent by:                 Subject:  Re: [Ipoverib] Comments on draft-ietf-ipoib-link-multicast-04.txt   
                      ipoverib-admin@ie                                                                                      
                      tf.org                                                                                                 

                      05/05/03 08:00                                                                                         

> An IP application may send the multicast packet to any group. One could
> request that the interface be notified of every MGID creation/deletion
but the
> application(s) might never send any data to those groups. Therefore,
> the wildcard report request is something that the implementation will
> have to weigh against usage. The choice is upto the implementor.

This wasn't clear to me from reading the spec. Let me see if I have
this straight:

- A listener MUST perform a "FullMember" join for the MGID of the
  multicast group

<VK> It won't receive any packets otherwise as per IB specification. <VK>

- A listener MUST be able to receive an IP multicast packet over either
  the broadcast GID or the MGID for the multicast group

- A router MUST be able to receive an IP multicast packet over either
  the MGID, all-router MGID, or broadcast GID

<VK>

See the discussion on IP router below.

<VK>

- If there is an on-link listener, a sender may send the packet using
  either the MGID or to the broadcast GID

<VK> I don't think the draft says that it is an either or between MGID and
broadcast GID if the MGID exists. If the MGID exists the sender sends to
the MGID. Oherwise it needs to send the packet to the router(s). Therefore
the packet needs to be sent to the MGID corresponding to all-router
address.

<VK>

- If there is not an on-link listener, a sender may send the packet to
  either the broadcast GID or the all-router MGID

<VK> If there is no on-link listener then the sender should send the packet
to the all-router MGID. I agree, as you had noted earlier, that if the
all-router MGID doesn't exist then the packet can be dropped.

<VK>

Is that correct?

And, if a sender wants to send packets using the MGID, it will need to
listen for MGID creation/deletion reports so that it can switch between
the MGID and the all-router MGID (or broadcast GID, I suppose) as
on-link listeners join/leave the group.

I do have to wonder whether having such a large number of optional
mechanisms to handle the mapping of IP to IB multicasting is really a
good idea.  Would we be better off just selecting one approach and
saying all IPoIB nodes should take that approach?

<VK>
 Both IP and IB have similar concepts and use the same kind of words
therefore let us append IP or IB in front of them. Instead of saying
multicast group it is better to say IP multicast group or IB multicast
group. If MGID is used then it is clear that IB is being talked about.

IP listener:

An IP listener would like to receive packets. For such a case the IP layer
has to instruct the link layer to receive packets on the correspondingly
mapped address. In the case of IB, the mapping is defined to the MGID. This
is not enough. Therefore the link layer (IB) must IB_join the MGID as a
FullMember. The MGID is created if it doesnt exist and the SM programs the
fabric such that the packets are received by the listener.

IP multicast router:

An IP router must be able to receive the multicast packets and forward them
if required - both into the subnet or out of the subnet. It gets to know of
the groups by way of a routing protocol. Within the subnet IGMP/MLD are
used. The router can therefore, on its link interface IB_join the MGIDs
when it knows the IP multicast address(implementations don't seem to prefer
this as per earlier discussions on this list) or it can listen
promiscuously. In IB it cannot listen promiscuosly; therefore the router
needs to request MGID creation/deletion reports from the SA/SM and the
routers IB_join/leave groups on receiving the reports.

The router gets to know of the IP groups to forward to from running a
routing protocol. If there are no IP listeners (and hence no corresponding
IB MGIDs)in the subnet and the router  doesn't create the corresponding
MGID itself(as noted in the previous paragraph) then there is no way that
IP packets destined for the remote IP multicast address can reach the
router unless the packet are sent to an MGID the router is already a member
of such as the all-router address. Therefore, in the absence of the
relevant MGID the sender needs to send the packet to the all-router MGID.
This implies that the router needs to be able to receive IP multicast
packets on the corresponding multicast MGID or the all-router MGID.

Since a router really wants to receive packets as well as send packets it
needs to IB_join as a FullMember or as a NonMember. A Non-Member IB join is
preferable in that it doesn't effect the creation/tearing down of the MLID
forwarding built into the switching fabric. But an implementation can
choose to IB join as FullMember if it so prefers. This is not an option
from IPoIb but just what IB provides.

IP sender:

Any IP host can send to any IP multicast address without itself being a
member of the IP group. This is not always the case at the IB layer since
the the switches need to be programmed to forward the MLID. Therefore a
sender needs to map the IP multicast address to the corresponding MGID (as
per mapping defined in this draft) and then IB join the MGID. It can then
transmit the packet to the MGID.

If there are IP listeners then there will be a corresponding IB MGID that
the sender will IB_Join. The router, because it IB_joins all new IB MGID
creations (see above) will also be a member and receive the packet. If
there are no local subnet IP listeners, the corresponding MGID wouldn't
exist. In such a case sending to MGID corresponding to all-routers IP
multicast address ensures that the multicast addresses are receieved by the
router.

One could also argue that the sender should, instead of the
SendOnlyNonMember join do a FullMember join. Then the MGID will be created
if it doesn't exist and the router will join in on receiving the creation
report. But, there is very likely a delay between the creation of the MGID,
the report to the router and the router joining the MGID; during this delay
the packet is likely dropped. Multicast is not reliable but in this
situation the chances of loss are quite high and hence it is not a
preferable way of transmitting multicast packets.

Vivek

>
> It does seem like a reasonable thing to request creation/deletion
> report on the all-router multicast GID.
>
>
> >
> It will work but how is this a better thing to do ? I also
> do not think that such behavior can/should be disallowed.

I'm just trying to understand if there is some way to simplify the
mapping of IP multicast groups to IB multicast groups.  My main concern
is with listening for IB group creation/deletion reports so a sender
can tell if/when to use the multicast group's MGID and when to use the
all-router MGID.  If the sender performs a "FullMember" join, then it
will always send using the MGID for the corresponding IB multicast
group.

But, based on what you said above, a sender can always use the broadcast
GID when sending to a multicast group for which it is not also
listening.  This does simplify the logic, but at the cost of forcing
every IPoIB node to filter the IP multicast packets for which they are
not interested.  Then again, it saves the SM from having to reprogram
the switches/routers in the IB fabric when the sender performs a
non-member join.  To be honest, I have no idea how to go about making
a rational choice on which approach will yield the better results.

Roy

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 6 May 2003 00:40:51 -0700

Vivek

                      Roy                                                                                                    

                      05/05/03 08:00                                                                                         

but the

<VK>

<VK>

<VK>

<VK>

Is that correct?

<VK>

IP listener:

IP multicast router:

IP sender:

Vivek

Roy

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Tue, 6 May 2003 21:55:58 -0700 (PDT)

>For interoperatibilty.  The requirement is the host and router must agree 
>on using a common mechanism for exchanging the IP multicast packet. 
>Implementations are free to choose a mechanism as long as all other 
>implementations with which they communicate also choose to implement the 
>same mechanism.  I don't think this is an area where we can or should punt 
>- a minimal, common approach needs to be mandated such that all IPoIB 
>nodes implement that mechanism so that any two IPoIB nodes will 
>interoperate.

I still don't see any problem you may have in mind. As long as a sender
gets the pkt to the router, why it matter how it gets there? E.g.,
routers will listen to both the broadcast and the all-router multicast
addresses. Does it matter which address a pkt is sent to?

>Yes, it does, and if the text matched the code snippet then I wouldn't 
>have an issue.  But the text which supports the code snippet doesn't.

I can remove the text if it causes any confusion. I thought it is very
obvious if no router exists then there is no need to forward (as
illustrated by the code snippet).

>The 
>text basically says you could send to the all-router multicast group OR 
>the broadcast group.  And either should work, as long as the receiving 
>IPoIB node MUST accept the IP multicast packets regardless of the IB group 
>over which they are received (I'm not sure I've seen this explicitly 
>stated in the draft, but I may have missed it).

I thought this is required by IP already. But please tell me if I'm
wrong.

>But do we really need to 
>have multiple mechanisms to accomplish the same result?  Wouldn't we be 
>better off just choosing one and going with that?  And how are we going to 
>find multiple interoperable implementations for each option when the draft 
>puts no limits on the approaches which can be taken?

The multiple mechanisms were not there in the earlier draft, but were 
requested later by my co-author. I personally don't see any harm with
it but I don't mind taking it out if Vivek agrees with you.

>You are right - I must have missed the text.  But you need to change the 
>"should" to a "MUST" in that paragraph.

Since the caching part is a "should", the subscription part ought to be
a "should" too, right?

>However, you are mapping the IP multicast model to the IB multicast model, 
>and the two are not the same.  The reality is a sender must join an IB 
>multicast group in order to send to it (albeit as a "SendOnlyNonMember"),

This is the big difference that was debated in IBTA.

>and this step does not exist in the IP model.  I imagine that the vast 
>majority of the code/complexity of an IPoIB implementation is going to be 
>dealing with IB multicasting.  I was hoping there was some way to reduce 
>both at an IPoIB host.  Maybe it is just wishful thinking.

I actually have the same philosophy as well. I'm always against complexity
unless it's fully justified. But IB multicast just works differently than
IP multicast. We have gone through a number of alternatives and the
current one seemed to a good balance between complexity and correctness.
I don't remember all the tradeoffs but if we simply require a sender
to be a full-member, how would one know when to leave a group, when to
delete a group..., etc. Also wouldn't it enable anyone to write a silly
sender program to try to send to a million different addresses and cause
all the IB multicast resources to be used up? (The join call by 
IP_ADD_MEMBERSHIP at least places some resource limitation.)

>
>By the way, after thinking about this a little more, I'm not sure that 
>creating a group as I suggested makes sense in all cases.  For instance, 
>it is only useful for an IP multicast group which has a scope greater than 
>link-local.  For link-local scope, only nodes directly attached to the 
>IPoIB link are able to receive the packet.  Creating the group for the 
>purpose of forwarding to an IPoIB multicast router is simply wasteful, as 
>the IPoIB multicast router will not be able to forward the packet to 
>another IP link.
>
>A similar optimization can also be taken with the current approach 
>described in this draft - for an IP multicast group with link-local scope, 
>if the IB MGID for that IP multicast group does not exist then an IPoIB 
>node really doesn't need to send the multicast packet to the all-router 
>MGID or broadcast GID.

Correct. This level of details can arguably be left to implementation
because it doesn't affect interoperability. But I think this optimization
is simple and useful enough that I'll add a note to the draft.

>My feeling is enough to describe HOW an IPoIB node will USE the IBA 
>services, and this is one of those cases.  I would prefer to see the text 
>clarified if you do another revision of the draft, but given the code 
>snippet describes the correct behavior I wouldn't hold this draft up just 
>to clarify this paragraph.

Ok.

>Flexibility is often times good, but so is describing a minimal set of 
>functions necessary for interoperability.  If a node selects link 
>attributes when creating an MGID which does not match the broadcast GID's 
>attributes, then it is possible that not all nodes which are members of 
>the IPoIB link will be able to support those attributes.  If this happens, 
>then the two nodes will not be able to communicate using the IP multicast 
>group mapped to the MGID.  All of a sudden, an IPoIB node can't send an 
>ARP request or Neighbor Solicitation to another node.

I think some of these "deliberate" flexibility (or ambuguity) came from
the ambuguity of IB itself in some of the mutlicast attributes such as
SL, TClass..., etc. It is not clear to me if they must or even can
follow the ones for unicast (i.e. described by the broadcast group)
in all cases.

>
>Perhaps it would be better to state that a node which creates a multicast 
>group MUST use the same values for the attributes as the all-node 
>multicast/broadcast group, but a node joining a multicast group must not 
>verify the values match.  If the node joining the group can't support the 
>values for the multicast group, then it should log the error.  That way, a 
>future RFC could describe why/when/how a node would choose to use 
>different values for the link attributes but still be able to interoperate 
>with IPoIB nodes.

I'll leave the above for IB folks to comment.

Jerry

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 7 May 2003 08:36:28 -0700

<snip..>

> different values for the link attributes but still be able to
interoperate

That is a valid idea. The other alternative is to list the
parameters that may differ from the ones used by the
broadcast-GID.

The parameters for an IB multicast group are:

             Q_Key, MLID, MTU, TClass, P_Key, Rate, SL, PacketLifeTime,
             FlowLabel, Hoplimit, scope.

We certainly need the P_Key and MTU to be the same as the
broadcast MGID. Similarly all multicast mappings in a subnet
must have the same 'scope'.

The other parameters can be different if it is ensured that they
are all supported by all the members of the IP subnet. There is
no well defined method of ensuring this - none is being defined
by us.  On the other had, by definition all members support the
parameters supported by the broadcast-GID.

Therefore I agree with you... Jerry, I think we should change the
draft to state the following about the parameters:

             1. unicast communication use the parameters used by
             broadcast-GID

             2. multicast GIDs created should use the same parameters as
             the broadcast-GID.

Vivek

----- Forwarded by Vivek Kashyap/Beaverton/IBM on 05/07/03 08:28 -----
                                                                                                                                        
                      [email-masked]                                                                                             
                      nux.ibm.com                  To:       Roy Brabson/Raleigh/IBM@IBMUS                                              
                      Sent by:                     cc:       "H.K. Jerry Chu" <[email-masked]>, <[email-masked]>              
                      [email-masked]        Subject:  Re: [Ipoverib] Comments on draft-ietf-ipoib-link-multicast-04.txt          
                      rg                                                                                                                

                      05/06/03 15:43                                                                                                    

Roy,

Yes, I agree that a common method must be described for interoperability.
Though, in this case, even with the flexibility
interoperability is maintained - the onus being on the router :

The packet may be sent to the routers on broadcast-GID or the
all-router GID. The router itself will listen on both -
broadcast-GID since it is required by IPoIB and all-routers as a
result of other specs (IPv6 & IGMP). Therefore the packet will be
received by the router irrespective of the MGID used as long as
the router accepts any multicast packets on these MGIDs. I have
described the need for this in my previous reply.

We could do one of three things when the relevant MGID does not
exist:

1. The host must always forward on broadcast MGID
             and the router must always accept IP multicast packets on the
             broadcast GID.

2. The host must always forward on all-router MGID and the router
             must always accept IP multicast packets on the all-router
             MGID

3. The host must always forward on broadcast GID or all-router
   MGID. The router must accept IP multicast packets on both the
   broadcast MGID and all-router MGID.

We have chosen case 3 . Is there a strong feeling that one of the
other cases is preferable?

It appears that the method chosen(case 3) needs to be
explicitly stated in the draft.

The
> stated in the draft, but I may have missed it).  But do we really need to

That is a valid idea. The other alternative is to list the
parameters that may differ from the ones used by the
broadcast-GID.

The parameters for an IB multicast group are:

             Q_Key, MLID, MTU, TClass, P_Key, Rate, SL, PacketLifeTime,
             FlowLabel, Hoplimit, scope.

We certainly need the P_Key and MTU to be the same as the
broadcast MGID. Similarly all multicast mappings in a subnet
must have the same 'scope'.

The other parameters can be different if it is ensured that they
are all supported by all the members of the IP subnet. There is
no well defined method of ensuring this - none is being defined
by us.  On the other had, by definition all members support the
parameters supported by the broadcast-GID.

Therefore I agree with you... Jerry, I think we should change the
draft to state the following about the parameters:

             1. unicast communication use the parameters used by
             broadcast-GID

             2. multicast GIDs created should use the same parameters as
             the broadcast-GID.

Vivek

> Roy

__

Vivek Kashyap

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 7 May 2003 08:36:28 -0700

<snip..>

Vivek

----- Forwarded by Vivek Kashyap/Beaverton/IBM on 05/07/03 08:28 -----
                                                                                                                                        
                      [email-masked]                                                                                             

                      05/06/03 15:43                                                                                                    

Roy,

The

Vivek

__

Vivek Kashyap

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 7 May 2003 09:01:04 -0700

See below between <VK>

                                                                                                                                    
                      "H.K. Jerry Chu"                                                                                              
                      <[email-masked]        To:       Roy Brabson/Raleigh/IBM@IBMUS                                              
                      n.com>                   cc:       [email-masked]                                                          
                      Sent by:                 Subject:  Re: [Ipoverib] Comments on draft-ietf-ipoib-link-multicast-04.txt          
                      ipoverib-admin@ie                                                                                             
                      tf.org                                                                                                        

                      05/06/03 21:55                                                                                                
                      Please respond to                                                                                             
                      "H.K. Jerry Chu"                                                                                              

<VK> As I noted in my previous mail... the onus is on the router. The
router is required to listen to multicast packets on both: a) all-router
MGID or b) broadcast-GID

We could make that explicit in the draft. The second option, which
apparently Roy favours is to specify that the nodes must, if the relevant
MGID doesn't exist, send the packet to the MGID corresponding to the
all-router MGID. This option is fine but it then adds another requirement:

   All IPoIB nodes MUST request creation request for all-router MGID
   whether the corresponding IP layer joins the all-router IP address or
   not.

If the fall-back is the broadcast-GID then the end-nodes do not have to do
anything special.

The current option allows the sender to choose between taking the simple
route of 'fall-back to broadcast GID' vs. 'must join all-router MGID'.

I am ok with either case and if the 'must join all-router MGID' case is
preferable then we can specify that instead.
<VK>

<VK>
Don't remember, could have on reasoning as described above.

I think we can take it out and replace it with the condition that the
packets are sent to the all-router MGID AND all IPoIB nodes (because any
can be a sender) need to request creation report for all-router MGID and
then IB_Join it.
<VK>

<VK> Yes,  a major consideration has been the limited multicast resources
avaialble in IB.  Some other issues were such as the case I described a
little while back: there is a very large chance of the router not receiving
the packet when the sender the group due to the possible delay in the
report reaching the router and the router IB_joining the group etc.
<VK>

<VK> Agreed. <VK>

Ok.

Jerry

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 7 May 2003 10:34:54 -0700

>    whether the corresponding IP layer joins the all-router IP address or

I'm not sure I understand why an IPoIB host need to request creation of the
all-router MGID (I do understand why an IPoIB router must, though).

<VK>
Oops! a typo. Not creation -- IB_join. All nodes need to IB_join
"SendOnlyNonMember" the all-router MGID whether the IP layer joins the
all-router IP address or not.

<VK>

 Wouldn't it be sufficient to try to "SendOnlyNonMember" join the
all-router MGID and, if the all-router MGID doesn't exist, stop?  If the
all-router MGID doesn't exist, then it means there are no IP routers on the
IPoIB link who can forward the request to any off-link listeners.  And,
since the multicast group MGID doesn't exist (otherwise, the IPoIB host
would have sent to that MGID), we already know there aren't any on-link
IPoIB listeners.

Roy

---

From: Vivek Kashyap <[email-masked]>
Date: Wed, 7 May 2003 10:34:54 -0700

<VK>

<VK>

Roy

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Sat, 10 May 2003 23:42:33 -0700 (PDT)

>I guess I thought that a receiver might choose to discard IB packets which 
>is sent to the wrong (M)GID.  If this isn't possible or allowed by IBA, 

I don't think it is possible to determine whether a destination link layer
address is "right" or "wrong" wrt to a dst IP addr in general.

>then you are right - it doesn't matter how a packet arrives at the local 
>QP, just as long as it arrives.  So, I guess an alternative to using the 
>all-router MGID or the broadcast GID would be to have an IPoIB host 
>somehow determine all listeners and unicast packets to each listener.  Not 
>a real good alternative, but it would work.

Yup.

>
>Even if there are many possible ways to make IPoIB work, we should be 
>choosing the "best" or "most appropriate" one(s) and documenting those. 
>This appears to be sending to the all-router MGID, or possibly the 
>broadcast GID.  Why not simply state these are the mechnisms defined by 
>IPoIB, but state that future standards may define additional mechanisms 
>for delivery of multicast packets to the IPoIB router.

Again the draft has been carefully written to separate the required
properties of IPoIB from implementation freedom. For a multicast receiver
the draft clearly states it MUST join the approiate MGID. But for a
multicast sender it doesn't really matter how the link layer manages to
get an outbound multicast pkt to its receivers (including all the MC
routers), as long as it does NOT make any undue assumption about
receivers. One specific consideration is that although we don't
encourage, we still want to allow implementations that must resort
to broadcast when sending MC pkts to be declared compliant. An
analogy is IP over Ethernet does not dictate implementations such as
multicast receivers mustn't resort to promiscuous mode when its hash
table overflows...

Perhaps the best is not to dictate but to suggest the preferred
implementation (i.e. the all-router MC group).

>> >> >  the previous comment), then wouldn't it be sufficient to check for 
>
>> >the 
>> >> >  existence of the all-router multicast group and, if it exists, 
>send 
>> >to 
>I responded to a separate post from Vivek on this topic.  I'm OK if both 
>mechanisms (send to the all-router MGID or broadcast GID) are left in, 
>although I do think the best approach is to send to the all-router MGID. 
>Doing so reduces the amount of traffic which needs to be filtered by IPoIB 
>hosts when there are no on-link listeners.

Ok.

>Yes, you are right.  Listening for creation/deletion reports doesn't make 
>any sense if the IPoIB node does not cache the MGID for groups which it 
>"NonMember" joins.
>
>Yes, there are issues which might well make it more complex.  As I said, 
>maybe it was wishful thinking.  From the sound of it, it was.
>
>Sounds good.
>> >> So the question becomes how much IBA info we need to repeat in an 
>IPoIB 
>> >RFC.
>Well, I hope they can, as that is what I was planning on doing for any 
>MGIDs which I had to create.  If they can't, I have no idea how to come up 
>with the appropriate values for the attributes.

The flexibility is not for you :-), but for those who for some reason
know how to come up with an optimal set of values which turn out to
be different than those of the broadcast group.

Well I'll study IBA spec more to see what needs to be change here.

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Sat, 10 May 2003 23:42:33 -0700 (PDT)

Yup.

Ok.

---

From: "H.K. Jerry Chu" <[email-masked]>
Date: Sun, 11 May 2003 00:48:54 -0700 (PDT)

Vivek,

>We could make that explicit in the draft.

I'm not sure we need to restate various properties that are required
of IP already (unless they are actually not required).

>The second option, which
>all-router MGID.

See my response to Roy (sorry for the delay). I'd suggest the following
paragraph:

   A) An IB multicast group must be explicitly created through the SA
   before it can be used.

   This implies that in order to send a packet destined for an IP
   multicast address, the IPoIB implementation must check with the SA on
   the outbound link first for a "MCMemberRecord" that matches the MGID.
   If one does exist, the MLID associated with the multicast group is
   used as the DLID for the packet. Otherwise, it implies no member
   exists on the local link.  If the scope of the IP multicast group is
   beyond link-local, and routers exist on the local subnet, the packet
   SHOULD be forwarded to locally connected IP routers through the use
   of the all-router multicast group or the broadcast group. This is to
   allow local routers to forward the packet to multicast listeners on
   remote networks. The all-router multicast group is preferred over the
   broadcast group for better efficiency
   
>This option is fine but it then adds another requirement:
>

I don't think the above is necessary. This is restating the requirement
of IBA when sending a MC pkt, just like repeating the requirement of IP
on joining the IP all-host MC group etc.

>Don't remember, could have on reasoning as described above.

I think you wanted (and I concurred) to allow the case if IB multicast
doesn't work, either due to implementation problems, bugs, flaws in
the spec..., one can fall back to broadcast (although not encouraged).

Jerry