Subject: Re: [Ipoverib] Comments on draft-ietf-ipoib-link-multicast-04.txt

From: "H.K. Jerry Chu" <[email-masked]>
Date: Mon, 5 May 2003 20:50:35 -0700 (PDT)

Roy,

>To: [email-masked]
>MIME-Version: 1.0
>From: Roy Brabson <[email-masked]>
>
>I've just started to look at IPoIB, so please forgive me if some of the 
>comments below have been discussed previously.  I did try to read through 
>the mailing list archive, but may have missed some of the relevant posts.
>
>- The draft contains a reference to draft-ietf-ipoib-architecture-01.txt. 
>  This is an expired draft which hasn't been updated in almost 18 months. 
>  Should this reference be deleted, or is referenced draft updated and 
>  published as an RFC?
>
>- In Section 9.0, am I correct in assuming that all routers MUST join the 
>  all-routers multicast group if it exists?

Yes, but this is not a new requirement imposed by IPoIB, but rather
an existing requirement from the IP layer.

>  And, if one does not exist, 
>  shouldn't the router create the group when it first connects to the IB 
>  subnet as a "FullMember"?  This section also makes it sound as though 
>  the IB administrator must create the all-router multicast group, which 
>  doesn't seem to be necessary.

You are right. In prior versions of the draft the all-router mulitcast
group played a more prominent role. Now it looks like the all-router
MC group can be treated just like many other multicast groups, and gets
created by demand when a router joins.

I'll change the "SHOULD" to "may" regarding the creation of the all-
router multicast group in the paragraph before section 8.0.

>  And it doesn't describe if/when/how a 
>  router is expected to join the all-router multicast group.

Isn't this determined by the IP software independent of IPoIB?

>
>- Section 9.0 indicates that "The packet SHOULD be forwarded to locally 
>  connected routers.  [snip] The specific mechanism for a sender to 
>  forward packets to routers are left to implementations".  At a 
>  minimum, I think the exact implementation needs to be specified.

Why? (The draft strives to avoid dictating implementation details that
do not affect interoperability.)

>  The 
>  text suggests using all-router multicast group or the broadcast group. 
>  If a router is required to create the all-router multicast group (see 
>  the previous comment), then wouldn't it be sufficient to check for the 
>  existence of the all-router multicast group and, if it exists, send to 
>  that?  If it doesn't exist, then there can't be any multicast routers 
>  on the link, and since there aren't any local listeners there are no 
>  on-link listeners there isn't any need to send the packet.

Don't the code sinppet following 9.B match pretty much what you described
above?

>
>  And shouldn't the IPoIB sender subscribe to the IB multicast group 
>  creation events using a wildcard MGID so that it can 
>  "SendOnlyNonMember" join the group if a listener appears on the local 
>  IB subnet, or "SendOnlyNonMember" join the all-routers group if a 
>  router appears on-link and creates the all-routers multicast group? 
>  And doesn't it need to subscribe to the IB multicast group deletion 
>  events so that it knows when to switch back to sending to the 
>  all-router multicast group and/or stop sending?

Aren't you saying the same thing (more or less) as what is described in
the section following the code snippet?

>
>  But I'm wondering why a node needs to do this at all.  Wouldn't it 
>  make more sense for the IPoIB node instead create the multicast group 
>  and join the group so that IPoIB multicast routers can join the group? 
>  Yes, the IPoIB sender will need create the group as a "FullMember", 
>  but this seems to provide a simpler mapping of IP multicast to IB 
>  multicast.

While what you suggested may simplify the procedure, it essentially
removes the difference between a multicast sender and a multicast receiver,
which is wrong. In the IP multicast model a multicast sender doesn't need
to be a member of a multicast group to send. The draft has tried to
keep the same distinction.

>
>- Section 9.0 also indicates that "A multicast sender must join the 
>  target multicast group as a "SendOnlyNonMember" before outgoing 
>  messages from it can be successfully routed." This text should be 
>  clarified to indicate this is only true if the node has not already 
>  joined the group as either a "FullMember" or "SendOnlyNonMember".  The 
>  If-Then-Else logic below correctly shows this behavior, but this 
>  paragraph does not reflect that behavior.

Although your comment is correct, I'd argue this kind of details can
be found in the IBA spec. It has nothing to do with IPoIB in particular,
So the question becomes how much IBA info we need to repeat in an IPoIB RFC.

Since the code sippet already covers this correctly, if you still believe
this needs to be clarified i can certain add more text to it.

>
>- Section 9.0 states that the MGID cache may need to be validated 
>  periodically.  I think it would be useful to indicate why this might 
>  be necessary, how a node should go about doing so, and what action 
>  should be taken if the MGID is found to be invalid.  I'm assuming that 
>  this would be done by checking with the SA on the outbound link for a 
>  "MCMemberRecord" that matches the MGID and, if not found, creating the 
>  multicast group and rejoining.

Good catch. The text comes from the old drafts before the MC group
creation/deletion traps were invented. Now with the trap support the
periodic cache flush is not needed anymore.

>
>- Section 9.0 indicates that "For the rest of attributes, it is 
>  recommended the same values from the all-node multicast/broadcast 
>  group be used." Should this instead say "For the rest of the 
>  attributes, the same values from the all-node multicast/broadcast 
>  group SHOULD be used." Regardless of the wording, this text seems to 
>  contradict the Section 7.0 which states a node must "use the rest of 
>  the link attributes associated with the group for all future 
>  communication on the link.  If Section 7.0 is correct, should this be 
>  a MUST instead of a SHOULD?  And, if not, under what circumstances 
>  would a node choose to not use the same values?

7.0 describes the attributes used by a node when conducting unicast
communication. 9.0 describes the attributes used to create a multicast
group, i.e., for multicast communication. So the question is:
Must path attributes used for unicast match with those for multicast?

The flexibility allowed in the multicast case in the current draft is
deliberate. I don't know in reality if this is useful, or even good to
have. (Comments are welcome.) My design principle has been to not to
make something a requirement unless it is needed for interoperability.

Jerry

>
>Roy
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Roy Brabson <[email-masked]>
Date: Tue, 6 May 2003 11:56:52 -0400

> do not affect interoperability.)

For interoperatibilty.  The requirement is the host and router must agree 
on using a common mechanism for exchanging the IP multicast packet. 
Implementations are free to choose a mechanism as long as all other 
implementations with which they communicate also choose to implement the 
same mechanism.  I don't think this is an area where we can or should punt 
- a minimal, common approach needs to be mandated such that all IPoIB 
nodes implement that mechanism so that any two IPoIB nodes will 
interoperate.

> Don't the code sinppet following 9.B match pretty much what you 
described

Yes, it does, and if the text matched the code snippet then I wouldn't 
have an issue.  But the text which supports the code snippet doesn't.  The 
text basically says you could send to the all-router multicast group OR 
the broadcast group.  And either should work, as long as the receiving 
IPoIB node MUST accept the IP multicast packets regardless of the IB group 
over which they are received (I'm not sure I've seen this explicitly 
stated in the draft, but I may have missed it).  But do we really need to 
have multiple mechanisms to accomplish the same result?  Wouldn't we be 
better off just choosing one and going with that?  And how are we going to 
find multiple interoperable implementations for each option when the draft 
puts no limits on the approaches which can be taken?

> the section following the code snippet?

You are right - I must have missed the text.  But you need to change the 
"should" to a "MUST" in that paragraph.

> keep the same distinction.

However, you are mapping the IP multicast model to the IB multicast model, 
and the two are not the same.  The reality is a sender must join an IB 
multicast group in order to send to it (albeit as a "SendOnlyNonMember"), 
and this step does not exist in the IP model.  I imagine that the vast 
majority of the code/complexity of an IPoIB implementation is going to be 
dealing with IB multicasting.  I was hoping there was some way to reduce 
both at an IPoIB host.  Maybe it is just wishful thinking.

By the way, after thinking about this a little more, I'm not sure that 
creating a group as I suggested makes sense in all cases.  For instance, 
it is only useful for an IP multicast group which has a scope greater than 
link-local.  For link-local scope, only nodes directly attached to the 
IPoIB link are able to receive the packet.  Creating the group for the 
purpose of forwarding to an IPoIB multicast router is simply wasteful, as 
the IPoIB multicast router will not be able to forward the packet to 
another IP link.

A similar optimization can also be taken with the current approach 
described in this draft - for an IP multicast group with link-local scope, 
if the IB MGID for that IP multicast group does not exist then an IPoIB 
node really doesn't need to send the multicast packet to the all-router 
MGID or broadcast GID.

> this needs to be clarified i can certain add more text to it.

My feeling is enough to describe HOW an IPoIB node will USE the IBA 
services, and this is one of those cases.  I would prefer to see the text 
clarified if you do another revision of the draft, but given the code 
snippet describes the correct behavior I wouldn't hold this draft up just 
to clarify this paragraph.

> make something a requirement unless it is needed for interoperability.

Flexibility is often times good, but so is describing a minimal set of 
functions necessary for interoperability.  If a node selects link 
attributes when creating an MGID which does not match the broadcast GID's 
attributes, then it is possible that not all nodes which are members of 
the IPoIB link will be able to support those attributes.  If this happens, 
then the two nodes will not be able to communicate using the IP multicast 
group mapped to the MGID.  All of a sudden, an IPoIB node can't send an 
ARP request or Neighbor Solicitation to another node.

Perhaps it would be better to state that a node which creates a multicast 
group MUST use the same values for the attributes as the all-node 
multicast/broadcast group, but a node joining a multicast group must not 
verify the values match.  If the node joining the group can't support the 
values for the multicast group, then it should log the error.  That way, a 
future RFC could describe why/when/how a node would choose to use 
different values for the link attributes but still be able to interoperate 
with IPoIB nodes.

Roy

---

From: Roy Brabson <[email-masked]>
Date: Tue, 6 May 2003 11:56:52 -0400

Roy

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 6 May 2003 15:43:33 -0700 (Pacific Daylight Time)

Roy,

Yes, I agree that a common method must be described for interoperability.
Though, in this case, even with the flexibility
interoperability is maintained - the onus being on the router :

The packet may be sent to the routers on broadcast-GID or the
all-router GID. The router itself will listen on both -
broadcast-GID since it is required by IPoIB and all-routers as a
result of other specs (IPv6 & IGMP). Therefore the packet will be
received by the router irrespective of the MGID used as long as
the router accepts any multicast packets on these MGIDs. I have
described the need for this in my previous reply.

We could do one of three things when the relevant MGID does not
exist:

1. The host must always forward on broadcast MGID
	and the router must always accept IP multicast packets on the
	broadcast GID.

2. The host must always forward on all-router MGID and the router
	must always accept IP multicast packets on the all-router
	MGID

3. The host must always forward on broadcast GID or all-router
   MGID. The router must accept IP multicast packets on both the
   broadcast MGID and all-router MGID.

We have chosen case 3 . Is there a strong feeling that one of the
other cases is preferable?

It appears that the method chosen(case 3) needs to be
explicitly stated in the draft.

> different values for the link attributes but still be able to interoperate

That is a valid idea. The other alternative is to list the
parameters that may differ from the ones used by the
broadcast-GID.

The parameters for an IB multicast group are:

	Q_Key, MLID, MTU, TClass, P_Key, Rate, SL, PacketLifeTime,
	FlowLabel, Hoplimit, scope.

We certainly need the P_Key and MTU to be the same as the
broadcast MGID. Similarly all multicast mappings in a subnet
must have the same 'scope'.

The other parameters can be different if it is ensured that they
are all supported by all the members of the IP subnet. There is
no well defined method of ensuring this - none is being defined
by us.  On the other had, by definition all members support the
parameters supported by the broadcast-GID.

Therefore I agree with you... Jerry, I think we should change the
draft to state the following about the parameters:

	1. unicast communication use the parameters used by
	broadcast-GID

	2. multicast GIDs created should use the same parameters as
	the broadcast-GID.

Vivek

> Roy

__

Vivek Kashyap
Linux Technology Center, IBM

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 6 May 2003 15:43:33 -0700 (Pacific Daylight Time)

Roy,

Vivek

__

Vivek Kashyap

---

From: Roy Brabson <[email-masked]>
Date: Wed, 7 May 2003 09:12:08 -0400

> explicitly stated in the draft.

My preference is for case 2, as sending to the all-router MGID reduces the 
amount of multicast traffic an IPoIB host must filter as oppposed to 
sending to the broadcast GID.  I can live with case 3, but if it is chosen 
would prefer the text to state that the all-router MGID SHOULD be used, 
but the broadcast GID MAY be used.
 
Roy

---

From: Roy Brabson <[email-masked]>
Date: Wed, 7 May 2003 09:12:08 -0400

Roy