Subject: Re: [Ipoverib] IP encapsulation Design Team

From: Vivek Kashyap <[email-masked]>
Date: Wed, 12 Dec 2001 23:32:27 -0800 (PST)

Bill,
        I notice that this message did not get posted to the mailing
        list. It didn't bounce back to me either. I did get the copy
        that I cc'd myself and so never realised it had not made it to
        the list. It is more of a fill-in-the blank to the thread at
        this time.

Vivek
---------------------------------------------------------------

Forwarded message:

---

From: Vivek Kashyap <[email-masked]>
Date: Mon, 29 Oct 2001 12:21:50 -0800 (PST)

Jerry,

        I have proposed the following mapping to IB MGIDs from IP
        addresses keeping the current specification in mind.

	   IPv4 -> FF12:3334:<P_Key>:<IPv4 broadcast/multcast address>	
	   IPv6 -> FF12:3333:<P_Key>:<lower 80 bits of IPv6 multicast address>

        You want to base the mapping on the proposal you made to the
        IBTA regarding expanding the IB multcast group name space with
        P_Key/Q_Key. 

        That sums up the basic difference between our idea of MGID
        mappings. The major difference therefore is whether the P_Key
        itself is part of the mapped MGID or not. Let us look at this
        issue in the IBTA this week and revisit this area after
        resolving the IB multicast rules there.

 Vivek 

---------------------------------------------------------------------

> >	
> >	The setup would therefore be:
> >
> >                A multicast group is setup by the administrator. The
> >                MCGroup for IPv4 would be (updated mapping from my
> >                previous mail) FF12:3334:<P_Key>::255.255.255.255.
> >
> >                All nodes MUST join this. By doing so they get the
> >                P_Key, Q_Key, and Path MTU among other things (section
> 
> Again you don't "get" the P_Key but you need to specify a P_Key.

        As you can see in the mapping given above the P_Key is
        included in the MGID. Thus, yes, the sender already knows the
        P_Key. The join operation 'confirms' the P_Key. 
	
        In my earlier draft I had suggested a single mapping that didn't 
        include the P_Key (sort of like you prefer it) and had
        supposed that the 'right' P_Key will be returned on successful
        join. The reading of the IB specification doesn't give a good
        idea on the process - thus my assumption was that the SA would
        ensure that the the requester is in the partition. The join
        request (Pg 726) doesn't include the P_Key. Under this
        assumption the requestor 'gets' the P_Key during the join
        process. 

        [ ................	
                This also allows for a default case as I had noted in
                my earlier mail

               >  One can also special-case the value of 0 in place of
               >  P_Key in the mappings when there is no need to define
               >  multiple IPoIB links. In this case the nodes take the
               >  P_Key from the all-nodes multicast group as the P_Key
               >  to use for all IPoIB communication. This could be the
               >  default case.

	...................]

        However if the mapping includes the P_Key in the MGID mapping
        ( IPv4 broadcast address to
        FF12:3334:<P_Key>::255.255.255.255) the P_Key is certainly not
        an 'out' but an 'in' parameter. I have no argument with that.

> IMO embedding P_Keys in MGIDs is completely unnecessary if IBA
> acknowledge P_Key as part of multicast id. The SA records are equally
> clear. Remember one must specific a P_Key when looking up a
> MCGroupRecord. Then why embed it again in MGID? If seems silly.

        If IBA agrees to the modifications. If it doesn't then we do
        have to fall back to including the P_Key in the mapping. As
        the specificaiton stands at this time (Pg. 726) only MGID is
        used to join a group.

        By embedding the P_Key we ensure the following:

                1. P_Key is a unique value and hence it ensures there
                   is no MGID clash.

                2. Allows an administrator (the human maintainer) to
                   distinguish records that are otherwise the same
                   more easily. The clear distinction of MGIDs will
                   also aid in troubleshooting.

                3. Stay consistent with IBA specification whether it
                   modifies the multicast group management or not.

> >> I proposed to carve out a chunk from the Q_Key space (0x9B1B0000 to
> >> 0x9B1BFFFF) for IPoIB QPs uses. And if you agree to make Q_Key part
> >> of IB MCGroup id, then there is no conflict.
> >
> >        I agree with using the P_Key as part of the mapping. See
> >        earlier mail. Am not convinced on the use of Q_Key. I don't
> >        see any additional advantage as yet.
> 
> The two benefits I see (as I've pointed out to you before) are
> 1. Muliticast name space issue - with Q_Key one can partition the 32-bit
> Q_Key space used in multicast to avoid usage conflicts.
> 
> 2. more granularity in setting IPoIB links. With P_Key-only, one gets
> only one IPoIB link per IB partition.
> 
> If you are still not convinced, please ask yourself what are the down
> sides? Like I've argued before including Q_Key doesn't violate the
> current IBA spec (which already contains numerous inconsistencies in
> this area) and requires no changes.

        The downside in IPoIB terms is having to include another
        configuration parameter - the Q_Key. The advantage doesn't
        seem to outweigh that in my view. 

> >
> >        I don't like the idea of knowing the Q_Key beforehand. Your
> >        suggestion about using a wild-card Q_Key is preferable.
> >        However it implies that the IBTA must define a wild-card
> >        value. The current specification doesn't allow any way for one
> >        to pass this information to the SM/SA.
> 
> Not sure what you mean here. The component mask allows one to wildcard
> any fields during a lookup of SA records.

        We have read the tables 170/171/Pg. 726 differently. Need to
        disambiguate the rules with IBTA.

> 
> >	The Q_Key in the packet
> >        (which must be a priviliged GSI Q_Key and hence one that the
> >        node will always use) is not checked by the SA except to
> >        verify that it is a priviliged Q_Key. The creation of
> >        MCGroupRecord doesn't allow one to specify any paramter but to
> >        specify the MGID.
> 
> Certainly it does. See Table 170 MCGroupRecord on Q_Key.

        I mistyped, I meant to say the MCMemberRecord. The creator
        supplies the Q_Key but the joinee to the multicast group
        cannot. And that is what is pointed to on page 726. Table 170
        is for the creation of the multicast group and not for the
        members that join the group. See Table 171 for that.

> 
> >	I don't know if IB relaxes this restriction
> >        anywhere else.
> >
> >	See Pg 726 in release 1.0a.
> >
> >                "A multicast group can be joined using the SunAdmSet()
> >                method by specifying Add Record in the edit modifier
> >                and the MGID (all other fields are zero)".

> >> provide a "soft" protection and is very easy to reprogram. (It can
> >> all be done by individual host in the OS privilege mode.) E.g. if
> >> servers are grouped according to departmental functions into different
> >> IB partitions that are centrally controlled, each department can still
> >> freely setup/reconfigure its IP subnets using Q_Key.
> >
> >        That sounds interesting but do you need such level of
> >        separation ?
> 
> I don't know what you mean by separation but obviously Q_Key only gives
> some weak, broadcast/multicast separation, but no hard separation.
> 
> >	Also, as I noted above allowing Q_Key
> >        differentiation for MGIDs in IB may be problematic.
> >
> >        Overall, I don't want Q_Key to become another configuration
> >        parameter. The final solution should include a workable
> >        wild-card scheme as you have suggested.
> 
> Yes it should.
> 
> >
> >
> >> 
> >> >
> >> >        Given the above discussion on scope and transient bits the
> >> >        mapping for IPv4 multicast would be:
> >> >
> >> >	FF12:<16-bit P_Key>::< IPv4 multicast address>
> >> >
> >> >        Question: Why do you suggest using only 24 bits of the address
> >> >        when we can use the full 32 bits? 24 bits were used in etheret
> >> >        mapping due to other compulsions.
> >> 
> >> Somehow I got the impression from rfc3171 that only 224/8 is being
> >> actively used for IPv4 multicast. This leaves only 24 (32-8) bits
> >> unique. I need to recheck.
> >
> >No, I don't think that is the case. The address space is still from
> >224/8 to 239.255.255.255.  Will check though. 
> >
> >> 
> >> >
> >> >	Similarly why not use all 1's for broadcast mapping i.e.
> >> >
> >> >	FF12:<16 bit P_Key>:: 255.255.255.255 
> >> >
> >> >	rather than  ::1 as you suggest. Why make it different ?
> >> 
> >> The concept of a special "broadcast" address is being phased out and
> >> replaced by multicast addresses. ::1 corresponds to the lower 24 bits
> >> of the all-node IPv4 multicast address (224.0.0.1).
> >> 
> >> We should pick one IPv4 all-node (local broadcast) address for mapping
> >> into the special MGID. Either 255.255.255.255 or 224.0.0.1. I picked
> >> 224.0.0.1.
> >
> >        Choosing the MGID to correspond to broadcast address makes the
> >        rule simple. The rule states that given an IP address the
> >        corresponding MGID is:
> >
> >	FF12:3334:<P_Key>::<IP multicast address>
> >
> >        In contrast mapping broadcast and all-nodes multicast to the
> >        same MGID introduces an exception to the rule. The exception
> >        says that when mapping the IPv4 broadcast address use the IPv4
> >        all-nodes multicast address for the mapping. At all other
> >        times use the IP address as is.
> 
> I guess I was trying to say that the broadcast address has been
> superceded by the all-node mulitcast address.
> 
> >
> >        Also, one doesn't need to join the broadcast address but does
> >        need an explicit join for receiving IPv4 all-nodes multicast
> >        packets. Why not keep separate MGIDs and keep the separation
> >        rather than mix it all up?
> >
> >> 
> >> >
> >> >        In the case of IPv6 it is important to avoid the IB's Raw IPv6
> >> >        reserved addresses.
> >> >
> >> >	FF12:3333:<P_Key>:<80 Low order bits of IPv6 multicast addresses>
> >> >
> >> >        The use of 0x3333 instead of a random value such as the Q_Key
> >> >        allows one to immediately know that the MGID is an IP MGID.
> >> >        This should aid in administration of the subnets. Secondly if
> >> >        the conflict with Raw IPv6 MGIDs or other non-IP MGIDs is to
> >> >        be avoided it is easier if a specific range can be set aside
> >> >        for IP.
> >> >
> >> >	Therefore, we could modify the IPv4 mapping to be :
> >> >
> >> >	FF12:3334:<P_Key>::<IPv4 multicast/broadcast address>
> >> 
> >> I need to think about this.
> >> 
> >> Jerry
> >
> >Vivek
> >-- 
> >Vivek Kashyap
> >IBM
> >[email-masked] 
> >[email-masked] 
> >503 578 3422 (o)
> 
> 
> _______________________________________________
> IPoverIB mailing list
> [email-masked]
> http://www.ietf.org/mailman/listinfo/ipoverib
> 

-- 
Vivek Kashyap
IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

-- 
Vivek Kashyap
IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

_______________________________________________
IPoverIB mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ipoverib