Subject: RE: [Ipoverib] BOOTP/DHCP chaddr field

From: "H.K. Jerry Chu" <[email-masked]>
Date: Fri, 15 Mar 2002 19:00:49 -0800 (PST)

>> > It is clear to me that we have a fundamental problem.  If 
>> DHCP and BOOTP
>> > were fringe protocols, I wouldn't be too concerned.  
>> However, they are
>> > fundamental to current network infrastructures.
>> > 
>> > The solutions explored so fare require changes in both 
>> client and server
>> > DHCP/BOOTP applications.  My business model does not give 
>> me the power to
>> > dictate the DHCP/BOOTP servers that my customers will use 
>> in their data
>> > center.  Given the initial market for IPoIB, there is very 
>> little incentive
>> > for DHCP/BOOTP server suppliers to modify their offerings 
>> to be IPoIB aware.

Agreed.

>> 
>> The only change a server has to do is to recognise the hardware type
>> and that is acceptable. Such a change is *required* for any new media.

I consulted a couple of DHCP folks and was told the "htype" is used
for comparision purpose but is never interpreted. Therefore everytime
a new link type is added it doesn't and shouldn't require changes to
host OS'es DHCP code.

>> The broadcast flag is an integral part of the protocol and so nothing
>> needs to be done. So, the outline I have suggested in earlier mails
>> should suffice.

If the above is true (no DHCP change is otherwise required), clients
setting the broadcast flag in DHCP requests may be a workaround for
this problem. Note that DHCPv6 doesn't have this problem. If you believe
IPv4 will go away someday then this workaround is only temporary and
perhaps is acceptable.

>
>Vivek,
>
>When your original mail said that this method was frowned on generally, I
>got the impression you were discouraging use of the BROADCAST bit in the
>flags field.  I agree that if the client sets this bit on all server
>requests, the server's core protocol processing of the frame would not
>change which is my fundamental concern.  How a server application sends a
>raw frame to the link is an OS implementation detail which very well may
>need to be IPoIB aware.  The fact that they will always be broadcast will
>make that much more straightforward.  But it needs to be localized at that
>level and not in core DHCP protocol processing.
>
>This still requires DHCP clients to be IPoIB aware at the protocol machine
>level but I think that is more manageable.

I think we need to first get a better understanding of the various DHCP
implementations. I don't see any changes to DHCP code, client or server
side for IPoIB can be possibly manageable.

>
>> 
>> It would be nice to use the LID instead of the GID as you say but we
>> we need to first weigh the use of broadcast bit against revisiting the
>> format of the link address. We have had lengthy discussions on some of
>> the issues.
>
>Alas, a LID-based hardware address would be my first choice too.

Or keep GID but drop QPN and use SIDR as you've mentioned too.

>
>> 
>> Is the broadcast bit acceptable ? Look for alternatives only if it is
>> not acceptable.
>
>Your emails have discussed broadcast in several contexts.  I have assumed
>your favored approach is now to have the client set the BROADCAST bit in the
>"flags" field of all server requests.  This works better than a conditional
>method that depends on IB subnet boundaries and embedding QPNs in the GID.
>
>I am still very uncomfortable with an inconsistent representation of an
>IPoIB hardware address - especially since we can fix it. (What would you
>expect from someone who is also uncomfortable with peeking into user data to
>infer LSAP?) :^)  But if I'm the only one, the method outlined above can get
>by. 
>
>Where does this IPoIB interoperability semantic for DHCP/BOOTP get
>documented?  I assume it belongs in one of the IPoIB drafts.

Sure.

Jerry

>
>Roy
> 
>> 
>> Vivek
>> 
>> > 
>> > For these reasons, I believe we need to reconsider the current IPoIB
>> > hardware address format and devise a structure that is 
>> uniform for all IP
>> > protocols.  To that end, I can see two possibilities.  One 
>> is to reduce the
>> > scope of an IPoIB link to an IB subnet and employ the LID 
>> instead of the GID
>> > for port identification.  The use of QPN and capability 
>> flags can remain the
>> > same.  If the group feels that the IPoIB link should retain 
>> its global
>> > scope, we can still use the GID but use SIDR to obtain QPN 
>> and capabilities.
>> > 
>> > Others?
>> > 
>> > Roy
>> > 
>> > -----Original Message-----
>> > From: Vivek Kashyap [mailto:[email-masked]]
>> > Sent: Wednesday, March 13, 2002 1:07 AM
>> > To: [email-masked]
>> > Cc: [email-masked]
>> > Subject: Re: [Ipoverib] BOOTP/DHCP chaddr field
>> > 
>> > 
>> > 
>> > > >        We could always have the DHCP server broadcast a 
>> reply always
>> > > >        - use the GID from the link-layer address in 
>> chaddr to occupy
>> > > >        all 16 bytes. The client receives the broadcast reply and
>> > > >        picks the one containing its GID. The broadcast option is
>> > > >        frowned on generally.
>> > > 
>> > > That's good if multiple interfaces are not defined per HCA port.
>> > 
>> >         In this such a case the client identifier option 
>> must be used
>> >         with a value unique to the interface. A GUID is 
>> unique to the port 
>> >         but a per interface GID, or per interface FQDN, or even a
>> >         GID+QPN (if the interface always uses that value) 
>> >         or a unique whatever can be used. Note that the same unique
>> >         identifier across boots can also be derived by including the
>> >         GUID+timestamp of first use+local-unique value. This
>> >         identifier can be written to local permanent storage for
>> >         consistent use across boots/leases.
>> > 
>> > 	The basic issue is : 
>> > 		
>> > 		(i) a unique identifier
>> > 
>> > 		(ii) the client must receive the reply
>> > 
>> >                 the client can receive the reply as a broadcast and
>> >                 then sift through to pick the one that 
>> applies to it -
>> >                 (say) using its own GID. Otherwise the DHCP server
>> >                 must be told the 'MAC'/link-layer address to unicast
>> >                 the reply to.
>> > 
>> > 	Therefore simplest solution is:
>> > 
>> >                 'chaddr' is used to include the GID. The Broadcast
>> >                 flag is always set.
>> > 
>> >                 or 
>> > 		
>> > 		a slightly different solution but still using the
>> >                 broadcast flag is to use the 
>> client-identifier option
>> >                 for the unique identifier.
>> > 
>> > 
>> >         The alternative is to have the replies unicast back to the
>> >         client. For this the DHCP server needs to know the
>> >         link-address and therefore the QPN.
>> > 
>> >         In the earlier mail I added this in the 
>> client-identifier but
>> >         that will not work - I re-read the RFC and it makes the
>> >         client-identifier an opaque value. The chaddr is the
>> >         hardware/MAC address. So, the addressing information must be
>> >         included in the chaddr field and not in the client 
>> identifier
>> >         field. The client identifier field can be the 
>> unique value for
>> >         the DHCP database lookup at the server. 
>> > 
>> >         A solution can be to include the QPN and the lower 
>> 64-bits of
>> >         the GID(the GID prefix is known to all) or the LID itself.
>> >         However such a solution can work in a single IB subnet only.
>> >         If the IPoIB subnet spans IB subnets then this 
>> information is
>> >         not enough - one needs the GID prefix too.
>> > 
>> >         So, when IPoIB subnet spans IB subnets the broadcast flag
>> >         needs to be used.  
>> > 
>> > 
>> > > >
>> > > >        IPoIB_MCAST describes the broadcast MGID. This 
>> MGID is derived
>> > > >        by including the P_Key of the subnet. Thus the 
>> P_Key must be
>> > > >        known to the client before it can broadcast. The 
>> P_Key is a
>> > > >        configuration parameter.
>> > > 
>> > > Yes. MGID's P_key is also a good (i.e. non-transient) 
>> identifier, I also
>> > think.
>> > > It can be an additional identifier onto port's GID (GID 
>> prefix + GUID
>> > index 0).
>> > 
>> > Not sure of that. I was saying that the P_Key is already 
>> known to the
>> > sender. Hence it need not be included. The receiving DHCP 
>> server host
>> > will accept the packet only if it is on the same broadcast MGID or a
>> > relay agent is on the same broadcast MGID. So, in my view P_Key
>> > doesn't enter the picture.
>> > 
>> > > 
>> > > >3. Usage of DHCP message fields
>> > > >
>> > > >        Following rules should be used when a DHCP 
>> client is run over
>> > > >        an IPoIB network:
>> > > >
>> > > >	'htype' (hardware address type) MUST be 32 
>> > > >	'hlen' (hardware address length) MUST be 16
>> > > 
>> > > If RFC2131(DHCP) is not extended, that's right,
>> > > of course, DHCP implementations must be current RFC2131 compliant
>> > >  (i.e. compliant to variable "chaddr length").
>> > 
>> > I am not sure what you mean by 'compliant to variable 
>> chaddr length'.
>> > DHCPv6 may take larger values but DHCP-2131 keeps it at 16.
>> > 
>> > 
>> > Vivek
>> > -- 
>> > Vivek Kashyap
>> > Linux Technology Center, IBM
>> > [email-masked] 
>> > [email-masked] 
>> > 503 578 3422 (o)
>> > 
>> > _______________________________________________
>> > IPoverIB mailing list
>> > [email-masked]
>> > https://www1.ietf.org/mailman/listinfo/ipoverib
>> > 
>> > _______________________________________________
>> > IPoverIB mailing list
>> > [email-masked]
>> > https://www1.ietf.org/mailman/listinfo/ipoverib
>> > 
>> 
>> 
>> -- 
>> Vivek Kashyap
>> Linux Technology Center, IBM
>> [email-masked] 
>> [email-masked] 
>> 503 578 3422 (o)
>> -- 
>> Vivek Kashyap
>> Linux Technology Center, IBM
>> [email-masked] 
>> [email-masked] 
>> 503 578 3422 (o)
>> 
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vivek Kashyap <[email-masked]>
Date: Sat, 16 Mar 2002 04:22:13 -0800 (PST)

> a new link type is added it doesn't and shouldn't require changes to

Yes, it is not interpreted - the real use is when the arp cache is
primed. With the broadcast bit set there is no need for updating the
arp cache. So nothing need change in the server.

> IPv4 will go away someday then this workaround is only temporary and

Yes to all the above. 

> side for IPoIB can be possibly manageable.

The server is realy not effected. The client is configured to set the
broadcast bit based on the need (IP layer can/can't accept unicast
when no address is assigned). Thus, as far as I can tell, the 'common'
code need not change for the client either. IPoIB specific changes to
add the right kind of client identifier etc. will have to be made
though. 

> Or keep GID but drop QPN and use SIDR as you've mentioned too.

I am more open to using LID + QPN rather than GID + SIDR.

For example a simple issue with SIDR and DHCP:

        As noted by Akihiko, and I agree, there may be multiple IPoIB
        interfaces over the same port but these interfaces would have
        different QPNs. Greater than one IPoIB interface per port and
        the server or the client side CM have no way to determine the
        QPN to send back; server doesn't have a good distinguishing
        value to give to the client CM.

Vivek