Subject: RE: [Ipoverib] More musing on address format

From: Yaron Haviv <[email-masked]>
Date: Thu, 21 Mar 2002 13:22:50 +0200

Roy,

If you look at Ethernet VLAN Tag which is the equivelent of P_Key 
Is not provided as part of the hardware address, and shouldn't be due to
security reasons 
The problems you mentions probably don't occur since each VLAN is vied as a
totally separate network, with its own forwarding tables, etc'

The partitioning and group participation should be managed separately 
We also implement SDP, no problem with P_Key, think of it on the host side
as few different NIC's working on different VLAN's

Yaron 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Larsen, Roy K
Sent: Thursday, March 21, 2002 12:13 AM
To: [email-masked]
Subject: RE: [Ipoverib] More musing on address format

> -----Original Message-----
> From: Vivek Kashyap [mailto:[email-masked]]
> Sent: Saturday, March 16, 2002 4:22 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Ipoverib] BOOTP/DHCP chaddr field

<snip>

>
> > >Alas, a LID-based hardware address would be my first choice too.
> >
> > Or keep GID but drop QPN and use SIDR as you've mentioned too.
> >
>
> I am more open to using LID + QPN rather than GID + SIDR.

Our SDP and uDAPL folks have just realized that the P_Key is not a formal
part of the IPoIB hardware address.  Both of these protocols are relying on
IPoIB for address resolution to obtain the GID and they are going to need
side/proprietary interfaces to the IPoIB driver to retrieve the P_Key
associated with an IP address.

Although I realize this is a personal problem for them, a hardware address
that consisted of P_Key + LID + QPN would be ideal from my point of view.
It is 6 bytes (how convenient), fully qualified within the scope of the IB
subnet, and would be a consistent representation of the hardware address at
the administrative level (ipconfig(8), arp(8), etc.).

The latter part of that benefit list is something that may warrant more
discussion.  That is, is there a desire to have a universal mapping of IPoIB
hardware address at the administrative interface?  For Ethernet, all
administrative tools in all OSs present the hardware address in a universal
and consistent fashion.  If we have a 20 byte IPoIB address, there are going
to be several OSs that will need to re-map that to a smaller value for
internal network stack consumption (this is how I stumbled onto the DHCP
issue).  If my re-mapping algorithm is different from others, administrators
will not see the same hardware address listed for the same node from system
to system.

I know there are some on the list that believe the data center will employ
lots of small-grain IB subnets interconnected by [relatively] expensive IB
routers.  I've never found that compelling.  Given all the ambiguity there
currently is with IBA routers, the considerations above just becomes more
food for thought on address format....

Roy

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "Larsen, Roy K" <[email-masked]>
Date: Thu, 21 Mar 2002 09:26:06 -0800

/rkl
Comments embedded
rkl/

-----Original Message-----
From: Michael Krause [mailto:[email-masked]]
Sent: Wednesday, March 20, 2002 10:59 PM
To: [email-masked]
Subject: RE: [Ipoverib] More musing on address format

>the administrative level (ipconfig(8), arp(8), etc.).

The P_Key is derived from the multicast group that one must join to get the 
GID so there is no reason to use it as part of the link address.

/rkl
I am aware of that.  The reason for adding the P_Key was to help support SDP
and uDAPL implementations that have a dependency on IPoIB for address
resolution.  These folks are going to need the P_Key for the subnet and I
was trying to get away from having dozens of IPoIB driver proprietary
interfaces to get it.  I know these protocols are not our concern.  However,
I tend to take a global view in making IBA a success.  The better we all
play together, the more compelling it becomes to install IBA in the data
center.
rkl/

>will not see the same hardware address listed for the same node from system

This is why we have ARP helper modules in some implementations so that the 
format isn't as critical any longer.

/rkl
I'm not sure how this addresses my question.  How important is it that an
administrator see the same IPoIB hardware address displayed when she does an
ifconfig(8) on the host system and an arp(8) on all the others?  Does it
matter that they don't match or even show the same number of bytes in the
address from system to system?  If your suggesting that there would be IPoIB
specific tools that would display all IPoIB information in it's true form,
that's an approach.  It's just a little awkward especially on data center
systems that have both IBA and Ethernet connections.
rkl/

>food for thought on address format....

I doubt there will be many routers used - I also doubt they will be very 
expensive.  I tend to view things as a series of partition domains that may 
span a portion of a subnet or across subnets.  These domains are centrally 
managed and the IP communication is per partition domain.

/rkl
This is a reasonable view and one I would like to support.  I'm just having
trouble implementing it without hacks to the OS and some protocols like
DHCP.  So the cost/benefit analysis puts me on the other side.  Certainly
network administrators are comfortable with multiple subnets in the Ethernet
world and I think they would be just as comfortable with it in the IBA
world.  They could still make their IPoIB subnets partition based - they
just wouldn't configure the partition to cross an IB subnet boundary.
rkl/ 

Mike

---

From: "Jim Pinkerton" <[email-masked]>
Date: Thu, 21 Mar 2002 11:19:19 -0800

Roy,

A quick comment on how I see SDP being plumbed, using IPoIB for address
resolution. 

Since SDP really sits at the transport layer, I'm assuming there is a
structure somewhere which maps the destination IP address to a link
address. This might be the ARP table entry, it might not. In addition,
you need the equivalent of a route table entry to select the local
interface to go out of. Note that this has nothing to do with P_Key - it
is probably a table created through normal routing algorithms. Once the
local interface is know, a variety of host specific methods can be used
to find out fun information about the local interface (P_Key, APM type
issues, etc, etc). 

Thus I don't see any of this being either visible to the user or visible
on the wire - particularly P_Key.

I believe pretty strongly that we haven't seen a compelling reason to
re-open the GID/LID debate - I'm hoping we can put this one to bed. I
agree though that the architecture we have come up with means you can't
simply re-use the existing Ethernet software infrastructure - it was a
pill we had to swallow quite some time ago though.

Jim

-----Original Message-----
From: Larsen, Roy K [mailto:[email-masked]] 
Sent: Thursday, March 21, 2002 9:26 AM
To: 'Michael Krause'; [email-masked]
Subject: RE: [Ipoverib] More musing on address format

/rkl
Comments embedded
rkl/

-----Original Message-----
From: Michael Krause [mailto:[email-masked]]
Sent: Wednesday, March 20, 2002 10:59 PM
To: [email-masked]
Subject: RE: [Ipoverib] More musing on address format

>the administrative level (ipconfig(8), arp(8), etc.).

The P_Key is derived from the multicast group that one must join to get
the 
GID so there is no reason to use it as part of the link address.

/rkl
I am aware of that.  The reason for adding the P_Key was to help support
SDP
and uDAPL implementations that have a dependency on IPoIB for address
resolution.  These folks are going to need the P_Key for the subnet and
I
was trying to get away from having dozens of IPoIB driver proprietary
interfaces to get it.  I know these protocols are not our concern.
However,
I tend to take a global view in making IBA a success.  The better we all
play together, the more compelling it becomes to install IBA in the data
center.
rkl/

>will not see the same hardware address listed for the same node from
system

This is why we have ARP helper modules in some implementations so that
the 
format isn't as critical any longer.

/rkl
I'm not sure how this addresses my question.  How important is it that
an
administrator see the same IPoIB hardware address displayed when she
does an
ifconfig(8) on the host system and an arp(8) on all the others?  Does it
matter that they don't match or even show the same number of bytes in
the
address from system to system?  If your suggesting that there would be
IPoIB
specific tools that would display all IPoIB information in it's true
form,
that's an approach.  It's just a little awkward especially on data
center
systems that have both IBA and Ethernet connections.
rkl/

expensive.  I tend to view things as a series of partition domains that
may 
span a portion of a subnet or across subnets.  These domains are
centrally 
managed and the IP communication is per partition domain.

/rkl
This is a reasonable view and one I would like to support.  I'm just
having
trouble implementing it without hacks to the OS and some protocols like
DHCP.  So the cost/benefit analysis puts me on the other side.
Certainly
network administrators are comfortable with multiple subnets in the
Ethernet
world and I think they would be just as comfortable with it in the IBA
world.  They could still make their IPoIB subnets partition based - they
just wouldn't configure the partition to cross an IB subnet boundary.
rkl/ 

Mike