Subject: RE: [ipoverib] reasons for not using LID ?

From: "Larsen, Roy K" <[email-masked]>
Date: Thu, 17 Jan 2002 17:46:32 -0800

Why do we need to force a one-world view?  The LID advocates believe they
have a marketplace where they can dictate fabric topology and configuration
in such a way that they don't need to consult the SM to resolve an address
if the ARP supplies the LID.  This isn't my marketplace, but I take them at
their word that their needs are valid.  As long as an ARP contains the GID,
my needs are met.  Why can't we enable both and move on!!!!

Roy

-----Original Message-----
From: Arvind Murching [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:57 PM
To: [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

The argument should be the other way around, i.e. fields need to justify
their necessity to be part of a protocol.

Some reasons why not:

- LID values cannot be used across IB subnet boundaries
- A LID cannot be used as part of a static ARP entry
- The SLID is already present in case someone needs it

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]] 
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

By now it is obvious that there are the ones that wants LID's in the ARP
packets

Could the ones that don't give their reasons for why not to
Not just things like "you don't need it", "we want to save 16-bit", etc'
But, reasons for what problems could arouse from having it in the packet
I haven't heard any real argument that would make me consider removing
it
until now
we then will review them and see if it is possible to deal with those
cases,
if not I personally would drop my support in LID's

If people want it for their reasons, the ones that don't also need to do
some explaining

Yaron

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "Vivek Kashyap" <[email-masked]>
Date: Thu, 17 Jan 2002 20:27:21 -0800

<VK> LID is the only part of the link address that is uniquely tied to a
port.  No communication reaches the port without the LID. It is the
hardware address. GID is not needed for communication unless sending across
the IB subnet which is not possible under IPoIB at present unless you
handcraft the infrastructure.

None of the reasons given below are a problem if the LID is included in the
link address.
<VK>

Some reasons why not:

<VK> In what way do you see cross-IB subnet communication being used in the
IPoIB context ?  A reason needs to be justified before it can be
considered. What is the advantage you see in IP subnets spanning IB subnets
?

There is no infrastructure supporting it.  There is no illustration of
advantages in IP spanning IB subnets. It will be a long time before IBTA
can duplicate things the way IP has; routing protocols, routing updates,
routing tables, host lookup and updates etc. etc. And finally you will need
to justify IP subnets spanning IB subnets.

The only cross subnet communication possible is while using proxy-ARP. It
works fine with LID in it. Rather including LID is the only way it will
work at present. Use of a random GID and no LID  without the routing
infrastructure in place will cause the node to depend on manual entries.

This reason is therefore invalid.
<VK>

<VK>

It is trivial to mark an entry incomplete when read from the static entry
table and update it from the SM if you wish to. Static entries are a
special case and you will handle them specially at startup.  It is not
reason enough to dictate the wire protocol.

BTW, it has been mentioned by SM developers that LID will be a constant. It
has been mentioned by GID proponents that they will use multiple GIDs per
IP address. There can be only one LID per port but N GIDs per port. It is
the LID that is constant and not the GID.

Note that if you think that is important you can always ensure that the
GIDs/LIDs are reserved for the set of nodes. Do it like PVCs in case of IP
over ATM. The SM vendors are going to in general ensure that the LIDs
hardly ever change. The SM can easily be asked to really, more than ever,
look after the static entries.

<VK>

<VK> It has been pointed out that it doesn't solve every situation. <VK>

Vivek

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

Yaron

---

From: "Vivek Kashyap" <[email-masked]>
Date: Thu, 17 Jan 2002 20:36:11 -0800

Below...

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 5:31 PM
To: Arvind Murching; [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

My remarks attached

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of
Arvind Murching
Sent: Friday, January 18, 2002 2:57 AM
To: [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

The argument should be the other way around, i.e. fields need to justify
their necessity to be part of a protocol.
[YH] I think many arguments where already raised, now it's the
opposition
turn to put their arguments
[Arvind] The need for a redundant field is not obvious.

<VK> It doesn't become redundant because you want to get it via another
means. Reasons have been given by LID proponents that are very valid.
Reasons given by you against the GID are weak. None of the uses you want
are blocked by including the LID. See the earlier mail in reply to your
mail.
<VK>

Some reasons why not:

- LID values cannot be used across IB subnet boundaries
[YH] you would any way need to store both LID (Router) and GID (node) in
this case somewhere, you wouldn't go to the SM for any new IP packet
wont
you
the routers in this case will any way need to change the LID in the ARP
packets (Proxy ARP and all that)
I believe this case is the one that benefits most from the LID, avoiding
the
long and complex multi-subnet resolution process
[Arvind] Consider the most general case of an IB router routing between
IB subnets. This router will not modify an ARP reply packet to insert
its LID into the payload. However, it will insert its SLID into the IB
header of packets that it forwards. Proxy ARP boxes can do the same,
too.

- A LID cannot be used as part of a static ARP entry
[YH] We didn't say not to have GID, you could still put GID as a static
entry

- The SLID is already present in case someone needs it
[YH] also regular ARP has it, so why not follow that path
[Arvind] This is not an argument for adding redundant fields.

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

Yaron

---

From: "Vivek Kashyap" <[email-masked]>
Date: Thu, 17 Jan 2002 20:45:28 -0800

Roy

<VK>
I totally agree with the sentiment. The architecture needs to provide a
solution that is valid for all solutions possible as long as they don't
break interoperability. The packet can include both the LID and the GID and
things will work fine for both camps.

<VK>

-----Original Message-----
From: Arvind Murching [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:57 PM
To: [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

Some reasons why not:

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

Yaron

---

From: "Vivek Kashyap" <[email-masked]>
Date: Thu, 17 Jan 2002 21:31:59 -0800

My comments in <VK> below.

<VK> It doesn't become redundant because you want to get it via another
means. Reasons have been given by LID proponents that are very valid.
Reasons given by you against the LID are weak. None of the uses you want
are blocked by including the LID. See the earlier mail in reply to your
mail.
<VK>

Some reasons why not:

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

Yaron

---

From: "Vivek Kashyap" <[email-masked]>
Date: Thu, 17 Jan 2002 21:35:20 -0800

<VK> Before we go this path we need to know why we need to support IPoIB
subnets spanning IB subnets. We know that IB architecture is lacking a lot
many things before it can even support such non-routing protocol based
communications i.e. IB routers themselves are not fully specified, P_Key
sharing across IB subnets is not done et.c .etc. IB subnets provide
isolation. Why should they be joined together in IPoIB ? .

[ BTW, even if this is done cross-subnet ARP can work as explained below
and in more detail earlier. However, the WG needn't consider IPoIB subnets
spanning IB subnets unless shown some valid reason for it. ]

<VK>

I am very confused, why will an IB router have to modify an IP ARP packet
at
all.  If there is something in the spec that REQUIRES that I think I would
be in favor of ripping it out and fixing this...

Consider IB routers as looking like Ethernet switches, it is just a layer 2
device that sends packets to the correct collision domain.  I see that as
what an IB router is for the IP layer on top of it.  You would have a hard
time convincing me that you want the router sniffing and modifing ARP
packets.  This would be a huge HUGE layer violation.

Now here is your arguement for not including LIDs in the ARP response, I
want a single ARP response for everyone that sends me an ARP request in my
IP subnet.  If this IP subnet crosses an IB subnet, this LID will not be
valid (or the router has to munge it for me)  This seems a lot of work to
get past the case of a simple lookup to the SM that the IBTA Architecture
would have you do when you are communicating with a new endpoint.

<VK> The SM cannot help you here. It knows nothing of the GID from another
subnet. For that you need to access the routing infrastructure. Considering
the hypothetical case when such an IPoIB subnet exists the ARP module will
have to get the LID from the routing infrastructure. If the LID is included
it will overwrite it with that supplied from the routing infrastructure on
noting a 'foreign' GID-prefix. It will do the same when only the GID is
used. So there is no difference.

In the case of within an IB subnet, the only type of subnet possible, the
non-LID version will have a different step to follow (go to the SA) but the
contains-LID option won't have to do anything. The contains-LID option
doesn't stop anyone from going to the SA though.

<VK>

Bill

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Yaron Haviv
Sent: Thursday, January 17, 2002 5:46 PM
To: 'Arvind Murching'; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

Arvind,

The only real argument you make is implementation in Routers
Since we build such boxes, I could tell you dealing with an ARP in the
router in case of IP span IB is complex anyway
Believe me, the simplification you are offering me creates more work than
help inside the router
As you also stated in the passed there are also very little amount of ARP
packets, so replacing it in one or two paces doesn't change anything

Yaron

-----Original Message-----
From: Arvind Murching [mailto:[email-masked]]
Sent: Friday, January 18, 2002 3:34 AM
To: [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

Below...

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 5:31 PM
To: Arvind Murching; [email-masked]; [email-masked]
Subject: RE: [ipoverib] reasons for not using LID ?

My remarks attached

-----Original Message-----

Some reasons why not:

My 2c
- Arvind

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, January 17, 2002 4:47 PM
To: [email-masked]
Subject: [ipoverib] reasons for not using LID ?

Yaron

---

From: "Deleganes, Ellen M" <[email-masked]>
Date: Fri, 18 Jan 2002 17:24:45 -0800

What I'm not clear about is why require the ARP responder to do the extra
work to figure out what LID to insert, when the correct SLID is returned in
the header? Those who want to use the LID and SL without performing a path
lookup can do so from the ARP response. All of this information is returned
in the completion status of a UD packet, as Jim Pinkerton pointed out.

It seems like from various other messages on this reflector that putting the
LID in the ARP response has potential interoperability issues if IP over IB
happens to span multiple IB subnets. Even though the infrastructure is not
yet there in IB, it seems foolish to build in something that could preclude
being able to do this in the future.

Ellen Deleganes

-----Original Message-----
From: Vivek Kashyap [mailto:[email-masked]]
Sent: Friday, January 18, 2002 4:44 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [ipoverib] reasons for not using LID ?

> 
> On Fri, Jan 18, 2002 at 02:25:12PM -0800, Vivek Kashyap wrote:
> > 
> > The way the router implementation was discussed with the router
> > looking into the packet was not clean.
> > 
> > But the router is not involved at all. See my earlier mail in this
regard.
> > 
> > Consider:
> > 
> > * ARP packet:   LRH+GRH+Payload(LID+GID)
> > 
> > * The router forwards using the GRH.
> > 
> > * The final destination is the ARP receiver - the destined endnode
> > 
> > * The ARP module picks up the packet payload
> > 
> > 	- if GID-prefix local
> > 		copy LID + GID as is
> > 	  else if GID-prefix not local
> > 		use GID-prefix to get the LID from the routing
infrastructure
> 
>   Are you proposing the addition of this LID+GID usage algorithm, or
> something similar, to the IPoIB specification? I'm concerned about 
> interoperability across multiple implementations. If it is decided
> that both LID and GID are included then there needs to be a requirment
> for their usage by a compliant implementation, to insure interoperability.

Things need to interoperate, I am fully behind that. Whenever the
IPoIB subnets get to span the IB subnets this algorithm would become
the requirement. 

> 
>   The fear is that as soon as a major implementation starts using the LID 
> without looking at the GID, all IPoIB subnets will be restricted to a 
> single IB subnet to maintain compatability.

        I don't think so. LID+GID are both used in the ARP packet. At
        present the implementations won't do cross-subnet packets but
        when they do the above algorithm will have to be followed.

> 
>   If LID+GID can be used anyway an implementation wishes, then the
> spec might as well come out and say that an IP subnet cannot span
> multiple IB subnets. I'm not saying that this is a bad thing, only
> that the intention should be made clear to avoid future confusion. 

        Do you mean that the spec should say that at present the IPoIB
        subnets don't span IB subnets? The architecture draft says
        that much(I have received some comments on fine-tuning the
        words but the meaning remains the same). The encapsulation
        draft can indicate that the GID-prefix needs to be checked.

        I am not in favour of solving the span IB subnet problem in
        general but the above doesn't block that.

Vivek

> -Libor
> --
> Libor Michalek
> [email-masked]
> Topspin Communications.

-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)