Subject: RE: [ipoverib] moving forward with the open Issues - end node configuration

From: "Aditya Dube" <[email-masked]>
Date: Fri, 25 Jan 2002 11:16:57 -0800

The current IPOverIB spec requires that the P_Key and MTU be configured
on each node. 

The IPOverIB spec should provide a default method for an end-node to
come online with no IPoIB-specific configuration.

During initialization, a node could query an IPOverIB specific
ServiceRecord (Table 164, pg 685). This query will return a P_Key and
ServiceSpecificFlags. The ServiceSpecificFlags can specify whether 2048
or 4096 is the link MTU. 

The ServiceRecord's ServiceID should have a default value specific to
IPOverIB. This number could be 0xFFFF:4000 in accordance with vol3
section on "ServiceID Namespace" (Table 128, pg 215).

The ServiceRecord needs to be created just as the all-node or broadcast
MCGroup is created, i.e. by the first node on the network. 

If the Network Admin wishes to put the end-node on a different P_Key,
then the Admin may override the values returned by the ServiceRecord.
That is outside the scope of my proposal.

Aditya Dube

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "Bill Strahm" <[email-masked]>
Date: Fri, 25 Jan 2002 11:56:49 -0800

Quick comment,

I thought Vol 3 was never an approved IBTA specification and was dropped...

I definately agree with you that a zero-config type environment would be
nice for IPoIB traffic, however there might be some other alternatives.

I can see DHCP extensions
I can see Service Records (can the IETF define these Mike Krause ?) like you
suggested
I can see manual configuration - technically not zero config at this point
I can see defaults in the protocol - Though SHOULD use 2K MTU and a P_Key of
mikimitdigoat or I'll even let you use a ascii representation of your
initials if you are vain <G> (Just don't tell anyone)

Do you think you could write a proposal describing how any/all of these
methods that you think will work and any others you might come across...

Thank you
Bill

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Aditya Dube
Sent: Friday, January 25, 2002 11:17 AM
To: [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Aditya Dube

---

From: Yaron Haviv <[email-masked]>
Date: Sat, 26 Jan 2002 01:00:27 +0200

I agree that some method should be specified in the future
However it is not so trivial to do it right now
a. There are issues between IBTA & IETF on things like service records etc'
b. You could have few VLAN's on the same IB subnet, how would you
distinguish between them, etc' (maybe a service ID range)

I would tend to go for an RFC with a static configuration, with known
defaults like:
The default P_Key value, 1500 MTU
Work on a follow on proposal either in IBTA or IETF to address the
automation

You probably wonder why 1500, some of the reasons originally for determining
that IPoIB supporting devices will have minimum MTU of 2K was to maintain
similarity with Ethernet
Since in IB you have either 1K or 2K, 1500 falls in the 2K domain, so the
device must support 2K, not to say the MTU should by 2K

Why 1500 again, lets say you have an Ethernet to IB bridge/router it will
have to fragment/reassemble the packets
This will create a lot of overhead on both the hosts and the routers that
will significantly affect performance

(it might be 1400 & something)

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Aditya Dube
Sent: Friday, January 25, 2002 9:17 PM
To: [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Aditya Dube

---

From: "Bill Strahm" <[email-masked]>
Date: Fri, 25 Jan 2002 15:37:17 -0800

The fact that there are transports in the middle of the network that do not
support larger MTUs is not a reason to lower your MTU value...

TCP will find the correct Path MTU, and UDP doesn't care.  Therefor why
should I limit my MTU on the local subnet because someone MIGHT have an MTU
much lower.  I would rather state the largest local wire MTU (2K - overhead)
and let the protocols lower it automatically...

Bill

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Yaron Haviv
Sent: Friday, January 25, 2002 3:00 PM
To: 'Aditya Dube'; [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Yaron

-----Original Message-----

Aditya Dube

---

From: "Sukanta Ganguly" <[email-masked]>
Date: Sat, 26 Jan 2002 10:33:07 -0800

Bill,
  By forcing a larger MTU value an overhead in injected in the path, where nodes not in the position to support larger MTU have to fragment/defrag the packets. This is processing overhead. From the position of sending the packet it may not seem like one. But as far as packet being delivered from one end to the other the overhead is injected. The Path MTU mechanism is to determine the lowest MTU for all the individual links and setting that to be your MTU. No packet frag/defrag will happen and hence not buffereing expected in the path for any intermediate nodes. 

SG

*********** REPLY SEPARATOR  ***********

On 1/25/2002 at 3:37 PM Bill Strahm wrote:

>From: ipove  [email-masked] [mailto:[email-masked]]On Behalf
>Of Yaron Haviv

---

From: "Bill Strahm" <[email-masked]>
Date: Sat, 26 Jan 2002 14:02:28 -0800

I know all of this...

No answer this.

In the case of TCP, PMTU will correctly determine the PATH MTU, not the
local link MTU and you don't have frag/reassembly problems in the network

In the case of UDP, you will have to handle these anyway, because UDP
packets are just as big as they are...

Now tell me what you have bought me by reducing the local MTU below its
maximum ?

Bill

-----Original Message-----
From: Sukanta Ganguly [mailto:[email-masked]]
Sent: Saturday, January 26, 2002 10:33 AM
To: Bill Strahm; Yaron Haviv; 'Aditya Dube'; [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Bill,
  By forcing a larger MTU value an overhead in injected in the path, where
nodes not in the position to support larger MTU have to fragment/defrag the
packets. This is processing overhead. From the position of sending the
packet it may not seem like one. But as far as packet being delivered from
one end to the other the overhead is injected. The Path MTU mechanism is to
determine the lowest MTU for all the individual links and setting that to be
your MTU. No packet frag/defrag will happen and hence not buffereing
expected in the path for any intermediate nodes.

SG

---

From: "Sukanta Ganguly" <[email-masked]>
Date: Sat, 26 Jan 2002 19:34:39 -0800

Response is embedded between <B-SG> and <E-SG>. Hope my explanation and reasonings are made clear.

Thanks
SG

>No answer this.

<B-SG>
What do you mean by the above ???
<E-SG>

>local link MTU and you don't have frag/reassembly problems in the network

<B-SG>
We both agree on this along with the world. 
<E-SG>

>packets are just as big as they are...

<B-SG>
OK.
<E-SG>

>Now tell me what you have bought me by reducing the local MTU below its

<B-SG>
So instead of asking me the question of what I have bought you by allowing the MTU selection to comply with the PMTU like approach, I would rather think that you should have concluded that just like UDP dumps packets on to the IP layer, IPoIB will do the same, then there is a point I see. I do not agree with the reasoning, but I do acknowledge that there must be some reasons why such a conclusion was made and I guess, not only me but others too would be asking this question as to why such a conclusion.

Coming back to the issue in hand, are we concluding that we don't really care about the potential issues of frag/defrag that may happen at layer three and dump what ever size of the IB PDU, that suits our need due to our IB protocol constraints/design/choices etc?

We will definitely see fragramentation at layer three for PDU sizes over 1500 bytes.  
<E-SG>

---

From: "Bill Strahm" <[email-masked]>
Date: Sat, 26 Jan 2002 20:10:08 -0800

Ok... Some comments inline
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Sukanta Ganguly
Sent: Saturday, January 26, 2002 7:35 PM
To: Bill Strahm; Yaron Haviv; 'Aditya Dube'; [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Response is embedded between <B-SG> and <E-SG>. Hope my explanation and
reasonings are made clear.

Thanks
SG

<B-SG>
What do you mean by the above ???
<E-SG>
<Bill>
Missing w, should be Now answer this
</Bill>

<B-SG>

<B-SG>
OK.
<E-SG>

>Now tell me what you have bought me by reducing the local MTU below its

<B-SG>
So instead of asking me the question of what I have bought you by allowing
the MTU selection to comply with the PMTU like approach, I would rather
think that you should have concluded that just like UDP dumps packets on to
the IP layer, IPoIB will do the same, then there is a
<Bill>
IPoIB DOES NOT Dump packets onto IP, it dumps IP packets onto a layer 2.
For this reason, why should I limit the size of my layer 3 packet because
some other layer 2 happens to like a 1500 byte MTU.
</Bill>
point I see. I do not agree with the reasoning, but I do acknowledge that
there must be some reasons why such a conclusion was made and I guess, not
only me but others too would be asking this question as to why such a
conclusion.

Coming back to the issue in hand, are we concluding that we don't really
care about the potential issues of frag/defrag that may happen at layer
three and dump what ever size of the IB PDU, that suits our need due to our
IB protocol constraints/design/choices etc?

We will definitely see fragramentation at layer three for PDU sizes over
1500 bytes.
<E-SG>
<Bill>
I disagree with this statement.  There are MANY other protocols that use an
MTU larger than 1540 bytes.  If we route over one of those networks we will
have fragmentation, TCPs PMTU will handle this for us, if we do not (we go
from one IB subnet, over a WAN protocol that has a large MTU onto another IB
subnet that has a 2K MTU) we will not have any additional fragmentation.
Why should we limit IB as a layer 2 because Ethernet has a 1500 byte MTU
limitation ?

Also note that in IPv6 the network is NOT allowed to fragment packets.
</Bill>