Subject: RE: [ipoverib] moving forward with the open Issues - end node configuration

From: Yaron Haviv <[email-masked]>
Date: Wed, 30 Jan 2002 00:54:05 +0200

Jerry,

I agree that we could start with your proposal using the MCGroupRecord,
maybe in the future we could add a service record or something, like Aditya
Dube suggested

The only problem is that you cannot define an MTU of 1500 this way
I would like to have this option, in a way that could be configured somehow
It doesn't have to be the default choice, just an optional choice

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
H.K. Jerry Chu
Sent: Sunday, January 27, 2002 10:57 PM
To: [email-masked]; Yaron Haviv
Subject: RE: [ipoverib] moving forward with the open Issues - end node
configuration

Yaron,

>I guess you just missed my point; it was not to say that manual
>configuration is good, and we need to avoid the SM
>I just said that the process of agreeing on an automatic solution is not
>simple, and I would like to get an RFC first, than quarrel over this
>mechanism

Disagree. See my other mail. IETF already provides a number of mechanisms
for autoconfiguration that apply to IPoIB as well as any IPoverFoo. All
we need to solve here is to autoconfigure IPoIB links.

>Not to mention that it might involve additional standardization from IBTA
>(e.g. determining ServiceID's, etc') which wont make it faster
>This is the same reasoning like abandoning the capability bits issue today
>
>As for your proposal I'm not so sure that it is so efficient going over all
>the P_Key's defined, in some cases there could be more than a few
>I don't have strong feelings against it, just rather see something more
>efficient, if possible

Note first that this link configuration process only occur once per node
reboot. The efficiency concern IMHO is even less convincing here than the
one that argues for inlcuding LID in ARP. Second, there is nothing that
prevents implementations from supplementing additional configuration
files on an endnode that cache/specify IPoIB P_Keys, for example. These
local configuration practices don't present any interoperability problem,
therefore does not need to be standardized.

>On the same query to the SM we might also get more info on that IP subnet
>that will ease the configuration, and shorten the booting process
>
>Regarding the 1500 MTU, I know all about PMTU
>The 2K adds some load on the network, which also depends on the topology
and
>traffic characteristics
>It could be acceptable by many people
>I would just want to enable setting the values to a number lower than 2K,
if
>the admin requires so
>I'm not against setting 2K values for anyone that wants it, or even as the
>default magic number for IPoIB as long as smaller numbers are also valid
>And from your answer it looks like you agree to that too

Yup. I've been against some earlier proposals to designate a fixed MTU size
(e.g. 2K) for IPoIB because I don't think that is necessary. This is covered
in the link and multicast draft by using the MTU field in the MCGroupRecord.
But an issue just came up to me if one wants to support 1500 MTU. Contrary
to the "IP MTU" that allows one to specify an arbitrary size as long as
it's not greater than the link MTU, the "IB MTU" in the MCGroupRecord
field only allows fives choices from 256 to 4096.

We can either
1. punt it, since other IPoverFoo doesn't support the use of arbitrarily
configured MTU size either. (How does bridge vendors solve this problem when
bridging FDDI with Ethernet?)

or
2. find some place to store the IP MTU (ServiceRecord? I prefer not.)

Any suggestion?

Jerry

>
>Yaron
>
>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]]On Behalf Of
>H.K. Jerry Chu
>Sent: Sunday, January 27, 2002 1:25 AM
>To: [email-masked]; Yaron Haviv
>Subject: RE: [ipoverib] moving forward with the open Issues - end node
>configuration
>
>Yaron,
>
>>I agree that some method should be specified in the future
>>However it is not so trivial to do it right now
>>a. There are issues between IBTA & IETF on things like service records
etc'
>>b. You could have few VLAN's on the same IB subnet, how would you
>>distinguish between them, etc' (maybe a service ID range)
>
>For each P_Key in a port's P_Key table, simply do a lookup to SA for
>an IPoIB MGID defined in the link and multicast draft. If the lookup
>succeeds, the IB partition defined by the P_Key will be used to host
>IPoIB, otherwise it won't. Isn't that pretty simple?
>
>>
>>I would tend to go for an RFC with a static configuration, with known
>>defaults like:
>>The default P_Key value, 1500 MTU
>>Work on a follow on proposal either in IBTA or IETF to address the
>>automation
>
>Static configuration is a thing of the past. Perhaps the motivation here
>is to bypass SM/SA again (like the LID proposal)? I can sympathize with
>the concern that SM/SA may be so complex that having a dependency on
>it is a bit risky. But let's see how feasible a bypass is. You can
>hardwire MTU, Q_Key, TClass... But you really need a broadcast mechanism
>(simulating broadcast by unicast is real messy) so you've got to have a
>MLID. Are you going to hardwire that too?
>
>If the concern is about the complexity of SM/SA, why not just dummy up
>a SM/SA implementation and hardwire all your favorite parameters there?
>This way we don't need to muck with IPoIB standard.
>
>>
>>You probably wonder why 1500, some of the reasons originally for
>determining
>>that IPoIB supporting devices will have minimum MTU of 2K was to maintain
>>similarity with Ethernet
>>Since in IB you have either 1K or 2K, 1500 falls in the 2K domain, so the
>>device must support 2K, not to say the MTU should by 2K
>>
>>Why 1500 again, lets say you have an Ethernet to IB bridge/router it will
>>have to fragment/reassemble the packets
>>This will create a lot of overhead on both the hosts and the routers that
>>will significantly affect performance
>
>There are two very different cases that you mentioned here, bridge and
>router. For the latter, no sweat, PMTU discovery takes care of it so
>you don't need to muck with individual link MTU. (That's the whole
>purpose of PMTU.) For bridges, no problem, just put your favorite
>MTU value in the MCGroupRecord when you set up an IPoIB link to bridge
>with an Ether segment.
>
>Jerry
>
>>
>>(it might be 1400 & something)
>>
>>Yaron
>>
>>-----Original Message-----
>>From: [email-masked] [mailto:[email-masked]]On Behalf Of
>>Aditya Dube
>>Sent: Friday, January 25, 2002 9:17 PM
>>To: [email-masked]
>>Subject: RE: [ipoverib] moving forward with the open Issues - end node
>>configuration
>>
>>
>>
>>The current IPOverIB spec requires that the P_Key and MTU be configured
>>on each node.
>>
>>The IPOverIB spec should provide a default method for an end-node to
>>come online with no IPoIB-specific configuration.
>>
>>During initialization, a node could query an IPOverIB specific
>>ServiceRecord (Table 164, pg 685). This query will return a P_Key and
>>ServiceSpecificFlags. The ServiceSpecificFlags can specify whether 2048
>>or 4096 is the link MTU.
>>
>>The ServiceRecord's ServiceID should have a default value specific to
>>IPOverIB. This number could be 0xFFFF:4000 in accordance with vol3
>>section on "ServiceID Namespace" (Table 128, pg 215).
>>
>>The ServiceRecord needs to be created just as the all-node or broadcast
>>MCGroup is created, i.e. by the first node on the network.
>>
>>If the Network Admin wishes to put the end-node on a different P_Key,
>>then the Admin may override the values returned by the ServiceRecord.
>>That is outside the scope of my proposal.
>>
>>
>>Aditya Dube
>>
>>
>>
>>_______________________________________________
>>IPoverIB mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>>
>>
>>_______________________________________________
>>IPoverIB mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib
>

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 29 Jan 2002 22:40:14 -0800 (PST)

Yaron,
        The MCGroupRecord provides the IPoIB link MTU not IP MTU. The
        IP MTU is set at the interface - by default implmentations
        would set it to the link MTU itself.

        If one wishes to set the IP MTU to 1500 one may do so. It
        would be wise to ensure that the IPoIB link supports 2K(set in
        MCGroupRecord) in this case otherwise every interface driver
        on the link must provide fragmentation/reassembly of the
        packets. 

        In IPv6 one can use router advertisement link option to set
        the IP MTU. I don't think there is a method for advertising a
        common MTU (though path MTU discovery will work within a
        subnet too) in a similar manner to IPv6 when using IPv4.

        Anyway why should this be any different from IP over any other
        media? In my opinion if some other solution is needed then
        that should be dealt with in terms of IP itself and not
        limited to IPoIB.

Vivek

-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

---

From: Michael Krause <[email-masked]>
Date: Thu, 31 Jan 2002 08:23:08 -0800

>         would set it to the link MTU itself.

Assumption is that the multicast PMTU while at least on one or more IB 
subnets is set to 2KB or 4KB (function of what the underlying fabric 
supports).  For unicast, the normal IP PMTU discovery process is used and 
nothing new is required to be specified.

Mike

---

From: Yaron Haviv <[email-masked]>
Date: Thu, 31 Jan 2002 20:36:34 +0200

Mike,

If you look at my last email from yesterday
I acknowledge that, and say that we need to look into in the future and not
in this RFC, so this issue isn't open anymore
The only open issue currently is the Ethertype field

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Michael Krause
Sent: Thursday, January 31, 2002 6:23 PM
To: [email-masked]
Subject: Re: [ipoverib] moving forward with the open Issues - end node

Mike

---

From: [email-masked]
Date: Fri, 01 Feb 2002 12:19:43 +0900

>Date: Thu, 31 Jan 2002 20:36:34 +0200
>From: Yaron Haviv <[email-masked]>

Please let me confirm this default MTU issue.

If Ethertype is not in payload, default IP-MTU is 2048.
Otherwise (=if Ethertype is in payload),
 default IP-MTU is not 2048.
It's 2048 - EterhType field length. (4? probably not 2)

>nothing new is required to be specified.

Thanks,
------------------------------------------------------------------------------
Akihiko Imai/Development Dept.5, Infrastructure Systems Division, Fujitsu Ltd.
  e-mail: [email-masked]  TEL: +81-044-754-3456 (ext)7113-8706
  address: 4-4-1 Kamiodanaka Nakahara-ku, Kawasaki 211-8588, Japan