Subject: RE: [ipoverib] moving forward with the open Issues - end node

From: "H.K. Jerry Chu" <[email-masked]>
Date: Tue, 26 Feb 2002 00:23:46 -0800 (PST)

I fail to see the need for specifying a "default" MTU for IPoIB, or
limiting the MTU choices to >= 2048. Can someone present a case for
these arguments?

IPoIB must always go through some link setup procedure in order to
have a multicast group for broadcast purpose. THe link MTU size will
be chosen then. What is the value of designating a "default" MTU then?
Note that both drafts mention the benefit of having a large MTU, but
leave the final choice at the hand of the adminstrator.

If the concern of allowing devices of smaller MTU is the difficulty
it may present for IB <-> Ethernet bridging or routing, router link
MTU option and PMTU should alleviate the headache from the MTU
discrepancy between the two, right?

Jerry

>Date: Sat, 23 Feb 2002 20:50:09 -0800
>From: "Sukanta Ganguly" <[email-masked]>
>To: "Grun, Paul" <[email-masked]>, [email-masked]
>Subject: RE: [ipoverib] moving forward with the open Issues - end node
>Mime-Version: 1.0
>Content-Transfer-Encoding: 8bit
>X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id 
XAA14313
>X-Mailman-Version: 1.0
>List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>X-BeenThere: [email-masked]
>
>I thought so too. I was beaten down to a pulp by Bill on this one. I do agree 
with him after understanding his reasonings.
>
>SG
>
>*********** REPLY SEPARATOR  ***********
>
>On 2/7/2002 at 4:15 PM Grun, Paul wrote:
>
>>I thought that there was consensus early on that we would assume that an
>>IBA
>>fabric supporting IPoverIB would provide a minimum MTU of 2K bytes?  Didn't
>>we agree not to pursue the smaller MTUs?
>>-paul
>>
>>> -----Original Message-----
>>> From: Vivek Kashyap [mailto:[email-masked]]
>>> Sent: Thursday, February 07, 2002 4:12 PM
>>> To: [email-masked]
>>> Cc: [email-masked]
>>> Subject: Re: [ipoverib] moving forward with the open Issues - end node
>>> 
>>> 
>>> > 
>>> > Yaron, 
>>> > Thanks for your response.
>>> > 
>>> > BTW, I think "what IPoIB-default-MTU should be" is an open issue.
>>> > right ?
>>> 
>>> Akihiko,
>>> 
>>>         We don't have a default MTU; the administrator is free to set
>>>         any of the 5 possible values (256,512,1024,2048,4096 bytes)
>>>         based on the capacity of the CAs/switches in the subnet.
>>> 
>>>         In general using a minimum link MTU (and hence choosing the
>>>         'right' equipment) of 2048 bytes makes sense. This link MTU
>>>         allows for the IP MTU to be set to 1500 (ethernet
>>>         internetworking) or 1280 (IPv6 minimum) if so desired. The
>>>         smaller MTUs are not optimal.
>>> 
>>>         No default though except in terms of :
>>> 
>>>         If No-ethertype in payload
>>> 
>>> 		IP MTU = IPoIB link MTU 
>>> 
>>> 	else
>>> 		IP MTU = IPoIB link MTU - sizeof('ethertype')
>>> 
>>> 
>>> 
>>> Vivek
>>> 
>>> > 
>>> > >-----Original Message-----
>>> > >From: [email-masked] 
>>[mailto:[email-masked]]On Behalf Of
>>> >[email-masked]
>>> >Sent: Friday, February 01, 2002 5:20 AM
>>> >To: [email-masked]
>>> >Cc: 'Michael Krause'; [email-masked]
>>> >Subject: Re: [ipoverib] moving forward with the open Issues - end node
>>> >
>>> >>Date: Thu, 31 Jan 2002 20:36:34 +0200
>>> >>From: Yaron Haviv <[email-masked]>
>>> >>Subject: RE: [ipoverib] moving forward with the open Issues - end node
>>> >>Mike,
>>> >>
>>> >>If you look at my last email from yesterday
>>> >>I acknowledge that, and say that we need to look into in the future and
>>not
>>> >>in this RFC, so this issue isn't open anymore
>>> >>The only open issue currently is the Ethertype field
>>> >>
>>> >>Yaron
>>> >
>>> >Please let me confirm this default MTU issue.
>>> >
>>> >If Ethertype is not in payload, default IP-MTU is 2048.
>>> >Otherwise (=if Ethertype is in payload),
>>> > default IP-MTU is not 2048.
>>> >It's 2048 - EterhType field length. (4? probably not 2)
>>> >
>>
>>_______________________________________________
>>IPoverIB mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>>
>>_______________________________________________
>>IPoverIB mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Yaron Haviv <[email-masked]>
Date: Tue, 26 Feb 2002 10:41:23 +0200

Jerry,

I agree that there is no real need for a default MTU
This could be set up on site, or done automatically by some management SW,
and allow override by manual configuration

I'm not sure that the assumption that this MTU covers also Unicast traffic
is valid; I think we agree on that
Especially when an RC based capability be introduced supporting large MTU's
The MCGroup MTU will probably serve for the Multicast, and for the default
for unicast over UD

Regarding the PMTU for bridges, I'm not sure that anyone agrees that adding
an additional roundtrip to all the non-cached routs is a good solution in
all the cases
This is why I thing that the ability to set lower MTU than 2K (1500 or 1K)
should be optional for administrators who seek that

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
H.K. Jerry Chu
Sent: Tuesday, February 26, 2002 10:24 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [ipoverib] moving forward with the open Issues - end node

Jerry

XAA14313

---

From: "Roger Ronald" <[email-masked]>
Date: Tue, 26 Feb 2002 08:25:41 -0600

I heartily concur with these comments. Most of the 1st generation silicon
out there does not handle 2K UD (at least not very well). While I don't
have any great vested interest in any existing hardware (beyond the
100 or so cards we use internally), I can't see why the use of a 
smaller MTU is so offensive. Sure, performance will be much better
with a larger MTU....but, in my book, performance is what comes after
interoperability. Once you have everyone interoperating, performance
enhancements and claims will naturally follow. 

RR

---

From: Michael Krause <[email-masked]>
Date: Tue, 26 Feb 2002 09:47:14 -0800

At 08:25 AM 2/26/2002 -0600, Roger Ronald wrote:
>out there does not handle 2K UD (at least not very well).

Majority of 1st gen is 2KB.  Majority of 2nd gen is 2-4KB.  2nd gen will be 
available from various vendors fast enough that most implementations will 
not suffer any schedule hits in getting this specification in place.

>While I don't have any great vested interest in any existing hardware 
>(beyond the 100 or so cards we use internally), I can't see why the use of 
>a smaller MTU is so offensive. Sure, performance will be much better with 
>a larger MTU....but, in my book, performance is what comes after

Because most of us would rather support the existing IP RFCs that require 
larger MTU support and most of us would rather avoid creating legacy 
problems from an interoperability point of view.

Mike

---

From: "Roger Ronald" <[email-masked]>
Date: Tue, 26 Feb 2002 13:02:40 -0600

I don't know if it's kosher to mention vendors by name, but I know of 
at least two that don't support 2K in their first generation products for
UD. I agree that few of these are in end-user hands, but I don't see
how supporting them hurts anyone.

I'll make the bold claim that there are NO problems that will be
created by allowing a lower MTU (unless you consider 
inferior performance a bigger problem than no performance.)
IP over IB implementations must already interoperate with legacy
implementations of IP elsewhere in the world.....many of which
have smaller MTUs.

I'm in full agreement that larger MTUs are better than smaller
MTUs. Much better. But an arbitrary mandate
doesn't solve any problems and it does eliminate the
opportunity to run this protocol over 1000s of ports (perhaps
tens of 1000s).

RR

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 26 Feb 2002 11:56:27 -0800 (PST)

I agree. There is no need for a default MTU. As noted in the
        drafts and here many a time, it is upto the administrator to
        decide on the right MTU. The maximum we can do is recommend
        that a link MTU of 2048 bytes be used for IPv6 subnets to
        support the IP MTU of 1280 bytes.

        [ Even with IPv6, if it is a stand-alone subnet i.e. not in
        the internet, one can use a smaller MTU. I don't see any
        prohibition on such a setup. ]

        Agree with the comment that this MTU applies to
        unicast/multicast over the UD mode only and not to other
        modes. 

        Regarding MTU for bridges - it is not a link MTU issue. The
        link MTU is only 256, 512, 1024, 2048 or 4096 bytes. The link
        MTU needs to be larger than the MTU needed for bridging. The
        admin will set it to 2048 bytes if bridging to ethernet. Note
        this is the MTU of the MCGRoupRecord.

        The IP MTU can then be lowered as needed. This is the maximum
        size of the IP data. For ethernet bridging one does one of the
        following (this is common over any media not just IB):

		- set it manually 
		- use DHCP 
		- use the routers advertisements

Vivek

-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)

---

From: Bill Strahm <[email-masked]>
Date: Tue, 26 Feb 2002 14:08:45 -0500 (EST)

The problem that you run into is that while IPv4 has a very small minimum
MTU requirement, IPv6 requires a much larger MTU.  ALL new protocols
coming out of the IETF have to be IPv6 aware, which means a minimum MTU of
1280 bytes ber the IPv6 RFCs.

For this reason we have made the decision (and I believe there is rough
concensus on this) to exclude devices that do not support at least a
minimum MTU of 2K.

There is no reason why those devices can not do non-standard
encapsulations, or be configured to allow only IPv4 traffic with a smaller
MTU, so I think this is the best of both worlds

Bill

---

From: David Edmondson <[email-masked]>
Date: Wed, 27 Feb 2002 08:00:19 +0000

* [email-masked] [2002-02-26 19:08:45]
> MTU, so I think this is the best of both worlds

In this last instance, with whom would those devices communicate ?  If
there are other IPoIB devices which _require_ this device to support
an MTU smaller than 2k, then they are not "IPoIB" devices.  If the
other devices don't require the smaller MTU, why would one want to use
it ?