Subject: RE: [Isis-wg] More vendor data on IP/GigE MTU

From: Jeff Parker <[email-masked]>
Date: Tue, 19 Jun 2001 15:57:26 -0400

Ran, Tony -

	I remember the history behind this showing up in IS-IS,
but suspect that some who may be interested in the outcome 
might not be watching the IS-IS list for a decision about MTU.  
I'm not sure where it should go, but I suspect that even if 
we get {N-1, 1, 0} consensus on one view or another, it will 
be judged moot unless we get this before a wider venue: some 
group in Transport or IP, perhaps.  

	Is there a lurking AD with an opinion?  

- jeff parker
- axiowave networks
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://external.juniper.net/mailman/listinfo/isis-wg

---

From: Jed Kaplan <[email-masked]>
Date: Tue, 19 Jun 2001 16:38:24 -0400

With respect to default IP MTU, although the draft did not specify a value,
4470 was desired; to be able to avoid fragmenting
large FDDI packets in some of the authors' ISP networks.

Its not clear which is needed more (or less), large FDDI or (somewhat large)
ATM. In the time between writing the draft and now,
another reason for jumbo frames became clear -  to avoid fragmenting MPLS
packets.  pMTUd rarely works.

Without more input on the predominance of FDDI vs ATM, 4470 seems the
correct choice for default.

-Jed

Jed Kaplan
Allegro Networks
12700 Fair Lakes Circle, Suite 100
Fairfax, Va 22022
email: [email-masked]
ph: (703) 802-2348

---

From: "Martin, Christian" <[email-masked]>
Date: Tue, 19 Jun 2001 17:59:06 -0400

From discussions I've had with folks that were closely involved with Cisco
from the late 80's into the mid 90's, the 4470 MTU was decided upon to
support Token Ring/FDDI-HSSI-Token Ring/FDDI connectivity without
fragmentation.  This MTU was carried over to the PODS3/POS cards in order to
reuse system designs/components, as well as allow for a migration to these
technolgies while preserving the aforementioned environment.  As such, it
would seem that the idea was/should be to have the WAN link MTU be >= to the
largest LAN MTU that is commonplace.  In the early/mid 90's, TR and FDDI
were more common than jumbo-GE/10GE are today, relatively speaking.  Today,
however, there are very few LANs utilizing either of these technologies
(except perhaps big SNA shops).  So logic would dictate that we not reinvent
the wheel, keep it simple, and continue to support 1500 (or 4k if needed).

As to Vijay's point about >4k MTU traffic being fairly smaall, I think that
data provided by providers would indicate that the majority of 4k packets
are BGP updates, and this generally only occurs during session flaps (where
updates can be aggregated because of matching path attribs and
MinAdvertisementInterval is ignored).  As far as customer traffic, 'fairly'
is an understatement.  Words like diminishingly, vanishingly,
infinitessimally, and nil fit the bill nicely, in fact.  The only exception
would be the various tunneling mechanisms in use today (MPLS, GRE, L2TP),
and this still only adds a few hundred bytes, max.

The only need for 9k MTUs would be server-to-server traffic (I am still
waiting for my OC-3 connection to my home and my 10GE NIC for my PC ;>).  If
this has to be done across the Internet at large, then the argument is moot
as this requirement is not going to drive every ISP to change all the
POS/DS3 MTUs in their networks.  If the WAN is under the control of the
operator from end-to-end, then this small subset of hardware purchasers need
to make enough of a stink to get their respective vendors to allow for 9k
buffer allocation, in hardware, on the respective WAN interfaces.  I doubt
this will happen without serious clout (clout = $$$), especially given
Tony's quite fascinating discussion on interface hardware design.

So, we have a few choices:

1) Leave the MTU at 9k as it has been deployed and allow PMTUD to determine
MTU's.  For those vendors that wish to support 9180k MTUs on routers, design
a WAN interface that allows for 9k MTUs to get around fragmentation.
2) Set the MTU to 4470 to match the thousands of deployed links that are in
use today.  
3) Restandardize the PPP/HDLC framing for use over POS to 9180 and force all
the hardware vendors into compliance.
4) Scrap the whole jumbogram idea altogether.

As I see it, these are ranked in order of preference, and perhaps in order
of likeliness as well.

An Operator's Perspective,
-chris

> 
> Ran, for various reasons, including some I sent earlier to 
> the list, from
> an operational perspective, I've never seen any demand for > 
> 4470+overhead
> MTU. Therefore, any devices that go into networks that I've 
> worked on have
> been required to support an MTU of ~4470, and I do not see 
> this changing
> in the future. The amount of traffic at even 4470 is fairly small, but
> there have been customers who have asked for a 4k MTU. I see no
> operational customer based demand to go greater than 4k, and I suspect
> this will be true for a while.
> 
> /vijay
> 
>

---

From: "Richard A. Steenbergen" <[email-masked]>
Date: Tue, 19 Jun 2001 21:29:01 -0400 (EDT)

On Mon, Jun 18, 2001 at 09:13:32AM -0400, RJ Atkinson wrote:
>
> >Also, 9180 is NOT necessary to support page flipping.
> >4470 would seem to very nicely support a single 4k page.
>
>         True, but 8K pages are commonly used these days, not 4K,
> so its of much less benefit than it was in the FDDI era.

Not really, but they will probably be more common w/IA64 and other
architectures, which will probably NOT want to use 4K pages.

>         More generally, *most* Gigabit Ethernet products that have
> shipped in the past 3 years *already* support the 9180 MTU, including
> big fast routers with GigE interfaces.  So you forgot to tell the rest
> of the world about your plan for 4470 to be the standard MTU.  This
> means that fragmentation is *already* inevitable on GigE links (if
> PMTU is turned off or broken in some way).  My main goal is to
> document widely deployed existing practice.  I happen to agree with
> the reasons that such is the existing practice as well (obvious to
> anyone reading this thread).

In my experience, most GigE vendors do NOT support 9180. I've seen all
kinds of numbers all around that, and some not even close depending on the
product line or specific hardware.

>         As I noted at the outset, an Informational RFC is totally
> non-binding (and the content of standards-track RFCs are pretty
> frequently ignored as well), so I don't see why you are getting
> spun up.

I think the most useful thing this could do is establish a definition for
"real" jumbo frame support. At this point I don't really care what it is,
8192 and some bytes for headers, 9180 is fine, but we need a STANDARD. Any
such document should also spend time covering the benefits of jumbo frame
support, since it seems that a number of people just don't "get it".

I have to admit I really don't understand the IEEE. GigE/10GE is going to
be far more common on a router or at an exchange point then on a host for
a long time. There are simply too many legitimate reasons to desire
support for a common larger frame size on non-host ethernet to reject the
concept on the basis of someone's hardon for what their 1985 copy of
Merriam Webster says for "WAN" and "LAN".

Perhaps it would also be useful to focus on fixing Path MTU Discovery. For
example, it might make sense to inform the sender about fragmentation in
the TCP ACK, to prevent misconfigured icmp filters or rate limits. If
PMTUD worked right, we would be able to support the most optimal MTU size
seemlessly.

-- 
Richard A Steenbergen <[email-masked]>       http://www.e-gerbil.net/ras
PGP Key ID: 0x138EA177  (67 29 D7 BC E8 18 3E DA  B2 46 B3 D8 14 36 FE B6)

---

From: Radia Perlman - Boston Center for Networking <[email-masked]>
Date: Tue, 26 Jun 2001 22:00:20 -0400 (EDT)

Apologies in advance...I'm behind on email and haven't been
following this thread, so I hope I'm not duplicating what
someone else said.

>>Re: Why pad to full MTU size?

I vaguely remember the reason was that although you might think you're
on the same link as your neighbor, you might have bridges bridging
your packet over a sub-link with a smaller packet size. So the only
way to tell if you really can send a packet as big as you think you
can is to send such a big packet. Telling your neighbor "I can send
4000 byte packets" because you're on FDDI, and he's on FDDI, won't work
if the bridged path between you contains an Ethernet with MTU 1500.

I vaguely remember another reason was that someone told me that some
links have failure modes where small packets go through OK but
big ones don't, so you don't want to bring up the adjancency
if the link is that flaky.

Radia