Subject: Re: Reopening jumbo frames in IS-IS

From: [email-masked] (Scott Bradner)
Date: Thu, 23 Jun 2005 08:16:19 -0400 (EDT)

Tony reported:
> Sorry, but I was explicitly told to stop work.

Tony is quite correct

Erik then opined:
> My recollection is that the feedback from IEEE 802 was that we shouldn't 
> touch non-standard things like jumbo-frames, and as a result nothing 
> more was done in this area.
> 
> Perhaps we asked a suboptimal question back then; whether IEEE 802 liked 
> the idea of us doing something for jumboframes and IS-IS. Had we instead 
> asked them whether they were going to specify how jumboframes work with 
> IS-IS, we would probably have received a different answer.

my discussions with the IEEE at that time mentioned IS-IS but that 
was not a factor in the exchange - the IEEE said flatly that 
	1/ jumbo frames for Ethernet were not gonna happen in the IEEE
	2/ they did not want the IETF to make a standard that
	   defined or assumed jumbo frames on Ethernet

re: Erik's suggested question - in effect they did answer that question
'no - they were not going to specify how jumboframes work at all
(nevermind with IS-IS) and please do not that that answer to mean that
the IETF should do so'

Scott

_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion

---

From: Eric Rosen <[email-masked]>
Date: Thu, 23 Jun 2005 10:44:17 -0400

> they were not going to specify how jumboframes work at all
> the IETF should do so

Really, with regard to jumbo frames  the marketplace has spoken and the IEEE
has lost. Jumbo frames are a  defacto standard.  (Of course, just as happens
in theIETF,  the IEEE folks don't always  admit it when they've  lost out to
the marketplace.)

Given that  jumbo frames are  in use,  it seems silly  to say that  we can't
specify how to use them for IP protocols.  There is even some precedent; RFC
3032 (MPLS encaps)  mentions the possibility of using  ethernet frames which
contain 1504 or 1508 bytes.  

It's even sillier to say that (a) we're not allowed to do a carefully scoped
standard that solves a well-specified and immediate problem in the use of an
existing de  facto standard, but that (b)  it's fine to do  a standard which
tells folks to ignore IEEE entirely and do bridging in a completely new way.

---

From: Tony Li <[email-masked]>
Date: Thu, 23 Jun 2005 08:48:40 -0700

Scott,

> 	   defined or assumed jumbo frames on Ethernet

My understanding further extended that prohibition to any informational
or experimental RFC as well.  Those would have been (and still would be)
the best possible outcomes.  There was no requirement to make IS-IS over
jumbo frames into a standard.

Tony

---

From: [email-masked] (Scott Bradner)
Date: Thu, 23 Jun 2005 11:53:04 -0400 (EDT)

> or experimental RFC as well.

that was the case to start with, but as someone else pointed out,
a few years later the IEEE said that they would accept an info RFC
(thought they were not all that happy)

Scott

---

From: Tony Li <[email-masked]>
Date: Thu, 23 Jun 2005 09:05:13 -0700

> (thought they were not all that happy)

First I hear of it.  In any case, I would very much like to progress the
work.  Given the non-standard status of jumbo frames, I have no problem
moving forward with informational or experimental as the goal.

Tony

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 23 Jun 2005 12:49:10 -0400

> tells folks to ignore IEEE entirely and do bridging in a completely new way.

I have to agree that it is silly that IEEE does not acknowledge jumbo
frames.  The provider market has spoken and any large router or large
ethernet switch either implements jumbo frames or doesn't play in that
market.

The reason for jumbo frames is that packet fragmentation in the core
can destroy performance.  As long as MTUs larger than 1500 are in use,
and they are, and the provider POPs where they are handling very high
throughput interfaces of higher MTU, the ethernet MTU will be large
enough or ethernet won't be used.  Its as simple as that.

Lets not use the argument that IEEE should let IETF do things with
jumbo frames because they are allowing IETF to do something else.

Perhaps IETF can specify how ISIS works with the defacto ethernet-like
link layer protocol that happens to differ only in the MTU size and
make it informational.  Its not as if lack of an RFC has stopped any
implementations or deployments so far the benefit of anyone else that
wants to play it wouldn't hurt to provide an informational document.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 23 Jun 2005 12:59:17 -0400

> jumbo frames into a standard.

I think we should have just defied that and written and informational
document.  Its kind of silly for the IETF to expect vendors to keep
implementing toward expired internet-drafts long after a defacto
standard exists as is done all too often.

Lets face it.  Jumbo ethernet frames have been around and have been
widely implemented for about 10 years now.

Curtis

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 24 Jun 2005 22:37:18 +0200

> moving forward with informational or experimental as the goal.

This sounds like information to me, not like an experiment. So I'd support
the idea of aiming at an Informational RFC.

(Peanut gallery question - any other Informationals needed for jumbo frames,
or is IS-IS the only one that needs documenting?)

    Brian

---

From: Tony Li <[email-masked]>
Date: Fri, 24 Jun 2005 16:52:38 -0700

> the idea of aiming at an Informational RFC.

Thanks Brian.

> or is IS-IS the only one that needs documenting?)

I'm not aware of other significant protocols that are not using an
Ethertype encapsulation at this point, so no.

Tony

---

From: Pekka Savola <[email-masked]>
Date: Sat, 25 Jun 2005 09:51:36 +0300 (EEST)

On Fri, 24 Jun 2005, Tony Li wrote:
>> (Peanut gallery question - any other Informationals needed for 
>> jumbo frames, or is IS-IS the only one that needs documenting?)
>

But as the actual IP encapsulation in jumbo frames hasn't been 
specified officially anywhere, would that need to be done instead/in 
addition?  This would seem like an appendix for "IPvX over Ethernet" 
-like specifications..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Tony Li <[email-masked]>
Date: Sat, 25 Jun 2005 01:03:56 -0700

> But as the actual IP encapsulation in jumbo frames hasn't been specified
> officially anywhere, would that need to be done instead/in addition? 
> This would seem like an appendix for "IPvX over Ethernet" -like
> specifications..

Pekka,

I think that the IEEE would see that as an end-run around their process.
 And they would be right.

The documents that describe jumbo frames already suggest that everything
just runs as it would over normal frame sizes, so no additional
agreement is really necessary to achieve de facto interoperability.

Tony

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 26 Jun 2005 12:00:44 -0400

In message <[email-masked]>
> or is IS-IS the only one that needs documenting?)

Brian,

It probably would not hurt to have an informational draft describing
jumbo frames, the reason for needing a them, when to use them and
importantly when not to use them.

Jumbo frames exists only to avoid the performance drop that would
otherwise potentially cripple core networks using GbE or 10GbE and
aggregating higher MTU links.  Since jumbo frames could otherwise be
harmful, it might make sense to ask that equipment be capable of
forwarding jumbo frames but originate standard MTU frames (except when
verifying MTU such as in ISIS) and that end user devices such as NIC
cards never originate jumbo frames.  It should also be clearly stated
that jumbo frames be disabled by default and any equipment supporting
them at the very least put harsh warnings in the equipment's user
documentation about the feature being non-standard and has a potential
to cause interoperability problem if just casually enabled.

The fear of the IEEE folks might be that once word gets out of jumbo
frames a lot of people who don't really need them will be asking for
them and equipment in bridged environments won't be able to deal with
them.  In bridged environments fragmenting isn't a possibility and
today most equipment doesn't support jumbo frames, mainly just the
larger routers and switches sold into the large provider market.

(Peanut gallery answer.  Now all you need is an author.)

Curtis

---

From: "Stephen Sprunk" <[email-masked]>
Date: Sun, 26 Jun 2005 12:10:46 -0500

Thus spake "Curtis Villamizar" <[email-masked]>
> otherwise potentially cripple core networks using GbE or 10GbE
> and aggregating higher MTU links.  Since jumbo frames could
> otherwise be harmful, it might make sense to ask that equipment be
> capable of forwarding jumbo frames but originate standard MTU
> frames (except when verifying MTU such as in ISIS) and that end
> user devices such as NIC cards never originate jumbo frames.

That seems excessively restrictive.  There are many environments today using
jumbo frames (8k-10k MTU) on end hosts without problems.

This also begs for a definition of what exactly a "jumbo" frame is; does an
MPLS label (or several) on the front of a 1500-byte Ethernet frame count?
Does a packet on media with a >1500 byte native MTU count?  Should a host on
FDDI or a direct HDLC/PPP link be limited to 1500 byte packets -- even
though its native MTU is 4470 -- because we fear it might need to talk to an
Ethernet-connected host?

> It should also be clearly stated that jumbo frames be disabled by
> default and any equipment supporting them at the very least put
> harsh warnings in the equipment's user documentation about the
> feature being non-standard and has a potential to cause
> interoperability problem if just casually enabled.

I certainly agree that jumbo frames should be disabled by default, and the
warning should explicitly note that severe problems will result if hosts on
the same subnet have different ideas of what the MTU is.  I personally have
never seen a problem with jumbos when every device on a subnet supports them
and is configured with the same MTU, but perhaps someone else can offer some
enlightening anecdotes.

We might consider adding a means for hosts configured with a "jumbo" MTU to
determine if all of its neighbors (and intermediate L2 devices) can indeed
handle such packets.  If such a mechansim existed, we could allow jumbos to
be on by default since hosts could detect when it was necessary to fall back
to a non-jumbo MTU.

S

Stephen Sprunk      "Those people who think they know everything
CCIE #3723         are a great annoyance to those of us who do."
K5SSS                                             --Isaac Asimov

---

From: Daniel Senie <[email-masked]>
Date: Sun, 26 Jun 2005 13:59:37 -0400

>jumbo frames (8k-10k MTU) on end hosts without problems.

Indeed, it would seem to make sense for communications between front 
end and back end servers (e.g. app server to database cases). In 
controlled environments it should help boost database transactions, 
NFS and whatnot. This was certainly one reason FDDI was used in 
places in the past, as large frame sizes were permitted on that medium.

>be on by default since hosts could detect when it was necessary to fall back

Indeed, OSPF has had such a mechanism for years as a result of the 
existence of LAN media types with disparate MTUs (token ring, FDDI). 
Routers are permitted to pack LSAs into packets up to MTU size, and 
if another router on the LAN had a different MTU, significant 
problems resulted (and those problems could lie dormant in a live 
network for a long time before things broke, as it turned out).

It would certainly be worthwhile to have some means for testing. The 
trick is when a transport or routing session fails, some generic 
mechanism is really needed to think to ask the remote end for its 
MTU. That could be an ICMP message asking for MTU. Keep in mind, 
however, that a station which can't receive a frame due to MTU 
mismatch (recipient had MTU 1500, sender something bigger) will not 
even hear the frame (dropped at MAC level) so it can't respond with 
an error message.

The Ethernet MTU of 1500 allowed this issue to be ignored for a long 
time. Other media had issues with it, but they were mostly ignored as 
Ethernet dominated. It'd sure be nice if this were dealt with on a 
generic basis. Probably could have been piggybacked on ARP responses 
if there'd been a range of MTUs to worry about when ARP was proposed. Oh well.

---

From: Tony Li <[email-masked]>
Date: Sun, 26 Jun 2005 12:43:36 -0700

> aggregating higher MTU links.

Another, and perhaps more common, reason for jumbo frames is to support
high speed data center and storage applications.  Many modern systems
have an 8KB primary memory page size, and can be most efficient at bulk
transfers when they can make use of the page as a transfer unit.

Tony