Subject: Re: [Int-area] What is a jumbogram, was: Larger MTUs

From: Matt Mathis <[email-masked]>
Date: Wed, 1 Aug 2007 10:35:04 -0400 (EDT)

Jumbo just means larger than standard.  Given that there are so many standards
to choose from... well you get the idea.

To add to the confusion, to IEEE 802.3, jumbo means 1536 (I think I got the
number right).  This is for extra large Ethernet headers with a 1500 byte
payload. They use "superjumbo" to refer to 9kB, etc.

It is really important that we do not standardize 9k or any other fixed MTU as
the new Internet cell size.  The intent of 4821 was to permit the Internet to
support diverse MTUs at all scales.  With a couple of additional clicks on
Moore's Law 9k is likely to be deemed too small, but we really don't want to
have to do any part of this over again.

BTW for some background on the wider problem check out
http://www.psc.edu/~mathis/MTU

THanks,
--MM--
-------------------------------------------
Matt Mathis      http://www.psc.edu/~mathis
Work:412.268.3319    Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

_______________________________________________
Int-area mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/int-area

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 1 Aug 2007 16:46:04 +0200

Some of the terms related to Ethernet should be clarified, I believe.
The IEEE 802.3 standard does not recognize the terms 'jumbo' or
'superjumbo'. These are as you point out of the standard. The max frame
size in the original Ethernet standard was 1518. This was slightly
increased about a decade ago to 1522 when IEEE 802.1 added four bytes
for VLAN ID. More recently the IEEE 802.3as extended the max size of the
Ethernet frame to 2048. Also, there is no such thing as a 'extra large
Ethernet header'. The header of an Ethernet frame is of fixed size. It
is the payload in the Ethernet frame that may be of variable length. 

Dan

> -----Original Message-----
> From: Matt Mathis [mailto:[email-masked]] 
> Sent: Wednesday, August 01, 2007 5:35 PM
> To: Internet Area
> Subject: Re: [Int-area] What is a jumbogram, was: Larger MTUs
> 
> Jumbo just means larger than standard.  Given that there are 
> so many standards to choose from... well you get the idea.
> 
> To add to the confusion, to IEEE 802.3, jumbo means 1536 (I 
> think I got the number right).  This is for extra large 
> Ethernet headers with a 1500 byte payload. They use 
> "superjumbo" to refer to 9kB, etc.
> 
> It is really important that we do not standardize 9k or any 
> other fixed MTU as the new Internet cell size.  The intent of 
> 4821 was to permit the Internet to support diverse MTUs at 
> all scales.  With a couple of additional clicks on Moore's 
> Law 9k is likely to be deemed too small, but we really don't 
> want to have to do any part of this over again.
> 
> Evil is defined by mortals who think they know "The Truth" 
> and use force to apply it to others.
> 
> 
>

---

From: Matt Mathis <[email-masked]>
Date: Wed, 1 Aug 2007 10:56:41 -0400 (EDT)

A further suggestion: the performance of jumbo ARP/padded ND can be improved
if the router is configured in advance with the likely MTU sizes on the
subnet.  For the most common case, with one supported jumbo size plus the
standard, this approach only requires one extra directed ARP/ND per peer
(assuming that long term caching is ok).

This in not unreasonable, give that all >1500 Ethernet solutions requires at
least some minimal configuration on each device.

Thanks,
--MM--

On Tue, 31 Jul 2007, Iljitsch van Beijnum wrote:

> On 31-jul-2007, at 16:08, Iljitsch van Beijnum wrote:
>
> > So for full end-to-end jumbo use across the internet the additional
> > mechanisms need to be implemented on all intermediate ethernets (or
> > jumboframes must be enabled administratively) but with only RFC
> > 4821 you can still have out-of-the-box jumboframe capability for
> > TCP and maybe some other transports on the local subnet.
>
> Rethinking the type of MTU detection packets: a new mechanism that
> must be supported by both sides, or something that can be implemented
> by just one side?
>
> For the latter, the question would be: what type of packet is most
> appropriate? I see two options:
>
> 1. ICMP echo request/reply
> 2. Oversized ARP and neighbor discovery packets
>
> The advantage of ICMP is that using echo request/reply doesn't
> introduce new uses for an existing message, but it has several
> downsides:
>
> - replying to large packets with large packets could be abused for
> denial of service
> - replying with large packets is unnecessary but inherent to ICMP echo
> - many software firewalls, some enabled by default, block these messages
>
> I haven't been able to find a description of "jumbo ARP" but I assume
> this is simply an ARP request padded to the desired length and sent
> as unicast. (Multicasting large packets if it can be avoided is a big
> no-no on wifi because multicasts are typically sent at the lowest
> supported rate.)
>
> If an answer comes back, the target host is able to receive packets
> of the length used when sending. ARP packets are already often padded
> to 46 bytes to fill a minimum ethernet frame, so I assume this will
> work in many cases. IPv6 ND would work much the same way except that
> a padding option would have to be added to increase the packet size.
> Some systems may not be prepared to handle ND packets of arbitrary
> sizes.
>
> In each case, the test packet would have to be sent shortly after
> discovering a neighbor's MAC address using regularly-sized ARP or ND.
> Since it is unknown whether the neighbor supports jumboframes at all,
> let alone what size, it's probably prudent to test in two stages:
> first send a test packet of the minimum usable jumbo size (I'm
> thinking 1520 bytes, that would be 1500 + 20 byte IP-in-IP header).
> This wastes less time on the wire and is likely to upset old gear to
> a lesser degree than a maximum size jumboframe.
>
> If an answer comes back, note the increased MTU and the next try is
> with the largest packet size supported locally. If an answer comes
> back for that one, again note the larger MTU. If no answer comes
> back, do nothing. If the other side supports a smaller maximum size,
> then it will probe with that size at some point, and that will be our
> cue to try one last time with that size. This way, it's possible to
> find out the largest supported MTU between two hosts that both
> implement this mechanism without trying a large number of sizes.
>
> However, if only one end supports the mechanism, additional probes
> are necessary to discover the maximum supported MTU, or a
> conservative maximum must be chosen and the real maximum between two
> neighbors is never discovered.
>
> With IPv6 neighbor discovery, we can add an MTU option so that if
> both neighbors support the new mechanism, they'll tell the other
> their MTU without the need to send multiple large packets. (Also very
> useful to signal lack of jumboframe support.) If nodes run both IPv4
> and IPv6, this information can be reused for IPv4, and it only has to
> be validated by sending a jumbo ARP. In this case, we can probably
> get away with trying just 9000 bytes or the local maximum, whichever
> is smaller, if no neighbor specific information is known, as few
> hosts (but a larger number of switches) implement jumboframes < 9000
> bytes.
>
> Unless someone has other insights to offer, I think I'll update my
> draft according to the above.
>
>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 1 Aug 2007 17:53:47 +0200

> least some minimal configuration on each device.

In my opinion, it's extremely important to move away from any special  
configuration for larger packets. We're only going to make good  
progress if hardware that supports larger packets will automatically  
use this capability where appropriate.

However, if, until the time that any new mechanisms that we define  
are widely implemented, making configuration adjustments makes  
everything work better, that's fine, of course.

BTW, does anyone have a description of jumbo ARP? I want to see if  
there's the possibility to sneak in an MTU field so it's possible to  
check for the maximum supported MTU quickly rather than probe  
extensively, if both ends support this. Apparently padding ARP  
packets works most of the time. I'm assuming that the content of the  
padding probably doesn't matter, so putting the MTU somewhere in the  
padding shouldn't lead to trouble...

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 1 Aug 2007 14:43:09 -0700

> Ethernet frame to 2048.

The 1518 would be (14 header + 1500 ULP + 4 trailing CRC).
Focusing only on the ULP, can we say that 1500 bytes is
the nominal MTU of today's Internet, and therefore all
attached devices SHOULD configure a minMRU of 1500? Then,
can we also say that applications that send packets larger
than 1500 bytes (with DF=1) are RECOMMENDED to use RFC4821?
Finally, could we also say that tunnel decapsulators SHOULD
configure a minMRU of 2048 to account for encapsulations
that might extend a 1500 byte ULP out to the IEEE 802.as
maximum frame size?

Asked another way, can all of this be addressed in a brief
BCP that captures something like the above?

Thanks - Fred
[email-masked]      

> Also, there is no such thing as a 'extra large

---

From: John Heffner <[email-masked]>
Date: Wed, 01 Aug 2007 20:56:20 -0400

> BCP that captures something like the above?

This is the advice from RFC4821 (not in standards language):

    Since protocols that do not implement PLPMTUD are still subject to
    problems due to ICMP black holes, it may be desirable to limit to
    these protocols to "safe" MTUs likely to work on any path (e.g., 1280
    bytes).

I think it would be unrealistic to say tunnels SHOULD have a MRU of at 
least 2048 given the preponderance of gear that only supports 1500 
bytes.  However, it's relatively easy for non-4821-capable protocols to 
limit sizes to something a little smaller that will provide adequate 
headroom for tunnel headers.

   -John

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 2 Aug 2007 09:40:08 -0700

John, 

> headroom for tunnel headers.

But, what is more realistic? To ask decapsulating tunnel
endpoints to set an MRU that is slightly larger than the
MTU of the underlying interface, or to ask non-4821-capable
protocols (and/or applications) to reduce the maximum size
of the packets they send? I believe it is common for systems
to expose MRU as a configurable option in the former case,
whereas application/protocol changes might be necessary in
the latter case.

Thanks - Fred