Subject: Re: [Int-area] Comments on draft-van-beijnum-multi-mtu-04

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 21 Mar 2016 22:13:43 +0100

Hi Pat,

Not sure if you're still reading a year and a half later, but here goes...

On 14 Nov 2014, at 20:56, Pat Thaler <[email-masked]> wrote:

> Abstract and 1 mention sending an Ethernet maximum frame size of 1500 bytes  and the referenced RFC 4321 in 7.2 mentions backing that off to 1400 bytes to allow for tunnels.

RFC 4821, you mean.

What this RFC suggests is to start probing with 1400-byte packets. That's as good a value as any, see the discussions about probing back in November 2014.

> Support for the larger frame size is common so generally 1500 byte packets plus a reasonable amount of encapsulation should go through.  Wince RFC 4321 was completed around the same time as IEEE 802.3as, it doesn’t take the envelope maximum frame size into account, but now 8 years later there generally isn’t a need to back off transmitted packet size from 1500 bytes to allow for tunnels.

>  The draft might mention this. 

I've added an explicit reference to 802.3AS-2006 and frame expansion.

However, note that IPv4 and IPv6 over Ethernet do not reference 802.3, they are based on Ethernet II instead.

> 3. Defining Confirmed MTU as the largest packet successfully received or the largest packet acked assumes that the path is symmetric which it might not be. Equating MRU to MTU makes the same assumption. While often the L2 receive path is the same as the transmit path, that is not always the case.

I changed this.

> 9.1 There are diminishing returns in efficiency as MTU size increases and the disadvantage such as head of line blocking and congestion increase. For example, Taking into account Ethernet overheads (IPG, preamble, MAC header), IPv6 header and TCP header, throughput is 87.7% for 1500 byte MTU, 97.8% for 9000 bytes and 98.9% for 18000 bytes.

Data overhead is not the main issue, the fact that senders and receivers as well as routers and switches have to handle millions of packets per second is the main problem.

> 11. Sending of  large packets based on a neighbor’s willingness to accept them may degrade QoS for other traffic including traffic between other nodes on the subnet.  This could be a kind of denial of service to traffic that depends on low latency. This should be discussed as a security consideration.

Added.

> Annex B doesn’t mention that there can be physical layer limitations for large packets. Many  Ethernet PHYs have a clock skew FIFO to deal with differences in the transmitter’s clock rate and the receiver’s clock rate.

I added a paragraph on clock skew.

> B.1 The is no support provided for the assumption in the last paragraph. There are cases where higher speed links are used to reduce latency and latencies in the low microseconds are desired.

I added some text that addresses this.

> B.4, This section is incorrect. The Ethernet CRC has a hamming distance of 4 for up to the maximum packet size.  IEEE 802.3’s design goal for undetected packet error rate requires a hamming distance of 4 (i.e. an 3 bit errors in a packet are detected).  

> According to “32-Bit Cyclic Redundancy Codes for Internet Applications” Philip Koopman, the Ethernet CRC has a hamming distance of 4 to 91067 bits (11,450 bytes).

It seems that I interpreted the old paper incorrectly and used 3 where it should be 4. I have corrected this. Interesting that the newer paper comes out with a packet size that's a few bytes smaller. I've kept the number and reference to the old paper, though.

> B.5 There is no substantiation for the statements here. The disadvantage to small packet users may not be slight. Transmission rate ramping more quickly with large packet size may cause considerable congestion for all and take significant time to resolve. For those using large packets, TCP congestion control may not be able to reduce the rate enough. Have simulations or tests been conducted?

I don't know of any simulations or tests. However, the congestion control behavior of TCP and its interaction with packet size are clear in the models that aren't controversial, this should suffice for an experimental protocol.

> B.7 Conclusion – with my IEEE 802 hat off,  I agree that there are environments such as some data centers where 9000 byte jumbo frames will function fine, but I don’t think they should be unleashed on the big I internet and I don’t think that 64K jumbos are wise anywhere. Much of this draft doesn’t seem consistent with this conclusion rather it seems to support much larger jumbo frames.  Mention of 9000 bytes and an applicability statement should be placed in the body of the draft, not left to the end of an annex.

I added a paragraph about applicability, warning against MTUs larger than 11k.

Thanks again,

Iljitsch