Subject: Re: please read and comment on the MTU problems in draft-liu-rtgwg-mtu-config-ps

From: Curtis Villamizar <[email-masked]>
Date: Fri, 02 Aug 2013 16:54:52 -0400

Jeff,

OSPF does not intentionally send a full MTU sized packet as ISIS does.
The MTU is simply stated in the Database Description Packet.  The set
of values to use for common interfaces is stated in Path MTU Discovery
Table 1 which is cited for that reason in RFC2328 (OSPFv2).  If a
provider configures the nominal MTU for an OSPF interface for any
given interface type, then it doesn't matter if the vendor considers
the MTU to include or exclude L3 framing or VLAN or MPLS labels.

For example, if both sides configure 1500 as the value for OSPF to
send in the Database Description Packet, then it doesn't really matter
if the packet is sent over a PW and a 1500 byte packet would in fact
be fragmented along the way.

So for OSPF if the two sides are configured to the same nominal MTU,
then all is well.  OTOH if one has 1500 and the other side can send
Ethernet jumbo frames at nominal 9216 bytes, then the adjacency should
not come up until the problem is fixed as it would black hole traffic.

The current behavior of ISIS is arguably not broken either.  If one
side can send a packet that the other side will reject in error, then
a link should not come up.  ISIS goes through the trouble of
continuously checking the MTU (and wasting bandwidth) by padding every
LSPDU to the largest size that can be sent in addition to
negociating.  In practice a lot of hardware can receive packets
somewhat larger than the stated MTU but the attempt to verify MTU down
to the byte causes some problems.

My experience while at a provider was that if you figured out the
smallest practical MTU in the core and configured all interfaces to
that value so as never to fragment within the core, then OSPF always
just worked.

In both cases, it would be nice to standardize nominal MTU values,
allow negociation, and state what the MTU includes.  OSPF already
simply requires a nominal MTU to match eliminating the need to figure
out whether the MTU should be reduced for VLAN tag, or a label, etc.

My point was that this should be taken up in the ISIS WG and if needed
it should be taken up in the OSPF WG.

Curtis

In message <[email-masked]>
Jeff Tantsura writes:

Curtis,

In any known OSPF implementation if MTU on both sides doesn't match OSPF will get stuck in Exchange/Exstart. 

Regards,
Jeff

On Jul 27, 2013, at 5:29 PM, "Curtis Villamizar" <[email-masked]> wrote:

> 
> In message <[email-masked]>
> Alia Atlas writes:
> 
>> We are likely to add this draft to the list to be discussed in the
>> meeting.  It is discussing some general problems with MTU
>> configuration between vendors.  While it discusses this for ISIS,
>> some aspects also apply to OSPF.
>> 
>> Basically, there are three problems:
>> 
>> a) Some implementations use L2 MTU and others use the L3 MTU
>> b) Different implementations pad and check for different lengths.
>>   Consequences are that ISIS sessions do not make it to Up.
>> c) Different numbers of MPLS labels are assumed as the max, resulting in
>>   fragmentation, and there are generally no controls for it.
>> 
>> http://datatracker.ietf.org/doc/draft-liu-rtgwg-mtu-config-ps/
>> 
>> Thanks,
>> Alia
> 
> 
> First of all, the IGP issue is unique to ISIS and should be fixed in
> ISIS.  ISIS pads the LSPDU fragment to the full MTU to verify that the
> full MTU is supported.  THis is a misfeature IMHO, but that is the way
> it is.  OSPF does not have this misfeature.
> 
> The reason that ISIS does this is that IP fragmentation at full line
> rate is considered hard to do for high line rates relative to IP
> forwarding.  Some **very** old routers did this in software and would
> crash if you gave them a full line rate load of packets to fragment.
> The ISIS practice of padding to full MTU predated the existance of
> MPLS and any use at all of VLAN by providers (who still rarely use
> VLAN and probably never in core).  Many routers today can fragment at
> full line rate or close to it for average packet sizes (not
> tinygrams).  Some can't, but if the rate is very low, they should be
> avoided.  Providers need to test for this.  In modern networks, if
> most packets are 1532 or smaller, whether the MTU is somewhat over 4K,
> somewhat over 8K, or closer to 9K makes no difference.
> 
> TCP has Path MTU Discovery (PMTU) which uses the IP "Do Not Fragment"
> bit and looks for the ICMP Would Fragment packets as a hint to lower
> the MTU.  ISIS could do the same.
> 
> Note: Using TCP PMTU has an issue with some (broken IMHO) firewalls
> blocking all ICMP, rather than just blocking forms of ICMP that might
> be in some way dangerous.  This problem can be avoided by sending only
> some packets with "Do Not Fragment" and using SACK to figure out that
> those are consistently getting lost (and not counting that as a drop
> but rather as an indication to lower the TCP MSS value).
> 
> Instead of trying to figure out the *right* value to configure, the
> ISIS protocol needs to get smarter about dealing with different values
> of MTU, including small variations created by encapsulation in MPLS.
> 
> IMHO this should be resubmited as a problem statement to the ISIS WG.
> 
> Most providers have already figured out what *they* need to configure
> as the MTU on their links and do so by using either program generated
> router configurations or using programs to check router configurations
> for some level of consistency.  BTW- the 4470 number dates back to POS
> wanting to have a larger MTU than FDDI.
> 
> As far as configuring a value of 4470 on all routers without ISIS
> changes, the provider must first read the vendor literature to find
> out what is counted in the MTU, layer-2 MTU or layer-3 MTU and figure
> out whether further encapsulation will impact the effective MTU.
> Looking at the default value provides a good hint.
> 
> If all the traffic coming into a network is at 1532 MTU (or something
> in that range) and the provider just wants headroom for PW and other
> labels, whether the MTU on some links is 4470 or 4484 (or 9180) makes
> no difference at all.  Therefore inconsistencies in values over about
> 1600 make no difference at all for such a provider.  A control packet
> here or there might get fragmented but everything (except ISIS) should
> work with minor inconsistencies in the configured MTU.
> 
> Curtis
> _______________________________________________
> rtgwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtgwg

---

From: Alia Atlas <[email-masked]>
Date: Fri, 2 Aug 2013 17:06:51 -0400

In the RTGWG meeting, there was fairly clear interest in thinking about
this issue.  Adrian also suggested discussing it with the OPSWG.  I believe
that Vic is looking for a co-author or so who is interested in helping
explore what the right thing to document or do in the IETF context.

Alia

On Fri, Aug 2, 2013 at 4:54 PM, Curtis Villamizar <[email-masked]>wrote:

> > In message <
> [email-masked]>
> > Alia Atlas writes:
> >

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 6 Aug 2013 04:44:37 +0000

First, I strongly agree w Tony Li - the inconsistent support of "mtu" configuration by different vendors (or even in some cases different boxes made by the same vendor) will not be resolved by an IETF specification - however well motivated.

Although the problems seen due to inconsistent mtu configuration may manifest themselves as routing protocol neighbor bringup issues, this does not imply that the routing protocols themselves are broken and need fixing. In regards to IS-IS in particular I encourage people to read Sections 5 and 6 of RFC 3719. This provides a cogent description of how several different protocol parameters interact, how padding is used and why, and what protocol tools exist today to help prevent and detect problems related to inconsistent MTU.

If after reading the reference someone still feels that an IS-IS protocol change is needed please bring it to the WG - but be prepared to be very specific as to what you believe needs fixing and how you would propose to do it,

   Les

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Curtis Villamizar
> Sent: Friday, August 02, 2013 1:55 PM
> To: Jeff Tantsura
> Cc: [email-masked]
> Subject: Re: please read and comment on the MTU problems in draft-liu-rtgwg-
> mtu-config-ps
> 
> 
> Jeff,
>

---

From: Jeff Tantsura <[email-masked]>
Date: Tue, 6 Aug 2013 04:52:30 +0000

Les,

I have talked to the author, he's no calling for any fixes in routing
protocols but merely aiming to document how different vendors treat MTU
(IP vs MPLS vs L2, etc) and trying to bring to vendor's attention how
cumbersome it is to configure MTU in a multivendor environment.

Cheers,
Jeff

-----Original Message-----
From: "Les Ginsberg   (ginsberg)" <[email-masked]>
Date: Monday, August 5, 2013 9:44 PM
To: "[email-masked]" <[email-masked]>, Jeff  Tantsura
<[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: RE: please read and comment on the MTU problems
in	draft-liu-rtgwg-mtu-config-ps

>First, I strongly agree w Tony Li - the inconsistent support of "mtu"
>configuration by different vendors (or even in some cases different boxes
>made by the same vendor) will not be resolved by an IETF specification -
>however well motivated.
>
>Although the problems seen due to inconsistent mtu configuration may
>manifest themselves as routing protocol neighbor bringup issues, this
>does not imply that the routing protocols themselves are broken and need
>fixing. In regards to IS-IS in particular I encourage people to read
>Sections 5 and 6 of RFC 3719. This provides a cogent description of how
>several different protocol parameters interact, how padding is used and
>why, and what protocol tools exist today to help prevent and detect
>problems related to inconsistent MTU.
>
>If after reading the reference someone still feels that an IS-IS protocol
>change is needed please bring it to the WG - but be prepared to be very
>specific as to what you believe needs fixing and how you would propose to
>do it,
>
>   Les
>
>
>> In any known OSPF implementation if MTU on both sides doesn't match
>>OSPF will
>> get stuck in Exchange/Exstart.
>> 
>> Regards,
>> Jeff
>>

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 6 Aug 2013 04:56:53 +0000

Jeff -

I have the same impression as regards draft-liu-rtgwg-mtu-config-ps as you just confirmed. However, this thread has included some specific statements regarding OSPF/IS-IS (NOT by the draft authors). My remarks regarding routing protocols are in response to those statements - not in response to the draft. Frankly I think the routing protocol discussion has taken us off topic.

   Les

> From: Jeff Tantsura [mailto:[email-masked]]
> Sent: Monday, August 05, 2013 9:53 PM
> To: Les Ginsberg (ginsberg); [email-masked]
> Cc: [email-masked]