Subject: Problems w/ MTU

From: "Loureiro, Rodrigo - (Bra)" <[email-masked]>
Date: Fri, 19 Oct 2001 14:32:59 -0300

Hi all,

I am investigating a problem with packet forwarding/switching in a MPLS
scope regarding maximum transfer unit, and i would like to share my doubts
with you.

Imagine a four routers setup: PE1, P1, P2 and PE2. PE1 is connected to P1
via POS, P1 is connected to P2 via ATM and P2 is connected to PE2 via a
FastEthernet cross-connection. Imagine also that POS interface is configured
with a 1500 bytes MTU, ATM with 4470 bytes and FastEthernet with 1500 bytes
(consider this hardware is capable to generate Ethernet frames up to 1510
bytes of payload for labeling purposes). 

Now, consider that PE1 and PE2 has a MP-BGP session directly between them.
Considering that BGP is capable to generate messages up to 4096 bytes (RFC
1771), we should guarantee that packets greater than 1500 bytes sent from
PE1 toward PE2 are being fragmented. Instead of this, the packets will be
discarded in the FastEthernet connection between P2 and PE2, once MPLS will
not  fragment the packets. 

I made a test with ICMP from PE1 to PE2, generating packets greater than
1500 bytes, and i could see that packets were not being fragmented by PE1.
So, one question arises: What should be the correct behavior for IP packets
originated inside the router toward another router in a MPLS environment:
should it follow the interface MTU and fragment the packet or should it
ignore it and label switch the packet regardless of its lengh ?

Regards,
--

Rodrigo Loureiro
AT&T LA - Network Engineer

---

From: Alan Hannan <[email-masked]>
Date: Fri, 19 Oct 2001 10:59:56 -0500

This is a known issue, operationally.

  You must set the MTU of your Layer 3 transmittal driver to be small
  enough to allow needed Label Stacking.  I believe we set the MTU of
  much ethernet to 1490 or so to allow this, which also wreaks
  cosmetic habit with certain syslog-alarm-happy boxes...

  In practice, most POS MTUs are 4470, not 1500, FWIW.

  The router should fragment, and it will, but the switch will drop.

  -alan

Thus spake Loureiro, Rodrigo - (Bra) ([email-masked])

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 19 Oct 2001 20:04:06 +0200

Just to add to what Alan said the good practice in the following case
would be to configure your P2-PE2 ethernet switch/router to support
giant frames. Most ethernet platform/drivers allow this today. 

Fragmentation is the last resort but is used in a lot of production
environments running MPLS as the easiest patch :). Of course the
recommended place for fragmentation in your case is PE1 or speaking more
generally the edge of the network.

R.

> Alan Hannan wrote:

---

From: "Alan Hannan" <[email-masked]>
Date: Fri, 19 Oct 2001 11:26:45 -0700

Just want to add to Robert's note adding to mine....

Robert is right.  Often times one can overcome this issue by raising the
MTU of the L2 Ethernet Switch by invoking "Giant Frames".  Many Ethernet
Switches may support Giant Frames or "MiniGiants: (something like 1592
bytes), if not by default, by modifying the configuration.

Ethernet Switches from before the year 2000 may not support Giant
Frames.  Anecdotally, this is the issue to which I refer below.  Old
Catalyst 5500s didn't, if I recall correctly.

I'm still not absolutely positive we've addressed your issue below, so
to try and make sure, the Layer 3 drivers will fragment when the
transmitted packet is larger than the MTU.  If the Layer 2 subsystem
(Considering MPLS as L2 for this purpose) adds shim headers, it will
expand the byte-count.  This can cause the packet loss you describe
below. Solutions include A/ replacing ethernet switch, B/ changing MTU
on L3 box, C/ something else :-)

Best,

Alan

> -----Original Message-----
> From: Robert Raszuk [mailto:[email-masked]]
> Sent: Friday, October 19, 2001 11:04 AM
> To: Alan Hannan
> Cc: Loureiro, Rodrigo - (Bra); [email-masked]; [email-masked]
> Subject: Re: Problems w/ MTU
> 
> 
> > > scope regarding maximum transfer unit, and i would like to share
my
> doubts
> > > with you.
> > >
> > > Imagine a four routers setup: PE1, P1, P2 and PE2. PE1 is
connected to
> P1
> > > via POS, P1 is connected to P2 via ATM and P2 is connected to PE2
via
> a
> > > with a 1500 bytes MTU, ATM with 4470 bytes and FastEthernet with
1500
> bytes
> > > (consider this hardware is capable to generate Ethernet frames up
to
> 1510
> > > Now, consider that PE1 and PE2 has a MP-BGP session directly
between
> them.
> > > Considering that BGP is capable to generate messages up to 4096
bytes
> (RFC
> > > 1771), we should guarantee that packets greater than 1500 bytes
sent
> from
> > > PE1 toward PE2 are being fragmented. Instead of this, the packets
will
> be
> > > discarded in the FastEthernet connection between P2 and PE2, once
MPLS
> will
> > > I made a test with ICMP from PE1 to PE2, generating packets
greater
> than
> > > 1500 bytes, and i could see that packets were not being fragmented
by
> PE1.
> > > So, one question arises: What should be the correct behavior for
IP
> packets
> > > should it follow the interface MTU and fragment the packet or
should
> it