Subject: PPP over Sonet default MRU

From: Stephane St Hilaire <[email-masked]>
Date: Wed, 22 Jan 2003 16:20:37 -0500

While doing POS interoperability testing with Juniper and Cisco routers we
found the following behavior; Even though Juniper POS interfaces have a
default MTU of 4470, no MRU information is sent during LCP negotiations. 
Even so Cisco routers that use the same default MTU, but DO send their MRU,
still send packets of up to 4470 out. My question are the following:

1) Is 4470 the actual default MRU to be used for POS (and not 1500 as
documented in rfc 1661) ?
2) If so, is this documented somewhere ?
3) If 1500 is in fact the default MRU, isn't there something "wrong" for
lack of a better term with Juniper's behavior in not sending it's MRU and
Cisco's in sending frames of more than 1500 bytes regardless.

The behavior observed might be specific to the actual routers and OS
versions used, but I am still curious as to what the actual "standard" is.

Thanks for any clarifications.

Stephane St-Hilaire
[email-masked]
T:514-906-4915

---

From: James Carlson <[email-masked]>
Date: Thu, 23 Jan 2003 08:05:03 -0500

> still send packets of up to 4470 out. My question are the following:

Yes; I've seen that behavior.

> documented in rfc 1661) ?

No.  The default is still 1500.  It's not changed by RFC 1619 or 2615.

> 2) If so, is this documented somewhere ?

I think it's just a configured MRU.  There's nothing wrong with a
configured (or even fixed) MRU.  RFC 1661 PPP provides the advantage
that the peers *can* negotiate the value on each end, providing very
helpful diagnostic capabilities, but there's no requirement that
anyone ever implement any of the options.  (That said, failing to
implement useful options seems like a rather poor choice to me.)

It's the same way with IP addresses: it's a very good idea to
negotiate the endpoint addresses (even if they're not distinct from
the addresses uses by other links) in order to avoid configuration
errors.  There's no requirement that anyone do so.

> Cisco's in sending frames of more than 1500 bytes regardless.

If the Cisco side doesn't have its MTU specifically tuned to 4470, and
it's sending packets larger than 1500 octets, then I'd worry about
it.  Otherwise, no.  Things that are agreed to by some outside
mechanism are fine.

All that RFC 1661 says is that (absent some manual configuration data
to the contrary) you have to be prepared to receive frames of at least
1500 octets in case LCP starts over.

-- 
James Carlson, Solaris Networking         <[email-masked]>
Sun Microsystems / 1 Network Drive         71.234W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.497N   Fax +1 781 442 1677

---

From: Dennis Ferguson <[email-masked]>
Date: Wed, 29 Jan 2003 22:41:42 -0800

Stephane,

> documented in rfc 1661) ?

4470 is the default MTU for POS when using what appears to still be the most
commonly configured link encapsulations on this type of circuit (i.e. not PPP).
For PPP over SONET, however, I can see no reading of RFC 2615 or 1661 which
would lead one to any conclude the answer is any number other than 1500.

> 2) If so, is this documented somewhere ?

In the not-PPP cases, not in an IETF document.

> Cisco's in sending frames of more than 1500 bytes regardless.

There's something wrong with the Juniper router behaviour, certainly in
the sense of not being "conservative in what you do".  What I think the
author of the code was trying to do was to express no opinion at all
on the MRU, since the earliest router hardware could receive as big a
packet as it was possible to send so whatever was configured as the MTU
on the other end of the link was fine, but I think he misread the fact
that there is no way to do this with PPP as specified.  This will be
fixed shortly.

As for Cisco, however, it isn't so clear to me that what they do is
in fact "wrong", despite the fact that it seems to violate RFC 1661.
Since I once, long ago, went through this issue and came up with a similar
answer I am going to guess that this could instead be an example of how old
protocol documents sometimes never die.  In RFC 1171 the text concerning
the length of the Information Field said

                                   The default maximum length of the
      Information field is 1500 octets.  By prior agreement, consenting
      PPP implementations may use other values for the maximum
      Information field length.

I know this was often interpreted as meaning that if you chose the option
of not implementing the MRU option at all then you were free to use the
method everyone used prior to that, i.e. using configured MTUs on
each end of the link.  As it was rare that a pre-PPP user interface
would even mention MRU, it is not surprising that this would be
interpreted in a way which both made the option truly optional
(i.e. you could have any MTU you wanted even if you didn't do the
option) and allowed the easiest backward-compatibility.

RFC 1331 removed any ambiguity by making it clearer that the option wasn't
really optional unless you were happy keeping all MRUs at 1500 bytes,
which is normally not what you want on "big" circuits.  The problem was
that this fix wasn't backward-compatible with RFC 1171 implementations
(likely including your own, if you had one), in that if you did an
implementation which insisted on RFC 1331 behaviour then installing it
on a box talking to (your own implementation of?) RFC 1171 will cause you
to drop your MTU on the circuit to 1500 with absolutely no way to change
it short of replacing the software on the box at the other end (and then
replacing the software of his neighbors, and so on).  This was a deployment
nightmare.

I think what I decided to do instead to relieve the deployment issue
while getting as close to the behaviour of the new standard as possible,
in an MTU-configuration-centric implementation, was to:

- Always send an MRU option, even if the value being sent was 1500
  bytes, omitting it only to deal with a -Reject (the configured MTU
  was the first choice for the MRU, the hardware maximum was sent to
  respond to a -Nak).  Always sending the option ensured that the
  new implementation would at least never end up confused talking to
  itself, or anyone else which always sends the option.

- Take no received MRU option to mean, use the configured MTU.
  Otherwise, if he sent an MRU, use the configured MTU for the active
  MTU if the received MRU was greater than or equal to this, the
  received MRU otherwise.

This appears to be approximately Cisco's behaviour too, for whatever
reason.  While I don't know their reason, if it is the same reason then
I'd point out that this is behaviour which, once you've done it, is hard
to ever undo, partly because you can't really tell when all the boxes
you needed to interoperate with like this are gone, and partly because
it allows a new generation of boxes which inadvertently depend on the
behaviour to come to exist.  And given that I now know boxes which have
a bug which depends on this I suspect I have a good idea of how that bug
will be fixed.

I'd also just point out that the view expressed here that PPP should always
negotiate MRUs (which is also the only way I can interpret RFC 1661
if the end result is supposed to be other than 1500), and that bad things
will necessarily happen if PPP is ever allowed to let the link come up
with an inconsistent MTU/MRU, is a bit PPP-myopic since there is at least
one common situation where the PPP checks are actually redundant to checks
made after the circuit is open.  If the circuit is between two routers
and OSPF (or IS-IS, I'm pretty sure) is in use, the IGP will also check
that the MTUs in use are compatible and will fail to form an adjacency
if they aren't, which prevents the circuit from ever being put to use
even if PPP is up.  In this case it may have been preferable to be able
to have PPP not bother with MRU negotiation itself, if this were an
option, since the problem will be caught regardless and it may be more
convenient to have the circuit up when you need to fix it so you can use
it to log into the neighbour box and repair its faulty configuration.  We
do have a pile of experience with link encapsulations which do not verify
MTUs themselves which still seem to work out just fine in this application.

And to share one other suprising thing I just noticed, while Juniper
has built quite a large number of HDLC over SONET interfaces, and while
I am aware of a big variety of equipment it has been connected to via
these interfaces, there has never been a user-initiated bug report
concerning this behaviour (it has be noticed internally a bunch of times,
though not fixed for hysterical reasons), even though it should be failing
to interoperate with implementations strictly conforming to RFC 1661 in
an unfortunate way.  I have no idea whether this means the Cisco
implementation of MRU negotiation is common, or whether it means the
use of PPP on interfaces of this type is fairly uncommon, however.

Dennis Ferguson

---

From: William Allen Simpson <[email-masked]>
Date: Thu, 30 Jan 2003 10:50:37 -0500

I'd thought this thread mercifully died off....  My comment is after 
the text:

James Carlson wrote:
>...
> I do think things would be much *worse* if we elevated the documents
> above proven interoperability.
>... 
> Blind adherence to RFCs generally, in my experience, produces garbage
> -- the whole point of this process is to produce interoperable
> implementations.  If you don't bother trying to make your
> implementation work with any other implementation, then you're
> completely missing the point.  You're confusing the map and the
> terrain.
>... 
> In other words, you can't do any of this while walling yourself off
> from the world.
>... 
> In *some* cases, there is such a big player, and he'll generally make
> the mistakes that others have to follow.  In *other* cases, there
> isn't such a player.  And there are some vendors who are more
> cooperative and collaborative than others.
> 
> The best case, I think, is that those with prototype implementations
> get together at a bake-off and settle the issues before the market
> ever sees them.  This doesn't always happen, though, nor does it
> always catch every important error.
>... 
> I still think you're missing the whole point of the IETF.  We're here
> to try to create interoperable solutions.  That's the shared goal.
> Those who are willfully engaged in producing non-interoperable junk
> are just not playing the game.  They're vandals.  Other than perhaps
> public humiliation, we don't really have any enforcement powers to
> make people "comply" with the IETF.
> 

Dennis Ferguson wrote:
>... 

As a historical note, RFC-1171 was tested for interoperability at 
InterOp 1990, and none (that's right, absolutely zero) of the vendors 
were able to interoperate using async, and only 2 with sync. 

This lead to the clarifications (and re-design) of RFC-1331, which had 
fairly widespread interoperability at InterOp 1991, such as: 

                  The default maximum length of the Information field
   is 1500 octets.  By negotiation, consenting PPP implementations may
   use other values for the maximum Information field length.

Eventually (when it was discovered that folks didn't understand 
whether to count Protocol and Padding, either) culminating in RFC-1661:

      The maximum length for the Information field, including Padding,
      but not including the Protocol field, is termed the Maximum
      Receive Unit (MRU), which defaults to 1500 octets.  By
      negotiation, consenting PPP implementations may use other values
      for the MRU.

As to the problem of failing to test against other vendors, in those 
days Cisco was the prime example.  They had the biggest market share, 
and just expected everybody to follow them....  

Indeed, this lead to "open" wars, such as the Open SPF.  To this day, 
Cisco still ships boxen defaulting to their proprietary HDLC variant, 
instead of PPP.

As noted by Carlson, sometimes we had to make allowances for the big 
guys.  We ended up tweaking the PPP FSA, because otherwise nobody could 
talk to Cisco.  Cisco had a terrible design flaw, where packets queued 
before a link went down, and then were sent out when the link came up.  
Every other vendor could flush their buffers.  So, we figured out how 
to toss all the Cisco bogons and successfully interoperate.

Later, there was an even more amazing example of failing to test an 
implementation.  Shiva came out with a relatively inexpensive box that
did well in the market.  But, after selling thousands of units, they 
discovered that they'd miscalculated the FCS.  They could only talk to 
themselves.  

This really clarified the need for interoperability testing at every 
step by everybody -- while drafting the RFC, and with quality assurance 
before sales, purchasing, and deployment. 
-- 
William Allen Simpson
    Key fingerprint =  17 40 5E 67 15 6F 31 26  DD 0D B9 9B 6A 15 2C 32