Subject: Re: Nr in data packets

From: Vernon Schryver <[email-masked]>
Date: Tue, 25 Jan 2000 15:03:26 -0700 (MST)

> From: Grenville Armitage <[email-masked]>
> To: "W. Mark Townsley" <[email-masked]>

> > I won't claim to be the foremost IP stack expert for a moment, but
> > common sense to me says that if I receive an IP packet that must be
> > fragmented followed by a bunch of smaller packets that do not, it makes
> > sense to go and and switch the smaller packets rather than queuing them
> > while I chew on the fragmentation.
>
> Then our definition of 'common sense' differs ;) Given a goal of
> optimizing end to end performance, the above is not common sense
> (although on the surface it is an appealing argument from the
> perspective of the node facing the sequence of packets you describe).
>
> If the packets do not belong to the same flow (or aren't even
> targetted at the same IP destination) then process in whatever
> order you like - but that's not "reordering" in the sense being
> discussed here.

Those are good responses, but a better is that any system that needs
significant time to IP fragment is broken.  IP fragmentation is a triviaL
operation, especially at the bit rates we are talking about.  IP
fragmentation consists of adding 20 bytes plus a lower layer header to
the middle of a packet, and doing thing else.  Those 20 bytes of new IP
header are identical to the previous IP header except for two or three
trivial adjustments that are independent of everything except the number
of intervening payload bytes.

> 	[..]
> > Even if
> > the packets are associated with the same TCP stream (which on a middle
> > node they are likely not) latency can be as much of a burden to
> > performance as packet loss (ask anyone in tcpsat about that).
>
> Latency happens. But reordering fragments of packet N
> in order to expedite packets N+1, N+2, .. of the same TCP stream
> is false economy. You'll just be screwing with TCP's mind, and
> in the end your goodput will be lower than if you'd simply let
> nature take its course and kept the stream's packets in order.
> ...

In other words, contrary to the claim, re-ordering a TCP stream as
described only hurts and never helps TCP latency.

Another way to look at that the issue is that if an L2TP box is
receiving 1500+ byte PPP frames and sending them over a UDP/IP link
with a smaller MTU cannot handle the inevitably implied heavy IP
fragmentation, then shuffling its output queue is not a solution.
If the router doesn't have the needed horsepower, rearranging its
output queue of deck chairs as it sinks isn't a good plan.

It is bogus to reason that since all IP protocols are required to
tolerate a little re-ordering, therefore turning off the L2TP
re-ordering protection is an ok idea.  Even if it were true that
IP protocols don't care about re-ordering (that is false), there
are other L2TP payloads that absolutely cannot be re-ordered.

Vernon Schryver    [email-masked]

---

From: Barney Wolff <[email-masked]>
Date: Tue, 25 Jan 2000 16:45 EST

There is an experiment that I wish someone with the necessary equipment
would try, that I think would shed some useful light on this issue.

SRC---FDDI---Router---Ethernet---DEST

Send alternating big (>1500) and small UDP packets from SRC to DEST,
and see if DEST indeed receives things out of sequence.

My reason for asking is that this is quite different from a scenario
where it is the L2TP stack's own network interface that is doing the
re-ordering.  In that case, I have to side with Vernon and say that
emitting things out of order is a bug.  But if an intervening router
can consistently re-order, not under the control of either LAC or
LNS, then the problem is very serious.  Perhaps in that case the
right thing to do is mandate DF and PMTU discovery, and force the
fragmentation to occur before the packet reaches anything that
will re-order.

Barney Wolff  <[email-masked]>

---

From: Vernon Schryver <[email-masked]>
Date: Thu, 27 Jan 2000 13:26:54 -0700 (MST)

> From: David Chen <[email-masked]>

> 1) It seems need more illustration to my point
>    For a L2TP packet such as PAYLOAD/TCP/IP1/PPP/L2TP/UDP/IP2 on the 
> protocol stack:
> The IP1, IP2, and UDP services does not guarantee packet delivery in sequence
> yet, TCP and PPP with flow control does.
> if L2TP service guarantee packet in sequence delivery then it does not
> help the PAYLOAD or TCP due to that the IP1 service still does not 
> guarantee packet delivery in sequence.

What is "PPP with flow control" RFC 1663?  If so, please consider the
implications of the title of RFC 1663.

The issue is not whether rare re-ordering by L2TP significantly harms TCP/IP.

I think the following are facts:
  - delivering out-of-order PPP packets breaks much of PPP.
  - significant miss-ordering significantly harms TCP. 
  - Rare miss-ordering is not significant to TCP.
I do not see any inconsistency between those facts and your observation above.

The case at hand is an L2TP router that does a lot of re-ordering and the
claim is that turning off L2TP sequence numbers makes the system useful.
That is false, because the broken router would continue to miss-order many
TCP/IP1 segments, and that would be a Bad Thing(tm) for TCP.

Or maybe the case at hand is your statement many messages ago that seemed
to me to be "L2TP Sequence Number Checking Considered Unnecessary."
I think the problems of PPP and especially CCP and ECP with re-ordering
bear on that statement.

> 2) The argument about LCP,NCP needs underlying protocol provide packet 
> delivery in
> sequence is not so clear:
> When sender send out Config-Req, it will not send any packet (to move the 
> statemachine)
> untill the receiver responds with Config-XXX....
> Can you illustrate how the PPP state machine send out more than one different
> protocol frame in one state... thereby need the packet sequence delivery 
> service

Consider {Conf-Req1, Conf-Req2, Conf-Ack99} re-ordered into
{Conf-Req2, Conf-Ack99, Conf-Req1}.
Yes, and as I said before, reasonable PPP implementations will respond to
the out of order Conf-Req1 by tearing down the PPP connection and all
upper CPs (but not necessarily lower CPs or modems etc), and starting over
with Conf-Req3.  Maybe you consider that good or at least OK.  I don't,
partly because if Conf-Req1 can be re-ordered, then so can Conf-Req3,
Conf-Req5, and so on forever.
   
Consider what a re-ordering L2TP box does to most CCP and ECP variations.
Re-ordering CCP can lead to user data corruption, because even the best
CCP checksums are weak and intended only to supplement sequence numbers
that detect lost packets.

> 3) For TCP's argument about needs packet sequence delivery service from IP 
> is impossible
> and its design drawback to handle this issue, please refer to the
> page 536, 3rd paragraph about "what to do with out of order segments" of 
> book "Computer Networks" by Andrew S. Tanenbaum, ISBN 0-13-349945-6.

I don't have a clue what is meant by "TCP's argument about needs packet
sequence delivery service from IP is impossible and its design drawback."

Instead of arguing by authority, I'd rather consider the facts directly.

If I were looking for a book about TCP/IP, Prof. Tanenbaums' is not among
those I'd think of, because I've not seen his name in the RFC index, on
respected TCP/IP code, nor in the IETF or IRTF mailing lists devoted to
TCP/IP technical issues.

By the way, is that the book in which Prof. Tanenbaum proclaimed that an
Ethernet cannot carry more than about 30% because he confused ALOHA with
CSMA/CD?  (I hope you know that 30% sounds egregiously silly to people
with valid clues about CSMA/CD.)  I've heard that textbook is good and
not bad as that example suggests.  However, that error does demonstrate
a difference between the expertise obtained by reading about things and
that earned by doing.  I bet many people here know more about current
networking topics than Prof. Tanenbaum, although I bet he knows more about
SNA, x.25, and even ALOHA.