Subject: On packet reordering... (was Nr in data packets)

From: Vernon Schryver <[email-masked]>
Date: Tue, 25 Jan 2000 16:19:36 -0700 (MST)

> From: "W. Mark Townsley" <[email-masked]>

> ...
> being switched to. For example, since traceroute is a series of pings to

Just as an asside, traceroute is not a series of pings if that means ICMP
Echo-Requests.  Classic traceroute uses UDP packets, each to a different
UDP port number, and so apparently parts of different streams.  Microsoft's
`tracert` uses ICMP Echo-Requests.  I've heard but not verified that
FreeBSD's 4.? traceroute can optionally use TCP, GRE, and ICMP.

> ...
> Vernon, I'm not disputing that preserving order for a TCP connection is
> not a Good Thing. Let me ask, at what point do you suggest we stop
> peeking into a packet to determine the flow? I doubt any router looks
> inside UDP, then L2TP, then PPP, then IP, then TCP to try and keep the
> flow ordered (or even the UDP packet to preserve L2TP flow alone). At
> some point, the cost of all of this lookup becomes more painful than the
> gain. I would be happy to see someone take as an action item researching
> this and providing the results. Even if it is proven that we should
> perform all this flow checking, it still cannot be guaranteed (as it
> certainly could be configured off).

You keep implying I support obviously stupid things.  Just as I never
supported the silly notion of fixing the order of already miss-ordered
TCP/IP segments, I've said nothing about arbitrary computations on
payloads.  My point is not that IP reordering must never happen; that
would be stupid.  It is not even that an L2TP router must never re-order
IP/PPP packets; rare reordering is ok (modulo CCP, BP, ECP, etc), and even
frequent re-ordering across streams.  My point is that turning off L2TP
sequence numbers in a router with badly broken IP fragmentation machinery
is a not a work-around.  I don't care why or how the fragmentation
machinery is broken.  It doesn't matter if some packet parsing assumptions
that worked elsewhere don't work for L2TP.  The router is badly broken,
and there is no choice but fixing it.  If the fix has has costs (what
doesn't?), tough luck.

Worse and genuinely shocking, you seem to support L2TP implementations
that have sequence numbers turned off all of the time, even on
packets that "MUST NOT" be reordered.

> ...
> Thus, if you are running IP only (and are running over an IP network as
> opposed to L2TP over ATM or some such), then I support disabling
> sequence checking after the CPs have finished. For other protocols,
> particularly stateful compression, encryption and PPP CPs, I recommend
> enabling sequence numbers.

I think that's reasonable.

Separately, any router that does a lot of re-ordering of IP packets within
a stream is badly broken and will eventually be trashed in the market.
Turning off L2TP sequence numbers does not fix or work around that serious
bug.

>                            That said, most implementations I see today
> simply have sequencing off from the start. 

Do you think that is reasonable?  I not only think it's unreasonable,
but I hope never to have to work with any of the people who wrote that
code, unless they are even now busy fixing it.

Vernon Schryver    [email-masked]

---

From: "W. Mark Townsley" <[email-masked]>
Date: Tue, 25 Jan 2000 23:29:00 -0500

Trying to summarize.

Let's get away from whether an IP stack is broken or not when it decides
to reorder packets. I don't have the time to discuss it further here.
Even if we came up with a BCP for IP stack ordering in the face of
fragmentation, etc. it is not something we are going to include in the
L2TP documentation. I will put my trust in those whose livelihoods
depend on this, and who have probably been through the same arguments a
thousand times. 

Barney, I am in full support of PMTU discovery for L2TP, and there are
those that are working on this already. Be on the lookout for a
proposal. That said, we still have to operate in the wake of PMTU not. 

Thus, I still say, for IP traffic tunneled by L2TP:

"Be conservative in what you send..." e.g. _try_ not to reorder IP
packets
"and liberal in what you accept" e.g. pass the out of order packets to
IP on receipt and let it sort out what to do. 

For non-IP packets, and those with ECP/CCP/VJ etc., I suggest checking
sequence as RFC2661 already states. I *think* I even heard Vernon state
that this is reasonable.

Those are the relevant points for L2TP as I see them. If I missed any,
please let me know. 

- Mark

---

From: David Chen <[email-masked]>
Date: Fri, 28 Jan 2000 10:50:33 -0500

My argument is that "the Ns bit in L2tp data channel is not needed".
Here is my analysis:
 From LAC to LNS:
For those PPP (out of order) packet sequence 1,2,4,5,6,7,8,3,9,10 arriving 
at LAC if
1) L2tp Ns bit is off
then the PPP in LNS will receive 1,2,4,5,6,7,8,3,9,10  ---> same sequence
2) L2tp Ns bit is on
then the PPP in LNS will receive 1,2,4,5,6,7,8,9,10 ---> packet 3 is droped

The question is "Does any protocol that is encapsulated by the PPP
will not break if a packet is dropped (case 2)
but will break if the packet delivery sequence is out of order (case 1) ?"

Are you suggesting those protocol other than IP that encapsulated by PPP
satisfy this question?

--- David

>Those are the relevant points for L2TP as I see them. If I missed any,

========================================
David Chen
Indus River Networks, Inc.
31 Nagog Park
Acton, MA 01720
U.S.A.
[email-masked]
(978) 266-8141
========================================

---

From: David Chen <[email-masked]>
Date: Fri, 28 Jan 2000 11:00:31 -0500

This is a correction of the following analysis of my last posting,
I apologize the confusion:

My argument is that "the Ns bit in L2tp data channel is not needed".
Here is my analysis:
 From LAC to LNS:
For those PPP packet sequence 1,2,3,4,5,6,7,8,9,10 arriving at LAC,
however the delivery from LAC to LNS is out of sequence, if
1) L2tp Ns bit is off
then the PPP in LNS will receive 1,2,4,5,6,7,8,3,9,10  ---> out of sequence
2) L2tp Ns bit is on
then the PPP in LNS will receive 1,2,4,5,6,7,8,9,10 ---> packet 3 is droped

--- David