Subject: [Int-area] Re: [tsv-area] Fwd: I-D ACTION:draft-heffner-frag-harmful-03.txt

From: Bob Briscoe <[email-masked]>
Date: Wed, 03 Jan 2007 13:45:42 +0000

Matt, John,

1/ During the lifetime of this draft its scope has become restricted to 
soley describing the problem. That's good (and it's a good solid draft), 
but the abstract or intro needs to explictly say what it is deliberately 
not setting out to say (not describing partial solutions, not proposing 
solutions).

2/ Given the new restricted scope, the title is wrong - it's snappy, but 
not appropriate for this draft any more. It should be "Field Wrapping 
Problems with IP Fragmentation" or some such.

"IPv4 Fragmentation Considered Very Harmful" attributes blame squarely on 
the fragmentation protocol (as opposed to re-assembly implementations - see 
later). This title effectively says "You SHOULD NOT fragment with a 16b ID 
field", which is beyond what the text dares to say, and it's beyond what an 
informational draft should say.

Even if this was a BCP rather than informational, I would say it would be 
wrong to deprecate 16b fragmentation anyway. There are good reasons why 
fragmentation is useful (e.g. in tunnels), so we need to try really hard to 
find robust ways to do it before writing it off as deprecated. Saying it's 
very harmful should only be a last resort if we /prove/ it cannot ever be 
done robustly.

As we know, one way to fragment with improved robustness is to use more 
bits for the ID field. But if 16b fragmentation is a problem now, 32b 
fragmentation will be a problem in the future (not so distant future as 
Matt pointed out on int-area 
<http://osdir.com/ml/[email-masked]/msg00545.html>). IP is 
sufficiently pivotal at the neck of the hour glass that anything we say 
about it should endure for decades. I would contend that the IETF shouldn't 
condone putting off a problem to a later date when we know it will return. 
The present title leads us towards that sort of solution, implying "32b ID 
fields aren't [ever] very harmful" - on a draft that isn't even meant to be 
discussing solutions.

Given hierarchical layering is here to stay (and always has been), it would 
be more fruitful to admit that we need to be able to do fragmentation 
robustly and so we cannot avoid choosing an ID field width that will
- either not be wide enough at some future time
- or will be overly wasteful today.

In this vein, it would be useful to focus everyone on designing better 
re-assembly /implementations/ around a 16b fragmentation /protocol/ (see a 
possible idea below). There is no proof yet that we have reached the end of 
our innovation potential on this.

A sketch idea for a more robust re-assembly implementation:
On receipt of each fragment, within the re-assembly implementation increase 
the precision of the ID field by adding a "received timestamp" of 
sufficient precision. Then on a first pass, match fragments only if the 
fragment IDs match AND the timestamps are within a certain narrow range of 
each other. Otherwise hold the fragment and, as a last resort later, widen 
the timestamp range that will cause a match - perhaps when the fragment is 
about to be expired from the buffer (...rest of implementation left as an 
exercise for the reader).

In summary, a 16bit fragment ID field should be innocent until proven 
guilty. As long as the culprit might be /implementations/, the title 
shouldn't presume the IPv4 fragmentation /protocol/ is guilty.

3/ The draft should say something about how the problem gets worse if the 
sender uses a pseudo-random number generator for the IPid field (as recent 
versions of OpenBSD and some versions of FreeBSD do). Then there is no 
longer a deterministic wrapping problem, but there is /always/ some small 
probability of a clash within the max packet lifetime. A good ref for this is:

S. Bellovin, ``A Technique for Counting NATted Hosts,'' Proceedings of the 
Second Internet Measurement Workshop, November 2002. 
http://www.cs.columbia.edu/~smb/papers/fnat.pdf

Bob

At 07:06 06/12/2006, Lars Eggert wrote:
>Hi,
>
>could the people who had commented on the -02 revision during IETF LC
>please take a look whether the latest revision addresses their issues?
>
>Begin forwarded message:
>>A New Internet-Draft is available from the on-line Internet-Drafts
>>directories.
>>
>>         Title           : IPv4 Fragmentation Considered Very Harmful
>>         Author(s)       : J. Heffner, et al.
>>         Filename        : draft-heffner-frag-harmful-03.txt
>>         Pages           : 9
>>         Date            : 2006-12-5
>
>Thanks,
>Lars
>
>
>

____________________________________________________________________________
Notice: This contribution is the personal view of the author and does not 
necessarily reflect the technical nor commercial direction of BT plc.
____________________________________________________________________________
Bob Briscoe,                           Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196 

_______________________________________________
Int-area mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/int-area

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jan 2007 11:46:16 -0800

Hello Bob,

Your message is very interesting, and I believe a fresh new
way of looking at the situation - especially for fragmentation
that occurs within tunnels. I appreciate what you are saying,
and look forward to further dialogue on it.

I have been looking at fragmentation over IP*-in-IPv4 tunnels
for some time, and 'draft-templin-linkadapt-03.txt' proposes
an Alternate Fragmentation (AF) scheme that occurs below the
transport layer segmentation (e.g., Packetization Layer Path
MTU Discovery) but above IPv4 fragmentation. It supports
segmentation/reassembly at the tunnel endpoints and uses a
dynamic segment size probing mechanism within the tunnel to
avoid in-the-network IPv4 fragmentation, yet remains compatible
with in-the-network fragmentation should it occur. Use of the
scheme is indicated by setting the reserved bit in the IPv4
header 'Flags' field (to be renamed as the 'AF' bit), thus
it obsoletes RFC3514 (an "April Fools Day" RFC).

This AF scheme codes the 'ip_id' field in the IPv4 header and
thus presents a shorter-than-16b ID (the current draft version
specifies only a 6b ID). The assumption (which I think is also
an assumption you are making) is that successful reassembly
will occur within a very short time window if it will occur
at all. This assumes that reassembly failure will normally
occur due to packet loss rather than gross reordering of
packets within the same flow.

There have been many studies on packet reordering within the
Internet, but I have not found one yet that can completely
characterize the *degree* of reordering, i.e., the expected
number of places by which a reordered packet is out-of-order.
Most of the studies I have seen seem to suggest that the
expected degree of reordering of packets within a short chain
of packets sent in rapid succession within the same flow is
typically very small, e.g., a reordering event such as
(1,2,4,5,3,6,...) may occur occasionally, while one such
as (1,2,4,5,...,64k,3,64k+1,...) most likely will not. Other
factors to consider are: 1) as you observe, lengthening the
ID field may be insufficient in the presence of gross
reordering, and 2) transports such as TCP are likely to
treat grossly reordered packets as loss anyway.

Benefits of this AF scheme in relation to IPv4 fragmentation
are:

  - avoidance of in-the-network fragmentation by dynamic
    segment size probing at the tunnel endpoints
  - authenticated probe response from the decapsulating
    tunnel endpoint to avoid false-positives from on- and
    off-path attackers
  - flow-control to avoid reassembly buffer overruns
  - reduced reassembly buffer memory requirements
  - avoidance of undetected reassembly errors via a
    trailing checksum that is dissimilar from the
    Internet checksum
  - larger MTUs for tunnels

Finally, this work has been around for some time now, and
I believe has been reviewed by many while few have commented.
Perhaps now is the time for discussion on a wider basis.

Thanks - Fred
[email-masked]

PS: Errata in the current draft version:

   - the current draft only speaks to IPv6-in-IPv4 tunnels
   - the ICMP messaging in the current draft may be too
     IPv6-centric
   - network byte ordering issue in the ip_id coding ('A';
     'P' bits should be MSBs; not LSBs)
   - network byte order coding of trailing checksum needs
     to be clarified
   - draft title should be changed to something like
     "Alternate Fragmentation for IP-in-IP Tunnels"

   The intention is to fix these in a -04 version, along
   with any other change suggestions that may come. 

> -----Original Message-----
> From: Bob Briscoe [mailto:[email-masked]] 
> Sent: Wednesday, January 03, 2007 5:46 AM
> To: Matt Mathis; John Heffner
> Cc: [email-masked]; [email-masked]
> Subject: [Int-area] Re: [tsv-area] Fwd: 
> I-DACTION:draft-heffner-frag-harmful-03.txt 
> 
> Matt, John,
>

---

From: John Heffner <[email-masked]>
Date: Wed, 03 Jan 2007 15:39:11 -0500

Bob, thanks for the insightful comments.  Response inline...

Bob Briscoe wrote:
> "IPv4 Fragmentation Considered Very Harmful" attributes blame squarely 
> on the fragmentation protocol (as opposed to re-assembly implementations 
> - see later). This title effectively says "You SHOULD NOT fragment with 
> a 16b ID field", which is beyond what the text dares to say, and it's 
> beyond what an informational draft should say.

This is basically what the IESG review said and we agree.  The text in 
the abstract is not clear.  The intent is not to say the IPv4 
fragmentation should never be used, but that it is not safe for use 
under some conditions.  Basically, we feel that for some (esp. low-rate) 
applications such as DNSSEC it's probably fine, but for other purposes, 
especially bulk transport, it's not suitable.  Note, however, that even 
low-rate fragmenting applications can be exposed to problems if a 
high-rate non-fragmenting application is running at the same time.

I think the text of the document is adequately clear on this, but we 
will be rewording the abstract.

About the title: it was meant to indicate that this document is 
documenting an additional problem (which is in fact very harmful when it 
occurs), and is extension, not a substitute, for the landmark Kent/Mogul 
"Fragmentation considered harmful" paper.  It seems to have caused 
sufficient confusion and misinterpretation that it is probably worth 
reconsidering the title.

> Even if this was a BCP rather than informational, I would say it would 
> be wrong to deprecate 16b fragmentation anyway. There are good reasons 
> why fragmentation is useful (e.g. in tunnels), so we need to try really 
> hard to find robust ways to do it before writing it off as deprecated. 
> Saying it's very harmful should only be a last resort if we /prove/ it 
> cannot ever be done robustly.
> 
> about it should endure for decades. I would contend that the IETF 
> shouldn't condone putting off a problem to a later date when we know it 
> will return. The present title leads us towards that sort of solution, 
> implying "32b ID fields aren't [ever] very harmful" - on a draft that 
> isn't even meant to be discussing solutions.
> 
> Given hierarchical layering is here to stay (and always has been), it 
> would be more fruitful to admit that we need to be able to do 
> fragmentation robustly and so we cannot avoid choosing an ID field width 
> that will
> re-assembly /implementations/ around a 16b fragmentation /protocol/ (see 
> a possible idea below). There is no proof yet that we have reached the 
> end of our innovation potential on this.
> 
> On receipt of each fragment, within the re-assembly implementation 
> increase the precision of the ID field by adding a "received timestamp" 
> of sufficient precision. Then on a first pass, match fragments only if 
> the fragment IDs match AND the timestamps are within a certain narrow 
> range of each other. Otherwise hold the fragment and, as a last resort 
> later, widen the timestamp range that will cause a match - perhaps when 
> the fragment is about to be expired from the buffer (...rest of 
> implementation left as an exercise for the reader).
> 

Originally we had thoughts of trying to prescribe an implementation fix. 
  There are certainly a lot of possible approaches.  Linux has just 
recently implemented a scheme where fragments time out not by wall clock 
time, but by the number of intermediate packets received, effectively 
having a tunable parameter on the amount of reordering tolerated.  This 
seems to work pretty well.

However, there really is a protocol problem that prevented us from 
trying to prescribe a solution.  The sender is responsible for not 
reusing IPIDs before the fragments time out, but the receiver is 
responsible for timing out fragments.  There is no way to the sender to 
know if a receiver has timed out the old fragments yet.

As an operations person, I might be able to deploy fragmentation with 
some confidence if I can ensure that all receivers I'm using are running 
recent Linux kernels with the workaround enabled.  But as an application 
developer, there's no way for me to be sure I won't ever run in to 
protocol trouble if I rely on fragmentation.

> 3/ The draft should say something about how the problem gets worse if 
> the sender uses a pseudo-random number generator for the IPid field (as 
> recent versions of OpenBSD and some versions of FreeBSD do). Then there 
> is no longer a deterministic wrapping problem, but there is /always/ 
> some small probability of a clash within the max packet lifetime. A good 
> ref for this is:
> 
> S. Bellovin, ``A Technique for Counting NATted Hosts,'' Proceedings of 
> the Second Internet Measurement Workshop, November 2002. 

Yes, we can add this.

Thanks,
   -John

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jan 2007 14:17:11 -0800

> > http://www.cs.columbia.edu/~smb/papers/fnat.pdf

Maybe consider adding this too:

"Beyond Folklore: Observations on Fragmented Traffic"
http://www.caida.org/publications/papers/2002/Frag/frag.pdf

Fred

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 04 Jan 2007 15:19:54 +0000

John,

>Originally we had thoughts of trying to prescribe an implementation fix.

I still think this is the way to go. ...Well, at least not actually 
prescribing the algorithm, but prescribing (or recommending) its desired 
behaviour.

> seems to work pretty well.

That's nice. I was only using wall-clock time because it was a strictly 
monotonically increasing number. Concatenating an increasing index would 
work just as well (in fact better), which is effectively the same as the 
Linux idea. And yes, I was imagining that the receiver would adapt the 
allowed gap between re-assembled packets to circumstances (ie tunable could 
mean per flow, rather than configured per machine).

Heuristically tuning this parameter per flow could potentially even handle 
the problem you mention where the sender is spreading PacketIDs over other 
faster flows. Tho, having heuristic adaptation in an IP stack is a bit 
scary - how many vulnerabilities might that open up?

>However, there really is a protocol problem that prevented us from trying 
>to prescribe a solution.  The sender is responsible for not reusing IPIDs 
>before the fragments time out, but the receiver is responsible for timing 
>out fragments.  There is no way to the sender to know if a receiver has 
>timed out the old fragments yet.

This is only a 'real protocol problem' if we define a correct re-assembly 
function as one that is exactly 100% reliable. Even if a well-known timeout 
had been standardised (whether 15mins, 1min, 1sec), re-assembly wouldn't 
have been guaranteed reliable. It would only be possible to guarantee that 
received packets were not re-using a number before a time agreed between 
sender & rcvr if we used loose clock synch (e.g. I did one like this myself 
in RFC4082).

I don't believe anyone wants IP to take that road. So strictly, we don't 
have a real protocol problem, if 100% is an unattainable goal anyway at 
reasonable complexity.

So, firstly we need to state up front that IP fragment re-assembly is not 
expected to be exactly 100% reliable. Then we can say a lot can be done 
through good implementation to get within a gnat's tittle of 100%.

>As an operations person, I might be able to deploy fragmentation with some 
>confidence if I can ensure that all receivers I'm using are running recent 
>Linux kernels with the workaround enabled.  But as an application 

An app developer can always say "rcvr app requires at least Linux vX.Y, 
Windows 2052 etc".

The only property we (the IETF) need to ensure is that highly reliable 
reassembly doesn't rely on sender behaviour as well. Even with 
pseudo-random PacketID sequences, I think that's do-able by building on 
implementations like the Linux one above.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 04 Jan 2007 16:51:19 +0000

Fred,

At 19:46 03/01/2007, Templin, Fred L wrote:
>with in-the-network fragmentation should it occur.

I've scanned it.

My initial reaction is that is seems somewhat over-specific in places. It 
would seem better if only the bare bones needed for tunnel fragmentation 
was in there, rather than tying together the fragmentation scheme with the 
checksumming and so forth. Also I suspect the references to specific link 
technologies will seem dated in a few years --- it needs to justify some of 
the min & max MTU choices for the long term. Also, it would be useful to 
say why existing v6 fragment header mechanisms aren't applicable.

>Use of the

This seems a rather specific use for this last reserved bit, given things 
in the IP header are meant to be generally useful to upper layers.

Couldn't use of the coding scheme be negotiated between tunnel endpoints 
for all tunnelled packets carrying a specific src-dest address pair?

I have to declare an interest in the reserved (evil) bit myself at this 
point. I've been presenting a draft myself in tsvwg 
<draft-briscoe-tsvwg-re-ecn-tcp-03.txt> that relies on using it. However, I 
like to think we have a strong justification for grabbing it, as the above 
draft is the culmination of a decade of work to fix the resource allocation 
and accountability problems with the current Internet architecture.

Are you aware of any other claims on that bit? I'm sure there are many.

>packets within the same flow.

I'm not quite making that assumption. I'm saying re-assembly over shorter 
gaps should take priority over longer gaps, so re-assembly over longer gaps 
should be deferred for a while.

>treat grossly reordered packets as loss anyway.

I guess it depends when you look. If you catch a re-route in progress into 
a shorter path, you might see something more like your second example if 
the re-route affects very fast flows. So, the typical case might not be the 
only case we have to allow for.

>Perhaps now is the time for discussion on a wider basis.

I guess one immediate problem springs to mind. Current re-assembly 
implementations have had to be hardened against frag, tear-drop etc 
attacks. In defining a new coding of the packetID field, it will have to be 
resistant to attack from malicious sources. A new coding would open up a 
new set of vulnerabilities that would all have to be dreamt up then 
patched. For instance, malicious sources could spoof tunnelled packets to 
the decap endpoint as if they were from the encap endpoint, but send 
sequences of fragments that don't fit together correctly, blowing its 
memory, causing it to hang, or whatever.

Bob

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 4 Jan 2007 16:01:03 -0800

Hi Bob, 

> >with in-the-network fragmentation should it occur.

OK - thanks.

> checksumming and so forth.

The trailing checksum is there to detect packet splicing errors,
memory errors, driver bugs, bogus segments, etc. and discard
corrupted data that might not otherwise be detected by the
Internet checksum. 

> Also I suspect the references to specific link 
> technologies will seem dated in a few years

Maybe so, but to a certain extent I believe injecting a bit of
history here is useful in terms of leveraging past experience
moving forward. For example, the ATM cell size is coincidentally
similar to the minimum IPv4 MTU of 68 bytes [RFC791], and AAL5
is coincidentally similar to the segmentation/reassembly being
proposed in the document.

> --- it needs to justify some of 
> the min & max MTU choices for the long term.

Well, for IPv6 we have a MinMTU of 1280 bytes for the long term
and I don't expect we will see that change soon. In terms of the
MaxMTU, my best-read of Jonathan Stone's thesis on "Checksums
in the Internet" was that the incidence of undetected errors
for 32b checksums becomes non-negligible for packet sizes larger
than about 9KB, which is coincidentally similar to the 9180 byte
MTU for IP over ATM. Future links that support even larger packet
sizes with strong error checking may allow for still larger
tunnel MTUs, but I'm not an expert on packet size interactions
with error detection mechanisms so would appreciate if others
who know more about it could comment.

> Also, it would be useful to 

IPv6 sees the tunnel as an ordinary link that needs to present
an assured MTU of 1280 bytes or larger. If the tunnel has to
segment IPv6 packets into smaller chunks in order to fulfill
the MTU it advertises to IPv6, then that is a L2 issue with
IPv4 being the L2 encapsulation protocol in question.

If the encapsulation were to include a full IPv6 header with
frag header in every segment, that would be 48 bytes of IPv6
header plus 20 bytes of IPv4 header leaving 0 bytes for ULP
data on links that support only the minimum IPv4 MTU of 68
bytes. It would also be inefficient in general even on links
with larger-than-68b MTUs, and something of a layer violation.

That said, it may be useful to include just the first 4 octets
of the IPv6 header in the encapsulation of non-initial segments.
That would give the 8b Traffic Class and 20b Flow Label which
could be used by the reassembler to further disambiguate flows.
 
> in the IP header are meant to be generally useful to upper layers.

Well, the bit occurs in the flags field associated with
fragmentation, so defining it to indicate an Alternate
Fragmentation (AF) scheme would seem like a natural fit.
 
> for all tunnelled packets carrying a specific src-dest address pair?

This might work for persistent configured tunnels that are
established through some initial negotiation phase, but 
might not be so practical for automatic tunnels or tunnels
that are short-lived and/or only carry a small number of
packets. It is also possible that this AF scheme may be used
by IPv4 packetization layers other than tunnels and may even
supplement or replace RFC791 IPv4 fragmentation.

> <draft-briscoe-tsvwg-re-ecn-tcp-03.txt> that relies on using 
> it. However, I like to think we have a strong justification for
grabbing it, 
> as the above draft is the culmination of a decade of work to fix the 
> resource allocation and accountability problems with the current
Internet architecture.

I know that RFC3514 had tounge-in-cheek when it used the term
"evil", but to be sure we are talking about precious real estate
here. I have been working on the tunnel MTU stuff since 2002
when the ngtrans wg closed and re-emerged as v6ops.

>From what I can tell one question comes down to whether we can
expect to see IPv6-in-IPv4 tunnels deployed on a wide scale and
on a long-term basis in the Internet? If so, we will need tunnel
MTU assurance that provides robustness and efficiency over
arbitrary Internet paths.

Another question comes to whether such an AF scheme could
supplement or replace RFC791 IPv4 fragmentation (along with
its myriad issues that are well documented) even for
non-tunnel applications.

> Are you aware of any other claims on that bit? I'm sure there 

I knew of one proposal several years back, but to my knowledge
no I-Ds were ever issued and I don't know of any other claims.
Maybe someone else knows of other proposals.

> I'm not quite making that assumption. I'm saying re-assembly 
> over shorter gaps should take priority over longer gaps, so
> re-assembly over longer gaps should be deferred for a while.

That's fine, but my intuition is that applications and
transports are not very tolerant of grossly reordered packets
and some may prefer to receive partial data (e.g. using UDPlite)
over data that arrives too late.
 
> I guess it depends when you look. If you catch a re-route in 
> progress into a shorter path, you might see something more like
> your second example if the re-route affects very fast flows. So,
> the typical case might not be the only case we have to allow for.

Route flaps happen all the time in the environments I care
about, and in many cases loss of one or a couple of packets
is unavoidable. Reliable transports already take care of this,
and mechanisms like UDPlite may be useful for applications
such as streaming media.
 
> memory, causing it to hang, or whatever.

The spec gives very specific instructions on how packets are to
be segmented that are actually a lot less flexible than RFC791
IPv4 fragmentation. In particular, all segments except the final
segment must be the same size (such that only a small integer
segment ID and not an offset are necessary) and the final byte
of the ith segment is the one that immediately precedes the first
byte of the i+1th segment. This makes reassembly much easier, and
if an attacker injects a bogus segment it will either be recognized
immediately as a martian or be caught by the trailing checksum
during reassembly. I'm sure this requires a more careful analysis,
however.

Thanks - Fred

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 10 Jan 2007 15:44:39 +0000

Fred,

>supplement or replace RFC791 IPv4 fragmentation.

If we're expecting to be able to fire a few tunnelled packets at a router 
and get it to decapsulate and re-assemble them to your spec, we either:
- have to know that all routers implement your spec (which we already know 
they don't)
- or somehow negotiate its capabilities with it first,
- or something else tells you it's capable (e.g. you are configured to 
already know).
The only case for using a per-packet flag would be if, within the same 
tunnel, some packets used your scheme and others didn't.

If you don't absolutely need to use the last available bit in the IPv4 
header, surely you are morely likely to get the proposal through without 
using it.

>arbitrary Internet paths.

Yes, we'll need a robust solution, but initial capability negotiation is no 
less robust than per packet negotiation if you don't need per packet 
negotiation.

>its myriad issues that are well documented) even for

Even if it replaces RFC791, there will still be a legacy.

>over data that arrives too late.

Surely that can be implementation dependent. There's no need to mandate 
that long gaps are thrown away if its possible to solve the problem by 
merely deferring them behind short gaps.

Cheers

Bob

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Jan 2007 09:10:40 -0800

Hi Bob, 

> tunnel, some packets used your scheme and others didn't.

In terms of initial negotiations, the encapsulator can
send the first packet out according to the AF scheme with
the Probe ("P") bit set in at least one segment and cache
the original packet. If a probe response comes back, the
encapsulator discovers that the decapsulator implements
the scheme and can continue to use AF for subsequent
packets. Otherwise, the encapsulator can use an algorithm
such as:

  if(cached_packet_len <= 1280) {
    do not set the DF bit
    send packet
  } else {
    drop packet
    send ICMPv6 "packet too big" to source
  }

and can then send all subsequent packets under the assumption
that this decapsulator does not implement AF.
   
> Even if it replaces RFC791, there will still be a legacy.

"Replace" is something I ought not to have said, and I want
to apologize for this and any other presumptuous tones my
previous messages may have taken. We are talking about *the*
Internet Protocol here, and there should be no presumption
that we now somehow have a way of obsoleting a core function
of IP. A better term to have used than supplement/replace/avoid
etc. may have been "coexist".

> merely deferring them behind short gaps.

I also prefer successful reassembly, and don't mean to mandate
any particular behavior. But I also believe (without proof)
that deferred reassembly for grossly-reorderd packets within
the same flow may be little/no different than declaring the
packets as loss after a reasonable timeout window.

Thanks - Fred