Subject: [Ietf-behave] Requiring NATs to process Out-of-order fragmented TCP packets

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 7 Mar 2005 15:27:10 -0800 (PST)

My understanding from this morning's meeting was that we were leaning 
toward not requiring Behave-conformant NATs to process out-of-order 
TCP fragments, with the expectation that endpoints would be doing Path 
MTU discovery anyway, so fragmentation shouldn't be an issue.

I ran into Mark Allman this afternoon and asked him if he had an 
opinion on the robustness of Path MTU discovery on the Internet. He 
does, and has recently co-authored a paper that discusses this 
(available at http://www.icir.org/mallman/papers/tcp-evo-submit2.ps). 
In section 4.3 of the paper, he's seeing that about one-third of the 
85,000 servers identified by the NLANR Web Caching project don't 
attempt Path MTU discovery at all, and another 17 percent attempt it 
and fail. The failure rate is even higher for a list of 500 popular 
web server (35%).

So I'm wondering if we need to think about this a little more?

Spencer

p.s. I copied Mark, just in case I'm misunderstanding the paper...

---

From: Bryan Ford <[email-masked]>
Date: Thu, 17 Mar 2005 10:07:34 -0800 (PST)

Hi Mark,

Very nice work; your paper answers a lot of important questions.  Thanks for 
pointing it out!

Bryan

On Thursday 17 March 2005 12:45, Mark Allman wrote:
> > > > I'm for processing of in-order fragments--from what I discovered
> > > > while working on DTLS, PMTUD is rarely turned on,
> > >
> > > [suresh] You are right, Path MTU discovery is rarely turned on. Many
> > > applications (especially UDP based) are not even MTU mindful.
>
> I have not been following this thread, but some of my least favorite
> comments to hear around the IETF are hand-wavy conjecture about how the
> network works and what the traffic looks like.  So, let me try to
> re-iterate and expand on what Spencer said based on his reading of our
> paper...
>
> (1) Of the 80K+ web servers we probed we were able to make good data
>     connections to 71K+ web servers and found:
>
>     About one-third did not set DF, meaning the majority of servers did
>     in fact express a desire to use PMTUD.
>
>     47% of the servers used PMTUD just fine.
>
>     20% of the servers were not able to use PMTUD.  I.e., they did not
>     respond to ICMP "too big" messages by reducing their packet size.
>     Presumably this is caused by some middlebox blocking the ICMP.
>
>     None of the servers did PMTUD black hole discovery.  I.e., in the
>     absence of an ICMP "too big" or a data acknowledgment none of the
>     servers responded by reducing the segment size.
>
> (2) Our study only covers TCP and HTTP.
>
> (3) I would be highly surprised if non-HTTP TCP traffic showed
>     drastically different PMTUD usage patterns from HTTP traffic.  I
>     have been surprised before.
>
> (4) Our study does not consider non-TCP traffic at all.  I can't even
>     begin to conjecture about UDP.  But, I will offer a quick
>     measurement.  I crunched all the UDP packets that traversed ICSI's
>     border yesterday (@M+) and found none had DF lit up.  It's possible
>     that ICSI is abnormal, but probably not *that* abnormal.
>
> Our paper is:
>
>     Alberto Medina, Mark Allman, Sally Floyd. Measuring Interactions
>     Between Transport Protocols and Middleboxes. ACM SIGCOMM/USENIX
>     Internet Measurement Conference, Taormina, Sicily, Italy, October
>     2004.
>     http://www.icir.org/mallman/papers/tcp-evo-imc04.ps
>
> (section 4.2 is on PMTUD)
>
> For the interested, a longer version of the paper with more results
> (but, not more PMTUD results) is:
>
>     Alberto Medina, Mark Allman, Sally Floyd. Measuring the Evolution of
>     Transport Protocols in the Internet. December 2004. Under
>     submission.
>     http://www.icir.org/mallman/papers/tcp-evo-submit2.ps
>
> Enjoy!
>
> allman
>
>
>
> --
> Mark Allman -- ICIR -- http://www.icir.org/mallman/

---

From: Pyda Srisuresh <[email-masked]>
Date: Thu, 24 Mar 2005 08:39:51 -0800 (PST)

Here is what I would sugegst.

A NAT device MUST be capable of processing all fragments of an inbound IP
datagram. These fragments cannot be assumed to arrive in order. The NAT device
must retain the assembly state pertaining to a fragmentation ID until all
fragments of the IP datagram are processed. By doing this, NAT is able to
process all fragments pertaining to an IP datagram using the same NAT Session
the IP datagram belongs to. 

As with any stateful processing, there may be exceptions and limitations on how
long a state can be retained and how many simultaneous states a NAT device  can
retain at once. Thsi depends on the design and memory limitations of a NAT 
device and is not within the scope of this document.

cheers,
suresh

--- Paul Hoffman <paul.hoffman@xxxxxxxx> wrote:
> At 6:30 PM -0800 3/23/05, Senthil Sivakumar wrote:
> >My suggestion is:
> >
> >"NATs should handle IP fragments that are seen in order and out of order.
> >When an in order fragment is seen NAT should store the L4 information of
> >the first fragment so that it can match the subsequent fragments that do
> >not carry L4 headers. No buffering of the fragments is required in case of
> >in order fragments. For out of order fragments, NAT must store the
> >fragment until the first fragment is seen. The fragments should be stored
> >for a period of N seconds before discarding them. NATs must not store more
> >than X fragments per flow and must not store more than fragments for Y
> >flows at any point of time"
> >
> >The values of N, X and Y are to be determined by the network administrator
> >who best knows the latency of his/her network, capability of the box.
> 
> Given the definition of SHOULD from RFC 2119 (which is normatively 
> referenced in the drafts), a different wording might be:
> 
> NATs SHOULD handle IP fragments that are seen in order and out of order.
> When an in order fragment is seen, a NAT SHOULD store the L4
> information of the first fragment so that it can match the subsequent
> fragments that do not carry L4 headers. No buffering of the fragments is
> required in case of in order fragments. For out of order fragments, a
> NAT SHOULD store the fragment until the first fragment is seen. The
> fragments SHOULD be stored for an administrator-assigned period of time
> before discarding them. A NATs SHOULD allow the administrator to assign
> the number fragments to be stored per flow, and SHOULD allow the
> administrator to assign the number of total fragmented flows being held.
> 
> --Paul Hoffman, Director
> --VPN Consortium
> _______________________________________________
> Ietf-behave mailing list
> Ietf-behave@xxxxxxxxxxxxxxxxxxx
> https://list.sipfoundry.org/mailman/listinfo/ietf-behave
>

---

From: Pyda Srisuresh <[email-masked]>
Date: Sun, 27 Mar 2005 11:59:08 -0800 (PST)

--- Cullen Jennings <fluffy@xxxxxxxxx> wrote:
> 
> Now, have no problem saying that a device MAY store out of order fragments.

[suresh] I think it is a good idea to mention storing and reassemply as an
option for frgament processign, not just for out-of-order packets. The NAT
device must retain the assembly state pertaining to a fragmentation ID until
all fragments of the IP datagram are processed.

> However, I will be very against any text which implies that an application
> can depend on them doing this because clearly we can't depend on it in a
> wide variety of cases. 

[suresh] What are the cases where the apps cannot depend upon? Are Dos attacks
and memory considerations on the NAT box the scenarios you are concerned about.
Those are what I heard so far on the list. Are there others?

Heck, routers in the network have exactly the same issues with Dos attacks and
memory considerations. They cannot guarantee 100% IP forwarding.  The paradigm
of IP networking is one of "best effort packet forwarding", not the guaranteed
packet forwarding (as with, say, X.25). Since when has it become a requirement
that NATs have to provide guaraneed pkt forwarding? The MUST simpy implies that
the NAT devices MUST attempt to process fragments, not guarantee processing
100% of the time.

>                        Because of this, saying MUST does not help any
> developers and just means that less NATs will be behave compliant with no
> real gain. This is an all round loose in my mind.
> 
[suresh] I disagree. See my comment above.

> So let me sketch a counter proposal ....
> 
> We say that devices MUST forward fragments that arrive in order.
> 
> We say that device MAY forward packets out of order and describe how if they
> do this they may want to limit the number of stored packets per session and
> how long they store them.
> 

[suresh] I disagree with the proposal. This serves no useful purpose.

cheers,
suresh

> 
> Cullen (not as chair)
> 
> 
> PS. We need clear requirements that help residential style NATs. No
> administrator adjustment are likely to happen on these.
> 
> 
> On 3/24/05 1:48 PM, "Paul Hoffman" <paul.hoffman@xxxxxxxx> wrote:
> 
> > At 1:09 PM -0800 3/24/05, Dan Wing wrote:
> >> There are lots of ways a vendor could provide for protection while
> >> trying to avoid interfering with a valid flow, and the vendor might
> >> have cool techniques to do so.  How about instead of your last two
> >> sentences we just say:
> >> 
> >>     A NAT SHOULD allow out-of-order fragment behavior to
> >>     be adjusted by the administrator, such as the number,
> >>     duration, or source IP address of fragments kept by
> >>     the NAT or a combination of these parameters.
> > 
> > Sounds fine.
> > 
> >> We need to explain if it's important for the NAT to reassemble the
> >> packets before sending to the internal host; that's missing from
> >> your proposed text.
> > 
> > It's missing because I don't particularly believe it. To the internal
> > host, the NAT looks like the upstream router. Routers don't normally
> > reassemble packets, do they?
> > 
> >> What do we want to say about overlapping IP fragments?
> > 
> > If by "overlapping IP fragments" you mean multiple fragmented flows,
> > why should we say anything in particular about them.
> > 
> > --Paul Hoffman, Director
> > --VPN Consortium

---

From: Pyda Srisuresh <[email-masked]>
Date: Mon, 28 Mar 2005 09:49:17 -0800 (PST)

--- Dan Wing <dwing@xxxxxxxxx> wrote:
> 
> On Mar 27, 2005, at 10:38 AM, Pyda Srisuresh wrote:
> 
> > [suresh] Why do you believe, this is a SHOULD and not a MUST? When 
> > this is a
> > SHOULD, there is no guarantee that the behave compliant NATs would 
> > support
> > fragments. Apps that use fragments (including several VPN tunnels) 
> > cannot count
> > on the intermediate NATs to support fragmented IP datagrams. I donot
> > understand, why we would want to make this a SHOULD.
> 
> My suggested text only discusses the administrator's ability to adjust 
> the behavior. 

[suresh] Administrator's ability to tune the parameters for fragment processing
is different from making the fragment prcessing a requirement, IMHO. We can
talk about tuning after the requirement is mandated.
 
>               I don't see an interoperability requirement to require 
> the administrator to adjust the out-of-order behavior.
> 
[suresh] Not sure, I understand this. Are you saying that you agree that NATs
must be required to support out of order fragment processing, but that there
shoudl bo no interoperability requirement on adjustign the parameters of
fragment processing (ex: fragment reassembly). If so, I agree.

cheers,
suresh