Subject: [Ietf-behave] What to do if a NAT runs out of binding entries?

From: Philip Matthews <[email-masked]>
Date: Thu, 27 Apr 2006 09:32:32 -0700 (PDT)

The current discussion on the timeout value for a UDP binding connection
is closely related to a question I brought up during the last WG meeting
in Dallas:
     What should a NAT do if it receives a UDP packet (on the  
private side
     of the NAT) with a new source (address,port) combination and  
all its
     binding entries are in use?

As a P2P application designer, I would like NATs to behave in some
deterministic and helpful manner when faced with this situation.
My guess is that most NATs today, when faced with this situation,
pick some existing binding, discard it, and then used the freed entry
for the new packet.
Think for a minute how this behavior appears to the P2P application  
which
is sending keep-alives on all its  connections at the appropriate  
interval
(every 1 minutes, 2 minutes, or whatever we decide).
Its connections seem VERY UNRELIABLE.
The application can always send UDP packets, but incoming UDP packets
on that connection will get mysteriously dropped if the binding entry  
for
that connection was one that got re-used by the NAT for some other  
purpose.
However, the connection will be re-established (at least temporarily)
whenever the applications sends the next keep-alive.

My suggestion is that NATs should implement the following behavior:

    If a NAT receives a UDP packet (on the private side of the NAT)
    with a new source (address,port) combination and all its binding
    entries are in use, then the UDP packet is dropped and an ICMP
    reply sent back.
This would give deterministic behavior to applications. A P2P  
application
would know that, if it succeeds in establishing the connection, then the
connection will stay up if the necessary keep-alives are sent.

Any comments?
Especially from folks implementing NATs?
- Philip

- Philip

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 27 Apr 2006 15:24:36 -0700 (PDT)

Philip,

Would section 2.3 of draft-srisuresh-behave-nat-icmp-01.txt, or
text very similar to it, satisfy your need?  The authors, chairs,
and a few others have had a long offline discussion about that
ICMP message.

-d

> -----Original Message-----
> From: ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 
> Philip Matthews
> Sent: Thursday, April 27, 2006 9:33 AM
> To: Behave
> Subject: [Ietf-behave] What to do if a NAT runs out of 
> binding entries?
> 
> _______________________________________________
> Ietf-behave mailing list
> Ietf-behave@xxxxxxxxxxxxxxxxxxx
> https://list.sipfoundry.org/mailman/listinfo/ietf-behave

---

From: Philip Matthews <[email-masked]>
Date: Fri, 28 Apr 2006 05:23:18 -0700 (PDT)

Dan:

I think that the section you refer to is part of the answer,
but not the entire answer.
That section says that _IF_ the NAT decides to reject an outbound  
session,
then it should send an ICMP packet back to the host.

But consider a NAT that never rejects a new outbound session
but simply discards some older session and re-uses its binding entry.
Does section 2.3 of draft-srisuresh-behave-nat-icmp-01 apply,
since outbound packets are never rejected? By my reading,
the answer is NO.
I feel we need an additional requirement that talks explicitly about
the situation when all binding entries are in use and a packet arrives
that would create a new outbound session.
My suggestion for the behavior in this situation is to say:
    The new session is rejected. An ICMP reply is generated back
    to the host (as described in draft-srisuresh-behave-nat-icmp-01).
Under this suggestion, the only way a NAT terminates an outbound
UDP session is when it times out.
Note that I am open to other proposals for how to handle this situation,
as long as the proposal gives a NAT behavior that an application can
work with.
- Philip

On 27-Apr-06, at 18:24 , Dan Wing wrote:

Philip,

-----Original Message-----
The current discussion on the timeout value for a UDP binding
connection
is closely related to a question I brought up during the last
WG meeting
on that connection will get mysteriously dropped if the
binding entry

would know that, if it succeeds in establishing the
connection, then the

- Philip

---

From: [email-masked]
Date: Fri, 28 Apr 2006 10:16:14 -0700 (PDT)

I have the same concern as Philip does. I remember that I mentioned this 
issue more than a year ago in this WG, but at that time, my concern did 
not get a consensus.

I, as an application developer, still cannot answer to this question:

"How many nodes can a local UDP socket receive packets from 
simultaneously via a NAT?"

The ICMP error does not help because an application developer needs to 
know "how many" ahead of time. Let's say, you are designing an 
application that runs behind a NAT and monitors status of many remote 
nodes concurrently. How many nodes can you expect to be able to monitor?
32? 256? 1024? - We really don't know. Even with '32', nothing 
guarantees you that minimum 32 is supported by all NATs.

- Yutaka

Philip Matthews wrote:
Dan:

That section says that _IF_ the NAT decides to reject an outbound session,

Philip,

-----Original Message-----

- Philip

---

From: Philip Matthews <[email-masked]>
Date: Fri, 28 Apr 2006 11:35:02 -0700 (PDT)

Yutaka:

My original thought was the same as yours
-- specify some minimum number of bindings that a NAT needs to support.
But after thinking about this for a while, I realized the following:
a) A number that makes sense for a small home NAT would likely not  
make sense for a large enterprise NAT
b) Whatever the number, the local application has no idea how many  
other applications are creating bindings
    in the NAT.
c) There are cases where an application can be behind a succession of  
NATs (e.g., behind both a home NAT
   and a ISP's NAT). In this case, which NAT does one ask?

So I believe the best things that an application can do is to try to  
obtain as many NAT bindings as it needs.
It may get 50 bindings one time, and 30 bindings the next, and 100  
bindings the third time.

All I am trying to do by my proposal below is to give the application  
a consistent and deterministic experience
once it has obtained whatever bindings it can get.

- Philip

On 28-Apr-06, at 13:16 , yutaka_takeda@xxxxxxxxxxxxxxxxxxxx wrote:

I have the same concern as Philip does. I remember that I mentioned  
this issue more than a year ago in this WG, but at that time, my  
concern did not get a consensus.

The ICMP error does not help because an application developer needs  
to know "how many" ahead of time. Let's say, you are designing an  
application that runs behind a NAT and monitors status of many  
remote nodes concurrently. How many nodes can you expect to be able  
to monitor?
32? 256? 1024? - We really don't know. Even with '32', nothing  
guarantees you that minimum 32 is supported by all NATs.

- Yutaka

Philip Matthews wrote:
Dan:
the situation when all binding entries are in use and a packet  
arrives
Note that I am open to other proposals for how to handle this  
situation,

-----Original Message-----
pick some existing binding, discard it, and then used the freed  
entry
for the new packet.

Think for a minute how this behavior appears to the P2P application
which
is sending keep-alives on all its  connections at the appropriate
interval
(every 1 minutes, 2 minutes, or whatever we decide).
Its connections seem VERY UNRELIABLE.
The application can always send UDP packets, but incoming UDP  
packets
on that connection will get mysteriously dropped if the
binding entry
for
that connection was one that got re-used by the NAT for some other
purpose.
However, the connection will be re-established (at least  
temporarily)
whenever the applications sends the next keep-alive.

     If a NAT receives a UDP packet (on the private side of the  
NAT)
     with a new source (address,port) combination and all its  
binding
     entries are in use, then the UDP packet is dropped and an ICMP
     reply sent back.

- Philip

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 28 Apr 2006 12:08:12 -0700 (PDT)

> simultaneously via a NAT?"

Let me understand this question a bit better.  

Are you wanting to know how many UDP bindings the NAT could support, or do
you want to know if, on a _single_ UDP binding, the NAT will accept incoming
UDP packets from NNNN different source IP addresses on the Internet?  From
your wording, I think you're asking about the latter?

-d

---

From: [email-masked]
Date: Fri, 28 Apr 2006 14:26:03 -0700 (PDT)

Thanks Philip and Senthil for your comments. I understand your points 
and that all make sense to me.

To answer to Dan's question first, I am more concerned about the latter:

"I want to know if, on a _single_ UDP binding, the NAT will accept 
incoming UDP packets from NNNN different source transport addresses on 
the Internet."

In this case, if a NAT is NB=I and EF=I, this is not an issue. If a NAT 
is NB=I and EF=AD or EF=APD, this becomes a concern because a UDP 
binding in the NAT needs to remember the remote host addresses for 
external filtering. My concerns is, there may be a dumb NAT that 
statically limits the number of remote hosts from which a single UDP 
binding (external port) can receive packets, at a very small number like 
8 for example.

I agree that an application can not know ahead of time and tell exactly 
how much resources on a NAT is available for it, especially when the 
resources are shared by others. But without a minimum requirement, it is 
also hard for developers to say go ahead and do it.

a) A number that makes sense for a small home NAT would likely not make 
sense for a large enterprise NAT
Mostly, yes. What about the minimum number of remote hosts per external 
port a NAT needs to support?

b) Whatever the number, the local application has no idea how many other 
applications are creating bindings
    in the NAT.
I agree. I am not so concerned about the total number of bindings a NAT 
can create, assuming that it is likely plenty.. The number of remote 
nodes from a local UDP socket is my concern.

c) There are cases where an application can be behind a succession of 
NATs (e.g., behind both a home NAT
   and a ISP's NAT). In this case, which NAT does one ask?
I see your point. Regarding the maximum number of remote hosts from a 
local transport address, whichever has a smaller number is the critical 
point.

So I believe the best things that an application can do is to try to 
obtain as many NAT bindings as it needs.
It may get 50 bindings one time, and 30 bindings the next, and 100 
bindings the third time.
How can we reserve bindings for remote nodes of which address is unknown?

All I am trying to do by my proposal below is to give the application a 
consistent and deterministic experience
once it has obtained whatever bindings it can get.
I completely agree.

Thanks,
- Yutaka

- Philip

- Yutaka

Philip Matthews wrote:
Dan:

-----Original Message-----

- Philip

---

From: Saikat Guha <[email-masked]>
Date: Sun, 30 Apr 2006 16:29:53 -0700 (PDT)

On Fri, 2006-04-28 at 16:00 -0400, Philip Matthews wrote:
> Since you have done some NAT limit testing,
> I am wondering if you can tell me how some of
> the NATs you have tested behave in the situation
> described below?
>
> Begin forwarded message:
> > From: Philip Matthews <philip_matthews@xxxxxxxx>
> > Date: April 27, 2006 12:33:12  EDT (CA)
> >
> >
> >      private side of the NAT) with a new source (address,port) 
> >      combination and all its binding entries are in use?

My experience relates to TCP (as you noted Philip), so it may not apply
here, but for TCP atleast, the NATs that I could hit the connection cap
on, would silently drop the new TCP SYN packet sent after its connection
table was full. To clarify, it would not randomly pick an established
connection and shoot it, nor send back some TCP RST or ICMP packet to
coarse the sender to fail quickly. 

That said, only 2 (rather old) NATs out of 13 had a connection cap low
enough for me to observe this; for the others, I ran out of file
descriptors on the sender before I hit the cap.

-- 
Saikat
Attachment:
signature.asc

---

From: Philip Matthews <[email-masked]>
Date: Mon, 1 May 2006 05:38:58 -0700 (PDT)

Yutaka:

Please ignore my previous reply. I misunderstood your question.

I now believe that you are talking about the number of filtering  
entries _per_ binding entry,
while before I thought you were talking about the total number of  
either filtering entries or binding entries.

I had just assumed that there would be no explicit limit in this  
area; the limits would simply be
on the total number of filtering entries and the total number of  
binding entries.

Can you (or anyone else on the mailing list) think of a good reason  
to want a limit in this area?  Off hand, I cannot.

One think that I _could_ see a NAT vendor doing is limiting the  
number of binding and filtering entries
consumed by a single IP address on the private side of the NAT. This  
I could see being done in
a large NAT box (one uses for a large enterprise, or used to NAT an  
entire ISP's network).
This would prevent a single computer from using up a large fraction  
of the entries.
However, I cannot see a good reason to limit the number of filtering  
entries used by a single binding entry.

- Philip

On 28-Apr-06, at 17:25 , yutaka_takeda@xxxxxxxxxxxxxxxxxxxx wrote:

Thanks Philip and Senthil for your comments. I understand your  
points and that all make sense to me.

To answer to Dan's question first, I am more concerned about the  
latter:

"I want to know if, on a _single_ UDP binding, the NAT will accept  
incoming UDP packets from NNNN different source transport addresses  
on the Internet."

In this case, if a NAT is NB=I and EF=I, this is not an issue. If a  
NAT is NB=I and EF=AD or EF=APD, this becomes a concern because a  
UDP binding in the NAT needs to remember the remote host addresses  
for external filtering. My concerns is, there may be a dumb NAT  
that statically limits the number of remote hosts from which a  
single UDP binding (external port) can receive packets, at a very  
small number like 8 for example.

I agree that an application can not know ahead of time and tell  
exactly how much resources on a NAT is available for it, especially  
when the resources are shared by others. But without a minimum  
requirement, it is also hard for developers to say go ahead and do it.

a) A number that makes sense for a small home NAT would likely not  
make sense for a large enterprise NAT
Mostly, yes. What about the minimum number of remote hosts per  
external port a NAT needs to support?

b) Whatever the number, the local application has no idea how many  
other applications are creating bindings
    in the NAT.
I agree. I am not so concerned about the total number of bindings a  
NAT can create, assuming that it is likely plenty.. The number of  
remote nodes from a local UDP socket is my concern.

c) There are cases where an application can be behind a succession  
of NATs (e.g., behind both a home NAT
   and a ISP's NAT). In this case, which NAT does one ask?
I see your point. Regarding the maximum number of remote hosts from  
a local transport address, whichever has a smaller number is the  
critical point.

So I believe the best things that an application can do is to try  
to obtain as many NAT bindings as it needs.
It may get 50 bindings one time, and 30 bindings the next, and 100  
bindings the third time.
How can we reserve bindings for remote nodes of which address is  
unknown?

All I am trying to do by my proposal below is to give the  
application a consistent and deterministic experience
once it has obtained whatever bindings it can get.
I completely agree.

Thanks,
- Yutaka

- Philip
On 28-Apr-06, at 13:16 , yutaka_takeda@xxxxxxxxxxxxxxxxxxxx wrote:
I have the same concern as Philip does. I remember that I  
mentioned this issue more than a year ago in this WG, but at that  
time, my concern did not get a consensus.

I, as an application developer, still cannot answer to this  
question:

The ICMP error does not help because an application developer  
needs to know "how many" ahead of time. Let's say, you are  
designing an application that runs behind a NAT and monitors  
status of many remote nodes concurrently. How many nodes can you  
expect to be able to monitor?
32? 256? 1024? - We really don't know. Even with '32', nothing  
guarantees you that minimum 32 is supported by all NATs.

- Yutaka

Philip Matthews wrote:
Dan:
I think that the section you refer to is part of the answer,
but not the entire answer.
That section says that _IF_ the NAT decides to reject an  
outbound session,
then it should send an ICMP packet back to the host.
But consider a NAT that never rejects a new outbound session
but simply discards some older session and re-uses its binding  
entry.
Does section 2.3 of draft-srisuresh-behave-nat-icmp-01 apply,
since outbound packets are never rejected? By my reading,
the answer is NO.
I feel we need an additional requirement that talks explicitly  
about
the situation when all binding entries are in use and a packet  
arrives
that would create a new outbound session.
My suggestion for the behavior in this situation is to say:
    The new session is rejected. An ICMP reply is generated back
    to the host (as described in draft-srisuresh-behave-nat- 
icmp-01).
Under this suggestion, the only way a NAT terminates an outbound
UDP session is when it times out.
Note that I am open to other proposals for how to handle this  
situation,
as long as the proposal gives a NAT behavior that an application  
can
work with.
- Philip
On 27-Apr-06, at 18:24 , Dan Wing wrote:
Philip,

-----Original Message-----
      of the NAT) with a new source (address,port) combination  
and

As a P2P application designer, I would like NATs to behave in  
some
deterministic and helpful manner when faced with this situation.

My guess is that most NATs today, when faced with this situation,
pick some existing binding, discard it, and then used the  
freed entry
for the new packet.

Think for a minute how this behavior appears to the P2P  
application
which
is sending keep-alives on all its  connections at the appropriate
interval
(every 1 minutes, 2 minutes, or whatever we decide).
Its connections seem VERY UNRELIABLE.
The application can always send UDP packets, but incoming UDP  
packets
on that connection will get mysteriously dropped if the
binding entry
for
that connection was one that got re-used by the NAT for some  
other
purpose.
However, the connection will be re-established (at least  
temporarily)
whenever the applications sends the next keep-alive.

My suggestion is that NATs should implement the following  
behavior:

     If a NAT receives a UDP packet (on the private side of  
the NAT)
     with a new source (address,port) combination and all its  
binding
     entries are in use, then the UDP packet is dropped and an  
ICMP
     reply sent back.

- Philip

---

From: Philip Matthews <[email-masked]>
Date: Mon, 1 May 2006 05:58:38 -0700 (PDT)

On 30-Apr-06, at 19:29 , Saikat Guha wrote:

My experience relates to TCP (as you noted Philip), so it may not  
apply
here, but for TCP atleast, the NATs that I could hit the connection  
cap
on, would silently drop the new TCP SYN packet sent after its  

Saikat:

Thanks for the info!

Even though you have data for only 2 NATs,
this is certainly encouraging news for TCP.
- Philip

---

From: [email-masked]
Date: Mon, 1 May 2006 12:06:57 -0700 (PDT)

My comments in-line.

Philip Matthews wrote:
I now believe that you are talking about the number of filtering entries 
_per_ binding entry,
while before I thought you were talking about the total number of either 
filtering entries or binding entries.
My wording was not clear. Sorry for the confusion.

I had just assumed that there would be no explicit limit in this area; 
the limits would simply be
on the total number of filtering entries and the total number of binding 
entries.
I agree. I like the terms 'filtering entries' and 'binding entries'.

Can you (or anyone else on the mailing list) think of a good reason to 
want a limit in this area?  Off hand, I cannot.
Me either.

One think that I _could_ see a NAT vendor doing is limiting the number 
of binding and filtering entries
consumed by a single IP address on the private side of the NAT. This I 
could see being done in
a large NAT box (one uses for a large enterprise, or used to NAT an 
entire ISP's network).
This would prevent a single computer from using up a large fraction of 
the entries.
However, I cannot see a good reason to limit the number of filtering 
entries used by a single binding entry.
Problem here is that what a NAT implementor implements may not be all 
intensional. The assumption "there is no rational reason to do this, so 
they would not do this." has been wrong in my experience in NAT space. I 
 have seen many NATs that do weird things.

Having said that, I have not actually tested with NATs to know how many 
'filtering entries' a restrictive NAT can hold on to. I understand that 
people in this WG may be wondering if it is worth well addressing this 
issue because of a lack of evidence. But again, this is definitely a 
application developer's concern.

- Yutaka

- Philip

In this case, if a NAT is NB=I and EF=I, this is not an issue. If a 
NAT is NB=I and EF=AD or EF=APD, this becomes a concern because a UDP 
binding in the NAT needs to remember the remote host addresses for 
external filtering. My concerns is, there may be a dumb NAT that 
statically limits the number of remote hosts from which a single UDP 
binding (external port) can receive packets, at a very small number 
like 8 for example.

local transport address, whichever has a smaller number is the 
critical point.

Thanks,
- Yutaka

- Philip

- Yutaka

Philip Matthews wrote:
Dan:

-----Original Message-----

- Philip