Subject: [Ietf-behave] NAT TCP Simultaneous-Open: Request for comments on proposals.

From: Saikat Guha <[email-masked]>
Date: Wed, 24 May 2006 04:22:25 -0700 (PDT)

The discussion on- and off- list had identified a bunch of issues that I
quickly summarize below. I would like to get some feedback (particularly
on Option 4 from the vendors) on the four potential requirements below.

Issues (simplified, see original thread for details):
1. TCP Simultaneous-Open is necessary for P2P apps. NATs shouldn't force
   such attempts to be aborted. But there is currently no way to do so
   with ICMP/TCP messages. Barring future work (improbable),   
   facilitating S-O would require NATs ignore the error (perhaps
   momentarily)
2. The intent of prior RFCs is to signal errors (MUST or SHOULD
   depending) which would hinder P2P TCP, but would follow the spirit of
   RFCs.

So here are the options:

Option 1: Current proposal in the draft
========
  "A NAT MUST silently drop inbound SYNs"

Issues: Totally against the spirit of old RFCs, making network
troubleshooting harder, but allows TCP S-O

Option 2: Proposed earlier on the list by myself
=========
 "A NAT MUST NOT force an end-host to abort a TCP connection that could
  be a S-O. It SHOULD return an error if the error doesn't force the
  end abort the attempt. Otherwise, it MUST silently drop the SYN"

Issues: Hard to implement and understand. Currently no possible to
implement the first two clauses. Still facilitates S-O, but is forwards
compatible in the unlikely event that the first two clauses become
satisfiable.

Option 3: Following the REQ-8 language in the UDP spec
=========
 "If enabling P2P TCP apps is most important, a NAT MUST silently
  drop the SYN. If enabling quick diagnosis of network errors is most
  important, a NAT SHOULD signal an ICMP port unreachable. The behavior
  MAY be configurable by the administrator."

Issues: Let the NAT admin pick whether to allow TCP S-O or signal
errors.

Option 4: Proposed earlier on list by Joe
=========

 "A NAT SHOULD signal an ICMP port unreachable. If it does, it MUST
  delay the ICMP error by at least 3 seconds and MUST cancel the ICMP if
  in that time an outbound SYN for the connection is seen. If the NAT
  does not have resources to delay the ICMP, it MUST silently drop the
  inbound SYN"

Issues: Delaying the ICMP gives the internal side's SYN or
retransmission a chance to trigger a S-O. This requires bounded state at
the NAT. In an attack (when the buffer is full), it allows the NAT to
silently discard SYNs. Together, it facilitates S-O regardless of
whether the NAT is low on resources or not. When the NAT has resources
to spare, it facilitates error detection (delayed by 3s). When out of
resources it degrades gracefully to support S-O but not fast error
detection.

I'd particularly like to know how vendors and others feel about Option
4.

cheers,
-- 
Saikat
Attachment:
signature.asc

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 24 May 2006 16:52:30 -0700 (PDT)

I prefer option 1 and when some appropriate ICMP stuff is defined,  
change the recommendation to something like 2.

4 is very tricky to know what times would be appropriate.

On May 24, 2006, at 4:22 AM, Saikat Guha wrote:

The discussion on- and off- list had identified a bunch of issues  
that I
quickly summarize below. I would like to get some feedback  
(particularly
on Option 4 from the vendors) on the four potential requirements  
below.

Issues (simplified, see original thread for details):
1. TCP Simultaneous-Open is necessary for P2P apps. NATs shouldn't  
force
   such attempts to be aborted. But there is currently no way to do so
   with ICMP/TCP messages. Barring future work (improbable),
   facilitating S-O would require NATs ignore the error (perhaps
   momentarily)
2. The intent of prior RFCs is to signal errors (MUST or SHOULD
   depending) which would hinder P2P TCP, but would follow the  
spirit of
   RFCs.

So here are the options:

 "A NAT MUST NOT force an end-host to abort a TCP connection that  
could

Issues: Hard to implement and understand. Currently no possible to
implement the first two clauses. Still facilitates S-O, but is  
forwards
compatible in the unlikely event that the first two clauses become
satisfiable.

  important, a NAT SHOULD signal an ICMP port unreachable. The  
behavior

  delay the ICMP error by at least 3 seconds and MUST cancel the  
ICMP if

retransmission a chance to trigger a S-O. This requires bounded  
state at

---

From: Philip Matthews <[email-masked]>
Date: Thu, 25 May 2006 12:40:02 -0700 (PDT)

I feel it is important to make TCP Simultaneous-Open work.

Regarding option 4:
I feel that coordinating peers so they do things at the same time can  
be difficult.
If peers A and B are trying to set up a connection between them, but  
peer B
gets busy doing something else for a few seconds and misses the 3  
second window,
then the whole connection attempt fails. In our product (a peer-to- 
peer telephony
system), we see situations where the CPU on one hard-phone can get  
very overloaded
for a few seconds. I could easily see that this option would lead to  
situations where
the connection would be established most of the time, but would  
randomly fail
from time-to-time because the 3 second window was missed.

Given this concern, my current preference is for option 1.

Saikat:
Might I suggest producing a brief presentation describing these  
options and asking
for presentation time at both the BEHAVE and TCPM meetings in Montreal?

- Philip

On 24-May-06, at 19:52 , Cullen Jennings wrote:

   such attempts to be aborted. But there is currently no way to  
do so

So here are the options:

  does not have resources to delay the ICMP, it MUST silently drop  
the
  inbound SYN"

whether the NAT is low on resources or not. When the NAT has  
resources

I'd particularly like to know how vendors and others feel about  
Option
4.

cheers,
--
Saikat
_______________________________________________
Ietf-behave mailing list
Ietf-behave@xxxxxxxxxxxxxxxxxxx
https://list.sipfoundry.org/mailman/listinfo/ietf-behave

---

From: Joe Touch <[email-masked]>
Date: Fri, 26 May 2006 09:04:28 -0700 (PDT)

Philip Matthews wrote:
> I feel that coordinating peers so they do things at the same time can be
> difficult.
> If peers A and B are trying to set up a connection between them, but peer B
> gets busy doing something else for a few seconds and misses the 3 second
> window,
> then the whole connection attempt fails. In our product (a peer-to-peer
> telephony
> system), we see situations where the CPU on one hard-phone can get very
> overloaded
> the connection would be established most of the time, but would randomly
> fail

It'd be useful if option 1 is chosen to note that this is "intended to
relax TCP requirements to accommodate NATs". For a similar relaxation,
see the discussion of ICMP processing in RFC4301. Like for IPsec, this
MUST be configurable if chosen.

Note, though, that 4301 falls short of defaulting the configuration to
'drop all'.

Joe

Attachment:
signature.asc

---

From: Pyda Srisuresh <[email-masked]>
Date: Fri, 26 May 2006 13:37:48 -0700 (PDT)

I agree. I am in favor of option 1.

cheers,
suresh

---

From: Philip Matthews <[email-masked]>
Date: Sun, 28 May 2006 18:00:44 -0700 (PDT)

Just one more point about option 4:

My concern (expressed below) about option 4 revolves around the short  
time
window (3 seconds) that was suggested. If this window was instead,  
say, 60 seconds,
or perhaps even 30 seconds, I would not have this concern.

I don't know if there is any way to make the window larger without  
losing all the
debug advantage of sending the ICMP message in the first place, but  
if there was,
this might be a way forward.

- Philip

On 26-May-06, at 12:02 , Joe Touch wrote:

Philip Matthews wrote:

Regarding option 4:
I feel that coordinating peers so they do things at the same time  
can be
difficult.
If peers A and B are trying to set up a connection between them,  
but peer B
second
peer
very
randomly

---

From: Joe Touch <[email-masked]>
Date: Mon, 29 May 2006 21:00:31 -0700 (PDT)

Philip Matthews wrote:
> My concern (expressed below) about option 4 revolves around the short time
> window (3 seconds) that was suggested. If this window was instead, say,
> 60 seconds,

The window can be based on things like the TCP retransmission timer, but
needs to also take into account the MSL (120 seconds). It is reasonable
to respond to an error in a timely fashion; delays of 30 seconds end up
rendering the ICMP response essentially useless for a timely use when a
S-O isn't what's going on. I.e., I would expect a delay of around 6
seconds to be OK, but longer to be inappropriate. Unless the endpoints
are dawdling, that ought to be sufficient, though.

That assumes the endpoints have some reason to believe they want to
contact each other; if they don't, 6 seconds isn't enough. If that's the
case, I can't think of a timeout that would work, though.

> debug advantage of sending the ICMP message in the first place, but if
> there was,

I don't see one; the issue here is the baby and the bathwater. By
silently dropping the SYN, you're saying that NATs value supporting S-O
over being responsive to non-SO unsolicited SYNs.

Allowing S-O appears to require a NAT to go directly against a SHOULD in
RFC1122. It might provide enough condition (only in a NAT,
only since it is necessary to support S-O), but that'd be for the IESG
to decide.

Joe

Attachment:
signature.asc

---

From: Philip Matthews <[email-masked]>
Date: Tue, 30 May 2006 07:09:39 -0700 (PDT)

To be honest, I don't have any solid data that would allow me to make  
a definite conclusion
one way or another whether 6 seconds would be sufficient. It seems  
short to me,
but that is just a gut feeling and not based on hard evidence.

- Philip

On 29-May-06, at 23:58 , Joe Touch wrote:

Philip Matthews wrote:

My concern (expressed below) about option 4 revolves around the  
short time
window (3 seconds) that was suggested. If this window was instead,  
say,
60 seconds,
or perhaps even 30 seconds, I would not have this concern.
The window can be based on things like the TCP retransmission  
timer, but
needs to also take into account the MSL (120 seconds). It is  
reasonable
to respond to an error in a timely fashion; delays of 30 seconds  
end up
rendering the ICMP response essentially useless for a timely use  
when a
S-O isn't what's going on. I.e., I would expect a delay of around 6
seconds to be OK, but longer to be inappropriate. Unless the endpoints
are dawdling, that ought to be sufficient, though.

That assumes the endpoints have some reason to believe they want to
contact each other; if they don't, 6 seconds isn't enough. If  
that's the
case, I can't think of a timeout that would work, though.

I don't know if there is any way to make the window larger without
losing all the
debug advantage of sending the ICMP message in the first place,  
but if
there was,
this might be a way forward.
I don't see one; the issue here is the baby and the bathwater. By
silently dropping the SYN, you're saying that NATs value supporting  
S-O
over being responsive to non-SO unsolicited SYNs.

Allowing S-O appears to require a NAT to go directly against a  
SHOULD in
RFC1122. It might provide enough condition (only in a NAT,
only since it is necessary to support S-O), but that'd be for the IESG
to decide.

Joe

---

From: Saikat Guha <[email-masked]>
Date: Tue, 30 May 2006 07:27:53 -0700 (PDT)

> but that is just a gut feeling and not based on hard evidence.

While 6s is device/app/load-specific to an extent, before S-O can take
place the endpoints already need to coordinate (things like port and
address information), so they are more-or-less in sync (say to the order
of 100ms or so).

Devices that go on coffee breaks aside, the 6s provide a buffer to get a
single TCP packet out, and perhaps one re-transmission in the unlikely
case that the network up to the NAT is congested.

That is about a 6000% tolerance from the anticipated case, but indeed it
is device specific.

My 2c.
-- 
Saikat
Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 30 May 2006 13:05:54 -0700 (PDT)

I'm thinking of a SIP call flow, and I don't see how you'd get within 100ms
of synchronization.

For SIP, I believe you'd do something like this; in reality there would
probably be separate STUNT servers for both Alice and Bob, or they might use
UPnP, Bonjour, MIDCOM, or some other mechanism (which isn't relevant) to
learn their public IP address and TCP port.  And of course there will be
multiple SIP proxies, each adding their own processing latency and each
causing a likelyhood of a SIP packet drop and requiring a retransmission.

  Alice   STUNT server    SIP proxy   Bob
    |          |             |         |
 1  |--------->|             |         |
 2  |<---------|             |         |
 3  |----------------------->|         |
 4  |          |             |-------->|
 5  |X <-------------------------------|
 6  |          |<----------------------|
 7  |          |---------------------->|
 8  |          |             |<--------|
 9  |<-----------------------|         |
10  |--------------------------------->|
    
At message 4, Bob knows everything about Alice and can begin trying to set
up a TCP connection with Alice.  However, it could be "Quite Awhile" before
Alice receives message 9, which contains Bob's IP address and TCP port.

-d

> -----Original Message-----
> From: ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 
> Saikat Guha
> Sent: Tuesday, May 30, 2006 7:28 AM
> To: Philip Matthews
> Cc: Behave WG
> Subject: Re: [Ietf-behave] Re: NAT TCP Simultaneous-Open: 
> Request forcomments on proposals.
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 30 May 2006 13:18:16 -0700 (PDT)

Hi, Dan,

When you say, "quite awhile", could you tell us what timeframes you're 
thinking about?

I'm thinking of a SIP call flow, and I don't see how you'd get within 
100ms
of synchronization.

At message 4, Bob knows everything about Alice and can begin trying to set
up a TCP connection with Alice.  However, it could be "Quite Awhile" 
before
Alice receives message 9, which contains Bob's IP address and TCP port.

-d
Thanks,

Spencer

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 30 May 2006 14:07:02 -0700 (PDT)

> When you say, "quite awhile", could you tell us what 
> timeframes you're thinking about?

If you don't have PRACK (RFC3262) and you have loss of a single packet,
Alice (the caller) won't know Bob's IP address and TCP port number until Bob
"answers".  If we assume Bob is watching TV when Alice "calls", that means
that Bob has to drop the TV's remote control on the floor, spill his beer,
and run into the computer room and click "Accept Call".  I could see Bob
taking several seconds to run into the computer room.

-d

> > I'm thinking of a SIP call flow, and I don't see how you'd 
> > get within 100ms of synchronization.
> >
> >

---

From: Philip Matthews <[email-masked]>
Date: Tue, 30 May 2006 14:18:03 -0700 (PDT)

On 30-May-06, at 16:05 , Dan Wing wrote:

I'm thinking of a SIP call flow, and I don't see how you'd get  
within 100ms
of synchronization.

For SIP, I believe you'd do something like this; in reality there  
would
probably be separate STUNT servers for both Alice and Bob, or they  
might use
UPnP, Bonjour, MIDCOM, or some other mechanism (which isn't  
relevant) to
learn their public IP address and TCP port.  And of course there  
will be
multiple SIP proxies, each adding their own processing latency and  
each
causing a likelyhood of a SIP packet drop and requiring a  
retransmission.

  Alice   STUNT server    SIP proxy   Bob
    |          |             |         |
 1  |--------->|             |         |
 2  |<---------|             |         |
 3  |----------------------->|         |
 4  |          |             |-------->|
 5  |X <-------------------------------|
 6  |          |<----------------------|
 7  |          |---------------------->|
 8  |          |             |<--------|
 9  |<-----------------------|         |
10  |--------------------------------->|
At message 4, Bob knows everything about Alice and can begin trying  
to set
up a TCP connection with Alice.  However, it could be "Quite  
Awhile" before
Alice receives message 9, which contains Bob's IP address and TCP  
port.

I agree with Dan's comment that it  could be "Quite Awhile" before  
Alice receives
message 9. Specifically, I am thinking of the P2PSIP case where most  
or nearly all
peers are behind NATs, and so peers act both as SIP UAs and also as  
SIP proxies
for other phones. In this case, the signaling messages between A and  
B may have
to hop through quite a few intermediate peers (with all the  
consequent chances
of lost packets). If we are doing ICE-TCP, does generating an error  
response mean
that the particular candidate pair is deemed invalid and never retried??
[I haven't read the ICE-TCP spec yet...]

- Philip

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 30 May 2006 15:02:16 -0700 (PDT)

Saikat, Joe,

(I adjusted the CC list.)

Considering my flow again, I had a thought:

I notice that S-O only occurs after Alice talks to her STUNT server, and
after Bob talks to his STUNT server, and after they both use their same
(local) socket to attempt to communite with each other.

Can we maybe take advantage of this?

Detailed thought:

Alice's NAT sees Alice generate a TCP connection to the STUNT server
(message 1 and 2), and then close it down normally with TCP FINs (which
isn't shown, but it happens immediately after message 2).  Alice will then
start listening on that same port, but of course Alice's NAT doesn't know
Alice has started listening on that port.  Once message 9 is received by
Alice, Alice's NAT will finally see Alice trying to establish a TCP
connection with Bob (message 10).

In order to prevent Alice's NAT from sending an error at message 5, how
about a requirement like this, which combines your (1) with Joe's (4), and
makes the delay only occur if Simultaneous-Open could occur -- assuming
that, from a NAT's perspective, that TCP S-O can only occur after you talk
to a STUNT server!

Here is the suggested requirement:

  * To enable S-O, if a NAT has closed a TCP binding normally (FIN or RST),
and
    the NAT receives a SYN on its public interface to that TCP binding
before 
    MSL seconds have elapsed since the binding was closed, the NAT MUST wait

    at least 3 seconds before generating TCP RST or ICMP error in response 
    to that SYN.

-d

> From: Saikat Guha [mailto:saikat@xxxxxxxxxxxxxx] 
> Sent: Wednesday, May 24, 2006 4:22 AM
> To: Behave
> Cc: Joe Touch; Dan Wing; 'Spencer Dawkins'; 'Cullen 
> Jennings'; tcpm-chairs@xxxxxxxxxxxxxx; 
> behave-chairs@xxxxxxxxxxxxxx; 'Lars Eggert'
> Subject: NAT TCP Simultaneous-Open: Request for comments on proposals.
>

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 30 May 2006 16:29:56 -0700 (PDT)

> From: Philip Matthews [mailto:philip_matthews@xxxxxxxx] 
> Sent: Tuesday, May 30, 2006 2:20 PM
> To: Dan Wing
> Cc: 'Saikat Guha'; 'Behave WG'
> > within 100ms of synchronization.
> >
> > would probably be separate STUNT servers for both Alice and Bob,
> > or they might use UPnP, Bonjour, MIDCOM, or some other mechanism
> > (which isn't relevant) to learn their public IP address and TCP
> > port.  And of course there will be multiple SIP proxies, each
> > adding their own processing latency and each causing a likelyhood
> > of a SIP packet drop and requiring a retransmission.
> >
> >      1  |--------->|             |         | 2  |<---------|
> >|         | 3  |----------------------->|         | 4  |          |
> >     |-------->| 5  |X <-------------------------------| 6  |
> >|<----------------------| 7  |          |---------------------->| 8
> >|          |             |<--------| 9  |<-----------------------|
> >            | 10  |--------------------------------->|
> >
> > At message 4, Bob knows everything about Alice and can begin
> > trying to set up a TCP connection with Alice.  However, it could
> > be "Quite Awhile" before Alice receives message 9, which contains
> > Bob's IP address and TCP port.
> >
>
> Alice receives message 9. Specifically, I am thinking of the
> P2PSIP case where most or nearly all peers are behind NATs, and so
> peers act both as SIP UAs and also as SIP proxies for other
> phones.  In this case, the signaling messages between A and B may
> have to hop through quite a few intermediate peers (with all the
> consequent chances of lost packets).  If we are doing ICE-TCP,
> does generating an error response mean that the particular
> candidate pair is deemed invalid and never retried??  [I haven't
> read the ICE-TCP spec yet...]

It's still an I-D, but draft-ietf-mmusic-ice-tcp-00 does contain
these paragraphs on page 13.  As you suspected, the receipt of an
"error" (TCP RST or ICMP error) causes ICE to give up -- note the
paragraph below that I highlighted with "*":

   ...
   If the remote transport address is not active, the agent moves this
   transport address pair into the Valid state.  The TCP connection on
   which the Binding Request was received is then linked with the
   candidate pair.

*  If the STUN transaction produces an error, the state machine moves
*  into the Invalid state.

   If an agent receives a successful STUN Binding Response, and the
   native transport address is active, the agent moves this transport
   address pair into the Invalid state.  Furthermore, the agent MUST
   compute a peer-derived candidate as described in Section 9.  In
   addition, the TCP connection on which the Binding Request was
   received is then linked with the peer-derived candidate pair.   
   ...

If we want ICE-TCP to work with S-O, we'll have to either describe
how to (a) get NATs to avoid sending the error, or (b) describe how an
endpoint should be aggressive when doing S-O even if it's receiving
errors (TCP RST or ICMP errors).  I don't think it's a good idea to
do (b).

-d

---

From: Joe Touch <[email-masked]>
Date: Tue, 30 May 2006 17:22:31 -0700 (PDT)

>      Alice   STUNT server    SIP proxy   Bob

That works only if somebody reuses the TCP socket pair before TIME_WAIT
(depending on how the connection closes); that violates RFC793.

In specific, NATs ought (IMO) to hold the binding as unusable for a
TIME_WAIT interval after the FIN, since they're acting as an endpoint to
both ends of the TCP connection, effectively, and they're the only party
that knows the port/addr pairs in use. This ought to have been noted
somewhere, but I don't know if it ever was.

Joe

Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 30 May 2006 17:29:43 -0700 (PDT)

> That works only if somebody reuses the TCP socket pair before 
> TIME_WAIT (depending on how the connection closes); that violates 
> RFC793.

The alternative is to allocate a fresh socket, learn the public
TCP port across your NAT (using STUNT), signal that new TCP port 
number to your peer (in SIP, let's say), and try another TCP S-O 
all over again.  Or, instead, use SO_REUSEADDR.

> TIME_WAIT interval after the FIN, since they're acting as an 
> endpoint to both ends of the TCP connection, effectively, and 
> they're the only party that knows the port/addr pairs in use.  
> This ought to have been noted somewhere, but I don't know if it 
> ever was.

I doubt it was specified, either.  The IETF has tried to speak no 
NAT, hear no NAT, and see no NAT.  The BEHAVE-TCP document seems
like a good place to document such a requirement.

On a similar note, though -- if the host re-used that local socket 
on purpose (SO_REUSEADDR), the NAT should re-use that same local socket 
-- that is, take it out of TIME_WAIT.

-d

---

From: Joe Touch <[email-masked]>
Date: Tue, 30 May 2006 18:30:21 -0700 (PDT)

> all over again.  Or, instead, use SO_REUSEADDR.

SO_REUSEADDR allows you to reuse the local side of the socket at *bind*
time. The 4-tuple must still be unique. I.e., this allows you to bind to
a socket when it is in TIME_WAIT, but the new connection will still be
blocked.

SO_REUSEPORT overrides the 4-tuple, but it's for multicast apps (i.e.,
not TCP anyway).

> -- that is, take it out of TIME_WAIT.

Not as per above; only for UDP (which doesn't have TIME_WAIT anyway,
tbhough).

Joe

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Wed, 31 May 2006 02:31:13 -0700 (PDT)

>   Alice   STUNT server    SIP proxy   Bob

What you mentioned is true for the particular SIP callflow you provided,
but it doesn't necessarily have to be that callflow. Consider this
modified callflow.

  Alice   STUNT server    SIP proxy   Bob
    |          |             |         |
 1  |--------->|             |         |
 2  |<---------|             |         |
 3  |---------INVITE-------->|         |
 4  |          |             |-------->|
 5  |X <-------------------------------|
 6  |          |<----------------------|
 7  |          |---------------------->|
 8  |          |             |<--1xx---|
 9  |<-----------------------|         |
10  |---------UPDATE-------->|         |
11  |                        |-------->|
12  |                        |<--200 U-|
13  |<-----------------------|         |
14  |---------UPDATE-------->|         |
15 ...                       |-------->| 
16  |<================================>|

In the above callflow, Bob realizes he must perform STUNT after packet 5
as before. He responds to Allice with a 1xx Ringing or some such message
(Packet 8) that includes his public address.

Alice sends a "SIP ping" that comprises packets 10--13. The "ping" is a
SIP UPDATE packet that Bob responds to ASAP. Alice measures the SIP RTT
for this ping.

Alice then sends another UPDATE (packet 14) to Bob. Alice then waits
RTT/2 before starting TCP S-O. Bob will receive the packet roughly at
RTT/2 (note that while the SIP path is symmetric, there may be some IP
asymmetry on each hop). As soon as bob receives the second UPDATE, Bob
initiates his end of TCP S-O ASAP.

Essentially, Alice and Bob invoke TCP S-O roughly at the same time. This
is independent of the delay between packets 1 and 9 which you pointed
out could be quite large.

I am not saying this callflow has to be followed, but even with SIP and
proxies, it is _possible_ to get synced to within a few tens of
milliseconds. 

We were actually able to get it down to about .5--1 milliseconds on a
LAN without any IP asymmetry. This was much lower than a single IP RTT
between the two ends on the LAN, and obviously much much lower than the
time it takes between SIP packets 1 and 9 (4 RTTs-ish). On the internet,
this sync will depend some on the IP asymmetry, but 100ms is certainly
possible.

cheers,
-- 
Saikat
Attachment:
signature.asc

---

From: Philip Matthews <[email-masked]>
Date: Wed, 31 May 2006 05:40:30 -0700 (PDT)

Saikat:

Very interesting!

Would I be correct in saying that your modified call flow
increases the chances of a connection being established,
at the cost of a longer overall delay in setting up the
connection, especially if there are multiple
SIP proxies involved?
Any other comments on the pros and cons of the two call
flows?
- Philip

On 31-May-06, at 05:31 , Saikat Guha wrote:

What you mentioned is true for the particular SIP callflow you  
provided,

In the above callflow, Bob realizes he must perform STUNT after  
packet 5
as before. He responds to Allice with a 1xx Ringing or some such  
message
(Packet 8) that includes his public address.

Alice sends a "SIP ping" that comprises packets 10--13. The "ping"  
is a
SIP UPDATE packet that Bob responds to ASAP. Alice measures the SIP  
RTT
for this ping.

Essentially, Alice and Bob invoke TCP S-O roughly at the same time.  
This

I am not saying this callflow has to be followed, but even with SIP  
and
proxies, it is _possible_ to get synced to within a few tens of
milliseconds.

We were actually able to get it down to about .5--1 milliseconds on a
LAN without any IP asymmetry. This was much lower than a single IP RTT
between the two ends on the LAN, and obviously much much lower than  
the
time it takes between SIP packets 1 and 9 (4 RTTs-ish). On the  
internet,
this sync will depend some on the IP asymmetry, but 100ms is certainly
possible.

cheers,
--
Saikat

---

From: Saikat Guha <[email-masked]>
Date: Wed, 31 May 2006 05:50:44 -0700 (PDT)

> (depending on how the connection closes); that violates RFC793.

TIME_WAIT is for a 4-tuple IIRC. I believe what Dan is suggesting is
that once the internal endpoint X:x establishes a connection to an
external Y1:y1 (STUNT server), and is allocated the mapping X1:x1, and
then closes the connection, then when the remote peer Y2:y2 sends a SYN
to X1:x1, the NAT should delay the RST/ICMP for 3 seconds. In that 3
seconds, the internal endpoint X:x may be able to send the SYN to Y2:y2.
So the TIME_WAIT state for (X:x-Y1:y1) shouldn't affect the new TCP
between (X:x-Y2:y2) unless I am mistaken. The internal endpoint still
needs to use SO_REUSEADDR to get around with bind() limitations to
re-acquire X:x, but that is not to circumvent T-W.

If I understand Dan's proposal right, this is essentially the same as
Option 4 mentioned earlier on the list with the optimization that allows
the NAT to signal the error before the 3s is up in the case where the
internal endpoint hasn't demonstrated any interest in S-O (identified by
a prior connection to a STUNT server). This proposal IMHO would be a way
forward if we were to consider a blanket 3s/6s delay of the ICMP error
unacceptable.

-- 
Saikat
Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Wed, 31 May 2006 05:54:50 -0700 (PDT)

> connection, especially if there are multiple

I believe that is an accurate statement. 

Although, the exact magnitude of this extra delay can likely be tweaked
with alternate callflows or some timestamping at the SIP proxies; but
absent special proxy help, the callflow I mentioned is one end-host-only
mechanism to boost connectivity chances.

-- 
Saikat
Attachment:
signature.asc

---

From: Joe Touch <[email-masked]>
Date: Wed, 31 May 2006 08:18:47 -0700 (PDT)

> re-acquire X:x, but that is not to circumvent T-W.

The NAT ought to have two states for the two sides of the connection:

        X:x-Y1:y1       X1:x1-Y1:y1

The first is as seen by X; the other is as seen by Y1.

Once that connection is closed with a FIN, both 4-tuples (known as a
socket pair in RFC793) ought to be in TIME_WAIT, IMO.

Then when Y2:y2 sends a SYN to X1:x1, wait 3 seconds for the ICMP.

During those 3 seconds, X sends a SYN to Y2:y2 using X:x (which may
require SO_REUSEADDR, but as you observe it's not a TCP issue).

Now we have X:x talking to Y2:y2 - the translation entry X:x-Y1:y1/X1:x1
doesn't appear to help. How does the S-O connect?

> forward if we were to consider a blanket 3s/6s delay of the ICMP error

The reasons the delay were unacceptable were (AFAICT):

        1) too hard to deploy
                this version is a superset of option 4 in complexity

        2) 3/6 seconds is too short
                this version uses that number too

I'm unclear how this addresses those concerns; as noted above, I'm also
a bit unclear how it works ;-) Can you explain either?

Joe

Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 31 May 2006 09:15:59 -0700 (PDT)

I have two ideas floating around in my head.  I hope this email clarifies
them.

Let's call this first idea "TIME_WAIT SYN drop".  The important part is
highlighted with "*":

    In order for Alice to learn her public IP address and 
    TCP port, Alice needs to first talk to a STUNT server,
    which tells her that information.  She does that over TCP
    and then closes down that TCP connection.  Alice's NAT
    puts that TCP socket into TIME_WAIT, so that Alice's NAT
    doesn't purposefully re-use that source TCP port for 
    another TCP connection.

    (Note that for TCP S-O to work, Alice will need to re-
    use that same ephemeral port (SO_REUSEADDR) on her
    host, and on the NAT, when she does TCP S-O to Bob.  To
    accomplish this, Alice's NAT will need to meet 
    requirement REQ-1 of draft-ietf-behave-tcp-00,
    "External NAT mapping is endpoint independent".  Many
    NATs on the market already meet that requirement.)

    She communicates her public IP address and TCP port to Bob 
    somehow.  Bob will immediately send a TCP SYN to that IP 
    address and TCP port.

    Bob's TCP SYN will be received by Alice's NAT.  Alice's
    NAT has that TCP port in TIME_WAIT state.

  * I am proposing that a NAT has a TCP port in TIME_WAIT 
  * state, and that NAT is trying to assist with TCP S-O, 
  * and receives a TCP SYN to that port, the TCP SYN
  * should be silently dropped.

    For other incoming TCP SYNs, Alice's NAT would behave
    normally -- per RFC1122.  

    This is the idea I was describing on the BEHAVE
    mailing list.  This idea is similar to your option 1,
    but instead of dropping all incoming TCP SYNs, we're
    only dropping TCP SYNs that arrive at ports that are
    in TIME_WAIT state.

Let's call this second idea "delay receiving errors".  The important part is
also highlighted with "*":

    A separate, but somewhat similar idea, which I had
    emailed you directly, uses another tact.  This one is
    similar to option 4 in your original email, but 
    involves the NAT receiving the error, rather than
    involving the NAT sending the error.

    Alice learns and communicates her public IP address
    and TCP port to Bob.  Bob immediately sends a TCP SYN
    to that IP address and TCP port (which is Alice's NAT
    or Alice herself - it doesn't matter in this case).
    Alice will immediately generate an error (TCP RST or
    ICMP error).

    Bob's NAT will receive the error, and will convey
    that error to Bob.

  * If Bob sends another SYN with a matching 4-tuple, 
  * Bob's NAT will NAT the SYN normally, but if that
  * SYN also elicits an error, Bob's NAT will _delay_
  * conveying that error to Bob for ~5 seconds.  This
  * delay allows Bob (the host) to have an additional
  * 5 seconds to establish TCP S-O with Alice.  Bob's
    host was told about the error immediately (the
    first error was conveyed to Bob) and in the event
    that first error was lost (802.11 network, for
    example), Bob's host will receive the ~5 second 
    delayed errors.

    This is similar to Saikat's number 4 (Joe Touch's
    idea to delay ICMP errors), except that it involves
    Bob's NAT (which receives the error) rather than 
    Alice's NAT (which sends the error).

-d

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 31 May 2006 09:47:20 -0700 (PDT)

> In the above callflow, Bob realizes he must perform STUNT 
> after packet 5 as before. He responds to Allice with a 1xx 
> Ringing or some such message (Packet 8) that includes his 
> public address.

Yes, of course.  But unless both Alice and Bob implement
PRACK (RFC3262), or we assume a non-lossy transport of signaling
messages (such as TCP) that 1xx message (message 8 and 9) is 
sent unreliably and could be dropped between Bob and Alice.  That
is what I was referring to in my reply to Spencer.

> Alice sends a "SIP ping" that comprises packets 10--13. The 
> "ping" is a SIP UPDATE packet that Bob responds to ASAP. 

PRACK (RFC3262) uses a 2xx message for this purpose.  I believe
you're describing something else?

> Alice measures the SIP RTT for this ping.
> 

Ok, I admit it's possible.  But that's a lot of SIP signaling 
in order to try to synchronize those TCP SYNs.

-d

---

From: Saikat Guha <[email-masked]>
Date: Wed, 31 May 2006 10:10:02 -0700 (PDT)

On Wed, 2006-05-31 at 09:47 -0700, Dan Wing wrote:
> But that's a lot of SIP signaling 

That's true. Perhaps something like this can be a fallback if a more
optimistic version fails. One optimistic one could be where the
end-hosts are using NTP (most recent OSs do); Bob (or his proxy) could
stamp his packet 8 with its local clock. Alice could then estimate RTT/2
to within NTP+skew+RTT and skip packets 10--13. 6 seconds still leaves
quite a margin for NTP+skew+RTT IMHO.

In any case, SIP+TCP-SO does both rendezvous and connection setup; and
will likely be comparable in connection latency to (SIP+)ICE+STUN/TURN
etc. For some classes of of applications (background apps like
bittorrent, or app's that "ring" for a while like IP phones) connection
setup overhead may be acceptable. But I admit that the signaling can be
heavy-weight for certain other apps (peer-to-peer instant messages etc.)

-- 
Saikat
Attachment:
signature.asc

---

From: Joe Touch <[email-masked]>
Date: Wed, 31 May 2006 10:14:42 -0700 (PDT)

>     For other incoming TCP SYNs, Alice's NAT would behave

It is more important to respond to TIME_WAIT with a RST than in other
states. This also requires that the NAT keep state on closed connections
for the TIME_WAIT interval (which I would agree they should anyway).

That said, I'm unclear on how this successfully helps S-O; the TCP
socket pair (the 4-tuple) can't be reused during TIME_WAIT anyway.

>     Alice's NAT (which sends the error).

There are two parts to this:

1) move the delay to the NAT that receives the error

Delaying sending the error happens exactly once - wherever the error
would have been generated. Delayed receipt can happen multiple places
(consider multiple layers of NATs). How do you know you're the right NAT
to delay receipt?

2) pass the first error, but delay the second one

The first error will shut Bob's connection that's issuing the SYNs, so
subsequent SYNs won't happen.

What is the benefit for either component?

Joe

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Wed, 31 May 2006 10:32:29 -0700 (PDT)

> The NAT ought to have two states for the two sides of the connection:

It may be easier to think of NAT state in terms of "mapping" and
"connection"/"session".

#1:  For each internal port (X:x) each NAT maintains one and exactly one
"mapping" to the allocated external address (X1:x1). That is if the NAT
adheres to REQ-1 which mandates endpoint-independent behavior. (non
compliant NATs would include in the "mapping" the remote end's address
as well, but they are irrelevant) -- (1)

#2:  For each TCP connection, the NAT maintains the remote end's address
(Y1:y1), a pointer to the mapping for the local end, and connection
state (sequence numbers, flags seen etc.).

> socket pair in RFC793) ought to be in TIME_WAIT, IMO.

When the connection is closed with a FIN, the connection-state (#2) is
updated to reflect that fact.

> Then when Y2:y2 sends a SYN to X1:x1, wait 3 seconds for the ICMP.

An unsolicited SYN is one that doesn't match any entry in #2, and either
matches no entry in #1 or matches an entry in #1 but the endpoint
filtering policy of the NAT forbids it from accepting SYNs from Y2:y2.

Now say, it waits 3s for the ICMP. 

For S-O, the case likely is that the SYN matches a mapping in #1 but
nothing in #2 (X:x just finished a STUNT address discovery so a mapping
in #1 exists).

(Dan's proposal says that if the unsolicited SYN is a result of a
non-existent entry in #1 then there is no need to wait for those 3s)

> require SO_REUSEADDR, but as you observe it's not a TCP issue).

When the SYN goes out, the NAT finds an already existing mapping in #1.
It creates a new entry in #2 with (Y2:y2, pointer to the mapping,
SYN_SENT) etc. So at the end of all this, the NAT state looks like:

#1: Mapping table
 M1    X:x <=> X1:x1

#2: Connection Table
 C1:   Y1:y1, M1, TIME_WAIT
 C2:   Y2:y2, M1, SYN_SENT

> doesn't appear to help. How does the S-O connect?

  Say the 3s timeout for the ICMP is now over. The NAT checks table #2,
  finds a connection entry for that connection (C2) and kills the ICMP.

  Y2 hits a TCP retransmission and resends his SYN. The SYN matches
  entry C2 and forwarded to X:x. S-O can now connect.

>               this version is a superset of option 4 in complexity

Agreed. It requires additional state at the NAT to delay the ICMP, kill
it if a connection entry appears, or send it otherwise.

>               this version uses that number too

Agreed. 3/6s may or may not be too short for various devices, or lossy
wireless networks or a combination. 

> a bit unclear how it works ;-) Can you explain either?

Dan's suggestion does allow the 3/6s to be increased to, say, 30s while
not impacting apps that almost certainly are not doing S-O (i.e. didn't
query a STUNT server). 

(PS: Request -- Lets not go back into the mapping vs. session vs.
binding debate :-p)

-- 
Saikat
Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 31 May 2006 11:08:16 -0700 (PDT)

Joe Touch wrote:

> Dan Wing wrote:
> > I have two ideas floating around in my head.  I hope this 
> > email clarifies them.
> > 
> > Let's call this first idea "TIME_WAIT SYN drop".  The 
> > important part is highlighted with "*":
> > 
> states.

I understand the importance of sending RST in response to receiving data
packets when you're in TIME_WAIT.

But can you explain the importance of sending RST in response to receiving
a SYN when you're in TIME_WAIT?

> This also requires that the NAT keep state on closed 
> connections for the TIME_WAIT interval (which I would agree they 
> should anyway).

Right.

> socket pair (the 4-tuple) can't be reused during TIME_WAIT anyway.

It's my understanding that unless you get the SYNs to cross in the
network the *first time* you send a SYN, you need to reuse the same
local socket (same 4-tuple) for your next attempt to get SYNs to
cross in the network.  

(It's unfortunate we don't have a paper to cite which discusses
what we're trying to accomplish, as it would make this design-
by-email easier.)

> > Let's call this second idea "delay receiving errors".  The 
> > important part is also highlighted with "*":
> > 
> (consider multiple layers of NATs). How do you know you're 
> the right NAT to delay receipt?

I hadn't considered that and that's a good point.  I'm not sure it's
severe enough to kill the idea outright, though.  An advantage of
this technique is it's "my NAT" that implements this function so
I can do S-O with someone else, rather than relying on someone 
else to have an upgraded their NAT to be capable of S-O.

> subsequent SYNs won't happen.

Based on my understanding of how TCP S-O would work -- even without 
NATs at all -- Bob's TCP stack will get RSTs (or ICMP errors) until 
Alice and Bob's SYNs happen to cross in the network.  Thus Bob's 
application will have to loop, re-using the same 4-tuple, and 
cause Bob's TCP stack to send another SYN and try to get the SYNs
to cross in the network.

The benefit of delaying the second error is to allow Bob's 
application to only have to enter this loop maybe once:  the
first SYN will elicit an error (which Bob's application will see)
and Bob's application will then retry using the same 4-tuple but
now his stack (and application) won't see an error again for ~5
seconds.

Perhaps we can adjust your earlier idea to include this behavior,
so we avoid the double-NAT problem you found with my idea.  Your
original idea, per Saikat's email:

    >     "A NAT SHOULD signal an ICMP port unreachable.  If it
    >     does, it MUST delay the ICMP error by at least 3
    >     seconds and MUST cancel the ICMP if in that time an
    >     outbound SYN for the connection is seen.  If the NAT
    >     does not have resources to delay the ICMP, it MUST
    >     silently drop the inbound SYN"

Let me know if the following works for you.  With this new
text I'm having the NAT delay sending an error only if there
is a reason to suspect the remote endpoint is trying to 
establish a TCP S-O connection with us.

    Upon receipt of a TCP SYN to a TCP port that is in 
    TIME_WAIT, a NAT SHOULD signal an error (ICMP port 
    unreachable or TCP RST) immediately.  If the NAT receives 
    another TCP SYN with a matching 4-tuple of a previous TCP 
    SYN the NAT SHOULD delay sending an error in response
    to that TCP SYN for 3 seconds.

This does mean the NAT has to maintain information about the incoming SYN
for each of its public TCP ports that are in TIME_WAIT, in order to decide
what action to take when another SYN happens to arrive on that TCP port
(send an error immediately versus delay for 3 seconds).  Maintaining that
additional information isn't pleasant, especially considering it's coming
from the public side of the NAT (which is where attacks come from).

Elsewhere we would have a requirement that receiving any TCP traffic while a
TCP binding is in TIME_WAIT should elicit an error response, as that is
important for TCP.  However, that isn't important for our needs to do
simultaneous-open, so I didn't include it in the paragraph above.  For S-O
to work better, we only care about delaying (or dropping) error messages in
response to TCP SYN.

Saikat, would that help with TCP S-O, or does that make things worse?

-d

---

From: Joe Touch <[email-masked]>
Date: Wed, 31 May 2006 21:26:43 -0700 (PDT)

Dan Wing wrote:
...
> a SYN when you're in TIME_WAIT?

It's at the bottom of page 71 of RFC793. It allows the other side to
pick a different set of ports more quickly, since these - even retried -
won't succeed until TIME-WAIT expires. Normally, a lost SYN would result
in a retry, which just keeps the same ports.

I.e., an application that didn't get a RST for a TIME-WAIT will end up
waiting about a minute for TCP to timeout on the connection attempt overall.

...
> local socket (same 4-tuple) for your next attempt to get SYNs to

Yes, but if one side ends up in TIME-WAIT, you have to wait until it
expires to reuse the port pair.

...
> else to have an upgraded their NAT to be capable of S-O.

That makes S-O depend on which side starts first, which is a bit odd for
a P2P net or the concept of the "S" in S-O. ;-)

> cause Bob's TCP stack to send another SYN and try to get the SYNs

The first RST/ICMP will make Bob go into TIME-WAIT. REUSEADDR won't help
that.

>     to that TCP SYN for 3 seconds.

Besides the state (noted below, though I think a NAT ought to have that
anyway), there's the issue of the first immediate error. That seems to
kill the other side - as noted above. Or did I miss something?

Joe

Attachment:
signature.asc

---

From: Joe Touch <[email-masked]>
Date: Wed, 31 May 2006 22:03:33 -0700 (PDT)

Saikat Guha wrote:
> #1:  For each internal port (X:x) 

That's address X and port x, isn't it?

> each NAT maintains one and exactly one
> "mapping" to the allocated external address (X1:x1).

That's address X1 and port x1, isn't it?

> That is if the NAT

REQ-1 requires that the map (external address _and_ port) be independent
of the endpoint (Y1:y1). I hadn't realized you're talking about only a
subset of NATs.

> state (sequence numbers, flags seen etc.).

OK.

> updated to reflect that fact.

The T-W affects two views of the TCP connection:
        X:x--Y1:y1 (as seen from inside the NAT)
        X1:x1--Y1:y1 (as seen from outside the NAT)

> nothing in #2 (X:x just finished a STUNT address discovery so a mapping

I didn't see anything in the udp doc that required that the mapping in
#1 be static, only that two concurrent connections have the property
that they would use the same source addr/port remapping pair. That
loophole needs to be closed for this to work.

Second, S-O after an open isn't really a S-O anymore. That's now making
a requirement to the behavior of the endpoints to 'tickle' the state on
the NATs a certain way - a way that they're not currently designed to
implement.

IMO, it's simpler to just delay all SYN-based ICMPs for 3 seconds.

> non-existent entry in #1 then there is no need to wait for those 3s)

Yes; that's an optimization of what I proposed that adds a lot of state
at the NAT (albeit state it ought to keep) _and_ a 'protocol' that
requires sequences of connections to make a S-O work. That's along run
for a short slide ;-)

> SYN_SENT) etc. So at the end of all this, the NAT state looks like:

OK.

>   finds a connection entry for that connection (C2) and kills the ICMP.

Yeah - we agreed that the ICMPs be held for 3 seconds anyway, and if a
connection comes that overrides the ICMP then it should be dropped.
That's the same as I proposed.

>   entry C2 and forwarded to X:x. S-O can now connect.

Ah - yes, you're describing what we already assumed was the case.

I.e., the only difference here is whether you decide to send the ICMP
immediately because you don't have the state from a previous connection.
As noted above, that helps only in some cases, but then also requires
that the TCP use two connections to establish one - where other
out-of-band coordination could suffice.

It's fine as an optimization, but IMO it's overkill to require it.

> it if a connection entry appears, or send it otherwise.

The part you describe here is the same as option 4; the difference is
that option 4 doesn't bother with the optimization case.

> not impacting apps that almost certainly are not doing S-O (i.e. didn't

Adding 30 seconds is substantial - it ends up not helping much in cases
where ICMP is supposed to help.

The issue with the optimization is that it may help, but you're really
assuming two things:

a) that all S-Os are preceded by a STUNT query
        that shouldn't be strictly required if the apps have enough     
        info to coordinate some other way

b) that all previous connections by a host are prefaces to S-Os
        i.e., any active host will probably always have an entry in #1,
        which means the use of that table to decide whether or not to
        send the ICMP immediately will be moot anyway

> (PS: Request -- Lets not go back into the mapping vs. session vs.

I'm glad I have no idea what this means ;-)

Joe

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Thu, 1 Jun 2006 03:08:30 -0700 (PDT)

On Wed, 2006-05-31 at 21:26 -0700, Joe Touch wrote:
> >     unreachable or TCP RST) immediately.  

What happens if the connection to the STUNT server isn't closed by the
time the S-O is attempted? With current stacks you can bind two TCP
sockets to the same port and initiate a connection to two different
destinations at the same time last I tried.

> >     If the NAT receives 

Also, last I tried the Windows stack (XP SP1 I think) would send 3 SYNs
for an app-initiated connection even when each SYN was met with an RST.
So the app doesn't see the error until all three SYNs get their RSTs.
Your approach would consequently shave off 3s out of 9s in detecting an
error. On Linux, it may shave off 3s out of 3s in certain, which is
probably what you were aiming for.

Overall, for an optimization that, as Joe mentioned, affects only some
connections, assumes how the address must have been learned, adds a fair
bit of state and complexity, requires the app to follow a certain
"port-knocking" protocol (i.e try twice), I am not convinced it provides
proportional real-world benefit (i.e. to Windows apps and in the
multiple-NAT scenario that Joe identified).

Certainly not enough to make it a requirement IMHO. 

But it might make sense to tweak the wording of Option #4 to not
preclude such an optimization if the vendor were to make it.

My 2c.
-- 
Saikat
Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Thu, 1 Jun 2006 03:39:41 -0700 (PDT)

> of the endpoint (Y1:y1). I hadn't realized you're talking about only a

The subset I am talking about behave compliant NATs. There are legacy
NATs that don't follow REQ-1.

> loophole needs to be closed for this to work.

By static, I assume you mean that the internal address and port (X:x)
uniquely determines the mapped address and port (X1:x1) for all time.
That is not possible -- by definition that mapping has to be mane-to-one
(the NAT has more internal hosts than external IPs). 

Say two internal endpoints (X:x) and (Z:z) get the same "static" mapping
(X1:x1). Then the two can no longer talk to the same external endpoint
Y1:y1 simultaneously.

Whats the UDP draft does say is that the mapping must remain active for
some period of time after the last activity -- i.e. for a few minutes
(5-ish), all connections from X:x will use that mapping and all incoming
packets (for new connections, policy permitting) will be routed to the
internal endpoint.

> IMO, it's simpler to just delay all SYN-based ICMPs for 3 seconds.

/agreed. Makes the fewest assumptions on why the SYN came, has IMHO
acceptable error notification lag, and supports S-O. 

> As noted above, [Dan's proposal] helps only in some cases, but then also 
> requires

Now that I think about it, Dan's proposal blocks a very time-saving
optimization. A large fraction of NATs are "port preserving" i.e. if you
use internal port :x then the NAT too will use external port :x for the
mapping (if available). Also, home NATs typically have one IP address.
So an application can opportunistically predict the mapping it is going
to get without ever talking to a STUNT server!! (almost 90% of the NATs
do this already; see section 5 in our IMC paper)

With this optimization, an endpoint can signal the necessary address and
port information OOB without incurring one RTT to the public server at
connection initiation, and also lessens the load on the STUNT server.
For home NATs with few active connections through it, such port
prediction will almost always succeed.

-- 
Saikat
Attachment:
signature.asc

---

From: Joe Touch <[email-masked]>
Date: Thu, 1 Jun 2006 07:34:47 -0700 (PDT)

> NATs that don't follow REQ-1.

I've never heard of anything more than "there are four kinds of NATs".
Is there really reason to assert that violation of REQ-1 correlates to
"legacy" NATs only?

> packets (for new connections, policy permitting) will be routed to the

Is that also implemented in all currently manufactured NATs? What is the
incentive for the NAT to keep that state around at all once the
connection is closed?

...
> do this already; see section 5 in our IMC paper)

Regarding citing percentages... (which seems to be prevalent in the
IETF), in this case:

Section 5 measures your sample of home users; 90% of the _cases_ != 90%
of the NATs [deployed in use].

Even if you normalize for the market info earlier in your paper,
the study you cite is based on market penetration of home WLAN NATs,
which are only a fraction of where NATs are deployed (consider a single
enterprise, NATs built into cable or DSL boxes that don't include WLAN,
etc.)

(IMC reviewers should have caught that!)

I don't know how many NATs do anything in particular. I haven't heard
any convincing evidence that any researcher does (someone at a large
company that deploys software that calls-in from behind NATs could try
to figure this out, e.g., MS during a windows update, but I haven't
heard them talk about it).

> prediction will almost always succeed.

Sure. Opportunistic is one way.

Joe

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Thu, 1 Jun 2006 08:30:10 -0700 (PDT)

> I've never heard of anything more than "there are four kinds of NATs".
There are two independent axis: 
Axis 1: mapping = (independent, address dependent, address and port
dependent, and connection dependent (for TCP) )
Axis 2: filtering = (indep, addr dep, addr and port dep, conn dep)

you can mix and match -- 16 possible, some nonsensical. The oft-heard
"four kinds" are 
- indep. mapping, indep. filtering
- indep. mapping, addr dep filtering
- indep. mapping, addr and port dep filtering
- everything not indep. mapping

(note the "four kinds" doesn't cover all possible kinds. like indep.
mapping and conn dep. filtering for example; nor does it differentiate
'good' nats from 'bad' nats inside that 'everything not indep. mapping'
class. See the IMC'05 paper for details.)

> Is there really reason to assert that violation of REQ-1 correlates to

REQ-1 forbids the "everything not indep. mapping". By "legacy" I mean
NATs that don't adhere to REQ-1 here.

> incentive for the NAT to keep that state around at all once the

You cannot "close" a UDP connection at the NAT; it can only time-out. 

> of the NATs [deployed in use].

Correct, I should have clarified. The paper indeed makes it clear that
this is for home NATs and not enterprise NATs. That said, with the UDP
requirements REQ-3, REQ-4 (i.e. port-ranges SHOULD be preserved, and
port-parity SHOULD be preserved), the draft falls just short of
suggesting that preserving ports when possible is not a bad thing.

While app developers cannot depend on this (most of the related behavior
is a SHOULD and not a MUST anyway), predicting ports opportunistically
without contacting a STUNT server is certainly is one optimization that
is a) beneficial, b) likely to succeed for most home users today, and
possibly in the future.

IMHO, coming up with a SHOULD that makes S-O contingent on the app
contacting a STUNT server and thereby blocking a useful, potentially
opportunistic, optimization is not the right solution.

-- 
Saikat
Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 1 Jun 2006 08:59:10 -0700 (PDT)

> is a) beneficial, b) likely to succeed for most home users today, and

As individual contributor:

But port prediction only works well if the NAT is quiescent.  If
you add another user surfing the web, doing BitTorrent, or an 
active PVR-like automaton, port prediction becomes fragile and
you will have TCP connections from your intended peer coming 
into your other clients.  That's a mess unless you want to do
some ICE-like magic cookie handshaking at the beginning of your
TCP connection to verify you're talking to the correct 
application.

"Hey, everyone else in the house, please get your hands off your 
keyboards, I need to run my S-O application now and the NAT is 
too busy."

-d

---

From: Joe Touch <[email-masked]>
Date: Thu, 1 Jun 2006 09:26:46 -0700 (PDT)

> On Thu, 2006-06-01 at 07:33 -0700, Joe Touch wrote:
..
> You cannot "close" a UDP connection at the NAT; it can only time-out. 

The place where this draft refers to the UDP needs to make that
differentiation clear - at what point should NATs handle TCP connections
differently. Or are you saying that NATs should treat TCP the way they
treat UDP, with a 5-minute idle timeout?

i.e., IMO they ought to have a much longer timeout, in the half-hour to
hour range, or not timeout at all until state needs to be cleaned up for
space savings or because a TCP segment comes through that matches the
addr/port of the map but not the sequence numbers.

-- it's useful to keep in mind that - in general - TCP doesn't 'clean
up' state of old connections until they get in the way of new ones. IMO,
NATs ought to follow that same principle for TCP entries.

> is a) beneficial, b) likely to succeed for most home users today, and

Agreed.

> opportunistic, optimization is not the right solution.

Agreed.

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Thu, 1 Jun 2006 09:36:25 -0700 (PDT)

On Thu, 2006-06-01 at 08:59 -0700, Dan Wing wrote:
> But port prediction only works well if the NAT is quiescent.
Agreed.

>   If
> into your other clients. 

Not necessarily. The particular port prediction I was referring to in
this thread is different from the port prediction mentioned in the
paper. 

PP in this Thread: (#1)
- NAT has independent mapping. App binds to local port, knows the number
  of the external port.

PP in the Paper: (#2)
- To handle NON endpoint independent mapping nats. App binds to port,
  _finds_ external port by talking to STUNT. Rebinds to same local
  port, and predicts the new port the (broken) NAT will assign as 
  old port + 1 or something.

(#2) is fragile -- if two users initiate a bittorrent connection at the
same time, they can interfere with each other. A uses :x, talks to STUNT
server, finds out his external port is :x1'. Just then B uses :y, talks
to STUNT and gets port :x1'+1. A (incorrectly) predicts his port for the
next TCP to be :x1'+1

(#1) is not fragile. Say two users initiate a bittorrent connection at
the same time. A uses local port :x and B uses :y. Each independently
predicts his external port to :x and :y. No problems here. The only
problem is when box A and B uses the _same_ internal port at the _same_
time -- rather unlikely except for applications that bind outbound TCP
sockets to a hard-coded local port.

So the PP in type #1 is NOT affected by simultaneous
BitTorrent/web/PVR/whatever sessions.

cheers,
-- 
Saikat
Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Thu, 1 Jun 2006 09:47:16 -0700 (PDT)

> treat UDP, with a 5-minute idle timeout?

Timeouts for TCP are a bit more complicated. The draft mentions this in
some detail, but you may be working off of a rather old version.

UDP timeouts are simple. 5 minutes no traffic, kill connection. Reap
mapping once all sessions for that mapping are dead.

TCP timeouts are as follows.
- TCP pending: at least 4 minutes no traffic, kill connection (MAY)
- TCP established: at least 2 hours no traffic, kill connection (MAY)
- TCP shutdown: at least 4 minutes no traffic, kill connection (MAY)
- TCP RST/reset: up to vendor (explicitly UNSPECIFIED)

> NATs ought to follow that same principle for TCP entries.

That was suggested a long time ago. The draft allows for NATs to do just
that -- i.e. a NAT MAY or MAY NOT close the connection after it is idle
for a while; it may even send a TCP keepalive and await an RST to see if
the endpoint crashed. The draft still allows vendors to proactively
clean up state, perhaps in preparation for a burst of new connections,
if they wish -- one possible definition for "in the way". The draft just
sets the minimum values for how long a connection should be held open;
the upper bound is for the vendor to decide.

cheers,
-- 
Saikat
Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 6 Jun 2006 12:17:34 -0700 (PDT)

> > As noted above, [Dan's proposal] helps only in some cases, 
> > but then also requires
> optimization.  A large fraction of NATs are "port preserving"
> i.e. If you use internal port :x then the NAT too will use external
> Port :x for the mapping (if available).  Also, home NATs typically
> have one IP address.  So an application can opportunistically
> predict the mapping it is going to get without ever talking to a
> STUNT server!!  (Almost 90% of the NATs do this already; see section
> 5 in our IMC paper)

So, would you recommend that an application be coded expecting this
behavior?  I wouldn't -- it's too fragile and breaks if there are 
two active hosts behind that same NAT.  By "breaks", I mean that
you will have TCP connections going to the wrong host.  

Note that BEHAVE has a milestone to write a document describing how
applications should be coded to work over a NAT -- expecting some
kind of port predictability is one of the things we'll have to 
address in that document.  

> With this optimization, an endpoint can signal the necessary 
> address and port information OOB without incurring one RTT to the 
> public server at connection initiation, and also lessens the load 
> on the STUNT server.

Yep, if you're not using the NAT too much, I admit this works 
pretty well.  

But you can't tell if some other host is using the NAT a lot
and consuming those same TCP port numbers.  That's the rub.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 6 Jun 2006 12:21:33 -0700 (PDT)

> Sent: Thursday, June 01, 2006 9:36 AM
> To: Dan Wing
> Cc: 'Joe Touch'; 'Behave'
> Subject: Re: port prediction [was RE: NAT TCP 
> Simultaneous-Open: Requestfor comments on proposals.]
> 
> - NAT has independent mapping. App binds to local port, knows 
> the number of the external port.

He "knows" this without talking to a STUNT server?  If he's not talking to a
STUNT server, he's assuming a quiet NAT.  That is fragile.

> (#2) is fragile -- if two users initiate a bittorrent 
> connection at the same time, they can interfere with each other.
>  A uses :x, talks to STUNT
> to STUNT and gets port :x1'+1. A (incorrectly) predicts his 
> port for the next TCP to be :x1'+1
> 
> 
> 

I don't understand or agree with your argument that it's unlikely that two
applications will use the same ephemeral source port.  Ephemeral ports are
assigned by the TCP stack, and unlike ISNs they aren't assigned randomly but
rather the first available prot in that stacks ephemeral range is assigned.
Thus, if the same two applications are running on the same OS they will both
start using the same port source numbers.

-d

---

From: "Francois Audet" <[email-masked]>
Date: Tue, 6 Jun 2006 13:02:41 -0700 (PDT)

Also, NATs that do these types of tricks are also the ones who tend
to be unpredictable (e.g., ones that start off as Indpoint independent
and even plain broken, as the number of endpoits trying to use the
same port increases).

> [mailto:ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Dan Wing
> Sent: Tuesday, June 06, 2006 12:18
> To: 'Saikat Guha'; 'Joe Touch'
> Cc: 'Behave'
> Subject: [Ietf-behave] port preservation
> 
> 
> 
> > optimization.  A large fraction of NATs are "port 
> preserving" i.e. If 
> > you use internal port :x then the NAT too will use external Port :x 
> > for the mapping (if available).  Also, home NATs typically 
> have one IP 
> > address.  So an application can opportunistically predict 
> the mapping 
> > it is going to get without ever talking to a STUNT server!! 
>  (Almost 
> > 90% of the NATs do this already; see section 5 in our IMC paper)
> 
> So, would you recommend that an application be coded 
> expecting this behavior?  I wouldn't -- it's too fragile and 
> breaks if there are 
> two active hosts behind that same NAT.  By "breaks", I mean 
> that you will have TCP connections going to the wrong host.  
> 
> Note that BEHAVE has a milestone to write a document 
> describing how applications should be coded to work over a 
> NAT -- expecting some kind of port predictability is one of 
> the things we'll have to

---

From: Saikat Guha <[email-masked]>
Date: Wed, 7 Jun 2006 01:51:49 -0700 (PDT)

> you will have TCP connections going to the wrong host.  

Certainly an application shouldn't require it, but the likelihood of
having TCP go to the wrong host depends on the application.

Skype, for example, randomly picks a source port at time of installation
and always uses that source port. Therefore it would be very unlikely
for two Skype clients behind a NAT to collide. 

In any case, as with other NAT traversal mechanisms, there needs to be a
final verification of the data-path (after it is established) to ensure
that you have the right guy. ICE/STUN do this with a nonce that is sent
along the signaling-path and echoed back on-path.

> address in that document.  

Port-prediction is an optimization that can cause (fixable) problems if
the prediction is incorrect. The behave-app document should mention
this.

> and consuming those same TCP port numbers.  That's the rub.

True, you cannot tell until after the TCP is established and you verify
the nonces; by then you have already incurred a few RTTs and need to
restart the procedure sans port-prediction.

-- 
Saikat
Attachment:
signature.asc

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 7 Jun 2006 04:54:34 -0700 (PDT)

Re: Skype picking a random port number at installation time and using it for 
every connection, to reduce the chances of collisions behind a NAT...

- I wish we could provide meaningful BCP guidance to application developers 
about this, and

- I wish we had discussions about the Birthday problem about that guidance 
(the problem isn't that someone collides with YOU, 1/65K, it's that someone 
collides with anyone else also behind the NAT, which is a lot bigger 
problem).

Thanks,

Spencer

---

From: "Francois Audet" <[email-masked]>
Date: Wed, 7 Jun 2006 09:06:27 -0700 (PDT)

Are you talking about port prediction in the App, as opposed to 
port preservation in the NAT, or both?

> Sent: Wednesday, June 07, 2006 01:52
> To: Dan Wing
> Cc: 'Behave'; 'Joe Touch'
> Subject: [Ietf-behave] Re: port preservation
> 
> 
> preserving" i.e. 
> > > If you use internal port :x then the NAT too will use 
> external Port 
> > > :x for the mapping (if available).  Also, home NATs 
> typically have 
> > > one IP address.  So an application can opportunistically 
> predict the 
> > > mapping it is going to get without ever talking to a 
> STUNT server!!  
> > > (Almost 90% of the NATs do this already; see section 5 in our IMC 
> > > paper)
> > 
> > behavior?  I wouldn't -- it's too fragile and breaks if 
> there are two 
> > active hosts behind that same NAT.  By "breaks", I mean 
> that you will 
> > have TCP connections going to the wrong host.
> 
> Certainly an application shouldn't require it, but the 
> likelihood of having TCP go to the wrong host depends on the 
> application.
> 
> Skype, for example, randomly picks a source port at time of 
> installation and always uses that source port. Therefore it 
> would be very unlikely for two Skype clients behind a NAT to collide. 
> 
> In any case, as with other NAT traversal mechanisms, there 
> needs to be a final verification of the data-path (after it 
> is established) to ensure that you have the right guy. 
> ICE/STUN do this with a nonce that is sent along the 
> signaling-path and echoed back on-path.
> 
> 
> > applications should be coded to work over a NAT -- 
> expecting some kind 
> > of port predictability is one of the things we'll have to 
> address in 
> > that document.
> 
> Port-prediction is an optimization that can cause (fixable) 
> problems if the prediction is incorrect. The behave-app 
> document should mention this.
> 
> > But you can't tell if some other host is using the NAT a lot and 
> > consuming those same TCP port numbers.  That's the rub.
> 
> True, you cannot tell until after the TCP is established and 
> you verify the nonces; by then you have already incurred a 
> few RTTs and need to restart the procedure sans port-prediction.
> 
> -- 
> Saikat
>

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 7 Jun 2006 09:42:04 -0700 (PDT)

> would be very unlikely for two Skype clients behind a NAT to 

But with that implementation you have another problem -- once 
they do collide, they will continue to collide until you 
uninstall and re-install Skype on one of those boxes, replace 
your Skype hardphone, or something functionally identical to
those operations.

There are, of course, other applications running on other
hosts behind the NAT -- PVR applications, refigerators 
communicating with the local grocery store, web surfing,
and so on.  Any one of those might also choose that same
port as a normal function of its operation.

Large ISPs, such as a cable company in Germany that I know
about and, supposedly, some ISPs in Korea, don't give their
users real IP addresses but rather give their users private
IP addresses.  There is a high chance that ephemeral ports
in those situations will already be allocated to other
subscribers -- those NATs are _very_ busy.

> signaling-path and echoed back on-path.

Yep, I agree that some kind of nonce is necessary.

> problems if the prediction is incorrect. 

I can see how to detect you're talking to the wrong peer -- he
doesn't echo back the correct nonce or doesn't echo back a nonce
at all (timeout or runs a different protocol on the port you
happened to connect with).

> The behave-app document should mention this.
> 
> you verify

Ah, so that's what you mean by 'fixable'.  

-d

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 7 Jun 2006 09:42:35 -0700 (PDT)

> port preservation in the NAT, or both?

Saikat is assuming the NAT does port preservation.

-d

---

From: "Francois Audet" <[email-masked]>
Date: Wed, 7 Jun 2006 09:56:00 -0700 (PDT)

This is not a good assumption at all.

> From: Dan Wing [mailto:dwing@xxxxxxxxx] 
> Sent: Wednesday, June 07, 2006 09:42
> To: Audet, Francois [SC100:3055:EXCH]; 'Saikat Guha'
> Subject: RE: [Ietf-behave] Re: port preservation
> 
>

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 7 Jun 2006 10:08:43 -0700 (PDT)

> Re: Skype picking a random port number at installation time 
> and using it for every connection, to reduce the chances of 
> collisions behind a NAT...
> 
> - I wish we could provide meaningful BCP guidance to 
> application developers about this, and
> 
> - I wish we had discussions about the Birthday problem about 
> that guidance (the problem isn't that someone collides with YOU, 
> 1/65K, it's that someone collides with anyone else also behind 
> the NAT, which is a lot bigger problem).

I agree with both your points.  

The BEHAVE how-to-write-applications-for-BEHAVE-NATs document,
which has been one of our milestones for awhile, is the best
place for such guidance and discussion, I think.

We don't yet have a WG document for this milestone, although
draft-ford-behave-app-02.txt might be close.  

-d

---

From: Saikat Guha <[email-masked]>
Date: Wed, 7 Jun 2006 15:05:49 -0700 (PDT)

> This is not a good assumption at all.

Not a good assumption by any stretch of imagination, no. Nevertheless,
about 60% of the NATs afford this luxury. 

That said, the original discussion was of whether it is reasonable to
require that an application _MUST_ perform a STUNT lookup before S-O is
allowed to work. The argument here is that a STUNT lookup before being
able to invoke a TCP S-O should not be a MUST. Advisable or not,
exploiting port preservation opportunistically is one example why; there
may be others.

I agree with Dan that the behave-app document should point out the
potential pitfalls of this particular assumption. But more on topic of
the possible alternatives to REQ-4 in the TCP draft (about how to handle
unsolicited SYNs that may or may not be part of a S-O) ... I'll include
Dan's proposal with the other ones and flag it as an OPEN ISSUE. 

cheers,
-- 
Saikat
Attachment:
signature.asc

---

From: Joe Touch <[email-masked]>
Date: Wed, 7 Jun 2006 16:01:29 -0700 (PDT)

> exploiting port preservation opportunistically is one example why; there

My interpretation of this thread is that:

        an app MUST perform a STUNT before S-O reliably works

That doesn't prohibit alternatives that don't reliably work. It does say
that an app that needs S-O to work MUST do a STUNT to know that it will
succeed (again, if I follow the thread right).

Joe

Attachment:
signature.asc

---

From: Saikat Guha <[email-masked]>
Date: Wed, 7 Jun 2006 16:16:10 -0700 (PDT)

>       an app MUST perform a STUNT before S-O reliably works

Agreed. Along the same line, the proposal to send back an instant-RST if
a STUNT hasn't been performed would roughly be:

  unless an app performs STUNT, S-O will not be possible

(although you could say that the first failed S-O attempt could be made
to look like a STUNT to the NAT so the second attempt succeeds. That
would require mandating how the NAT treats the RST, which the proposal
doesn't cover yet.)

> That doesn't prohibit alternatives that don't reliably work. 

I could be mistaken here, but if I understand Dan's original proposal
correctly, it _would prohibit_ the unreliable port-preservation approach
mentioned earlier in this thread.

-- 
Saikat
(who is slowly getting confused with all the double negatives)
Attachment:
signature.asc

---

From: "Robert R. Gilman" <[email-masked]>
Date: Wed, 7 Jun 2006 16:22:28 -0700 (PDT)

I'd think that a practical port-preserving NAT would require a new
ICMP message, e.g. "port already in use", to inform the transport
layer to try some other source port.  Perhaps it could suggest an
available port; even so, it might take some time to find one common
to the host and the NAT before a packet went through the NAT.
How much code would have to be modified to do this? :-)
-Bob
----------------------------------------------------
Bob Gilman       rrg@xxxxxxxxx      +1 303 538 3868
Dan Wing wrote:
So, would you recommend that an application be coded expecting this
behavior?  I wouldn't -- it's too fragile and breaks if there are 
two active hosts behind that same NAT.  By "breaks", I mean that
you will have TCP connections going to the wrong host.  
Certainly an application shouldn't require it, but the likelihood of
having TCP go to the wrong host depends on the application.
Skype, for example, randomly picks a source port at time of 
installation and always uses that source port.  Therefore it 
would be very unlikely for two Skype clients behind a NAT to 
collide. 

-d

---

From: "Francois Audet" <[email-masked]>
Date: Wed, 7 Jun 2006 16:27:09 -0700 (PDT)

OK, I think I get it.

You want some optimization for cases when port preservation is done,
right

> Sent: Wednesday, June 07, 2006 15:06
> To: Audet, Francois [SC100:3055:EXCH]
> Cc: Dan Wing; Behave; Joe Touch
> Not a good assumption by any stretch of imagination, no. 
> Nevertheless, about 60% of the NATs afford this luxury. 
> 
> That said, the original discussion was of whether it is 
> reasonable to require that an application _MUST_ perform a 
> STUNT lookup before S-O is allowed to work. The argument here 
> is that a STUNT lookup before being able to invoke a TCP S-O 
> should not be a MUST. Advisable or not, exploiting port 
> preservation opportunistically is one example why; there may 
> be others.
> 
> I agree with Dan that the behave-app document should point 
> out the potential pitfalls of this particular assumption. But 
> more on topic of the possible alternatives to REQ-4 in the 
> TCP draft (about how to handle unsolicited SYNs that may or 
> may not be part of a S-O) ... I'll include Dan's proposal 
> with the other ones and flag it as an OPEN ISSUE. 
> 
> cheers,
> -- 
> Saikat
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 7 Jun 2006 16:27:59 -0700 (PDT)

Robert R. Gilman wrote:
> layer to try some other source port. 

IMO, that's an existing 'port unreachable' message. There is - and
should be - no reason why the port is unreachable indicated.

> Perhaps it could suggest an

I would be very wary of an ICMP solution that suggested alternate ways
around a port that was not available. That's too much of an opportunity
for attack, IMO.

Joe

>>> installation and always uses that source port.  Therefore it would be
>>> very unlikely for two Skype clients behind a NAT to collide. 
>>
>>
>> But with that implementation you have another problem -- once they do
>> collide, they will continue to collide until you uninstall and
>> re-install Skype on one of those boxes, replace your Skype hardphone,
>> or something functionally identical to
>> those operations.
>>
>> hosts behind the NAT -- PVR applications, refigerators communicating
>> with the local grocery store, web surfing,
>>> In any case, as with other NAT traversal mechanisms, there needs to
>>> be a final verification of the data-path (after it is established) to
>>> ensure that you have the right guy. ICE/STUN do this with a nonce
>>> that is sent along the signaling-path and echoed back on-path.
>>
>>
>>> Port-prediction is an optimization that can cause (fixable) problems
>>> if the prediction is incorrect. 
>>
>>

Attachment:
signature.asc

---

From: "Dave Hudson" <[email-masked]>
Date: Thu, 8 Jun 2006 02:48:08 -0700 (PDT)

UDP P2P apps are an interesting class of application that benefit from
port preservation (where possible).  If a NAT gets restarted the
preservation of port mappings means that external hosts are far less
likely to send packets that get dropped by the NAT (assuming a local
host re-establishes the original binding - quite likely if there aren't
many hosts behind the NAT).  In situations where there's a port clash as
long as the second host establishes a new random binding that maintains
that binding consistently then we end up with behaviour identical to a
non port-preserving NAT that randomly picked the same port number for
the first local client.

FWIW my experiences with building and testing SOHO NATs is that port
preservation generally gives a better end-user experience as more
applications tend to "just work" and not require either ALGs or users to
configure things such as port forwarding rules or DMZ host features that
effectively do nothing but guarantee port preservation, but do so with
considerable security problems.

Clearly for a NAT with a large number of hosts behind it these
advantages rapidly diminish because there's a much higher probability of
port collisions but for NATs supporting small numbers of hosts these
advantages shouldn't be dismissed (since at the worst we end up with a
NAT that maintains consistent mappings still).

I would definitely agree that applications shouldn't be coded to expect
this sort of behaviour but there are a lot of apps around that already
do make these sorts of assumptions (it's really dismaying how many games
expect well-known ports if you wish to host a game server or P2P apps if
you want them to work properly).

Regards,
Dave

* -----Original Message-----
* From: ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx 
* [mailto:ietf-behave-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Dan Wing
* Sent: 07 June 2006 17:42
* To: 'Saikat Guha'
* Cc: 'Behave'; 'Joe Touch'
* Subject: [Ietf-behave] RE: port preservation
* 
* > > So, would you recommend that an application be coded 
* expecting this 
* > > behavior?  I wouldn't -- it's too fragile and breaks if there are 
* > > two active hosts behind that same NAT.  By "breaks", I 
* mean that you 
* > > will have TCP connections going to the wrong host.
* > 
* > Certainly an application shouldn't require it, but the 
* likelihood of 
* > having TCP go to the wrong host depends on the application.
* > 
* > Skype, for example, randomly picks a source port at time of 
* > installation and always uses that source port.  Therefore 
* it would be 
* > very unlikely for two Skype clients behind a NAT to collide.
* 
* But with that implementation you have another problem -- once 
* they do collide, they will continue to collide until you 
* uninstall and re-install Skype on one of those boxes, replace 
* your Skype hardphone, or something functionally identical to 
* those operations.
* 
* There are, of course, other applications running on other 
* hosts behind the NAT -- PVR applications, refigerators 
* communicating with the local grocery store, web surfing, and 
* so on.  Any one of those might also choose that same port as 
* a normal function of its operation.
* 
* Large ISPs, such as a cable company in Germany that I know 
* about and, supposedly, some ISPs in Korea, don't give their 
* users real IP addresses but rather give their users private 
* IP addresses.  There is a high chance that ephemeral ports in 
* those situations will already be allocated to other 
* subscribers -- those NATs are _very_ busy.
* 
* > In any case, as with other NAT traversal mechanisms, there 
* needs to be 
* > a final verification of the data-path (after it is established) to 
* > ensure that you have the right guy.
* > ICE/STUN do this with a nonce that is sent along the signaling-path 
* > and echoed back on-path.
* 
* Yep, I agree that some kind of nonce is necessary.
* 
* > > Note that BEHAVE has a milestone to write a document 
* describing how 
* > > applications should be coded to work over a NAT -- expecting some 
* > > kind of port predictability is one of the things we'll have to 
* > > address in that document.
* > 
* > Port-prediction is an optimization that can cause (fixable) 
* problems 
* > if the prediction is incorrect.
* 
* I can see how to detect you're talking to the wrong peer -- 
* he doesn't echo back the correct nonce or doesn't echo back a 
* nonce at all (timeout or runs a different protocol on the 
* port you happened to connect with).
* 
* > The behave-app document should mention this.
* > 
* > > But you can't tell if some other host is using the NAT a lot and 
* > > consuming those same TCP port numbers.  That's the rub.
* > 
* > True, you cannot tell until after the TCP is established and you 
* > verify the nonces; by then you have already incurred a few RTTs and 
* > need to restart the procedure sans port-prediction.
* 
* Ah, so that's what you mean by 'fixable'.  
* 
* -d
* _______________________________________________
* Ietf-behave mailing list
* Ietf-behave@xxxxxxxxxxxxxxxxxxx
* https://list.sipfoundry.org/mailman/listinfo/ietf-behave
* 
*

---

From: "Robert R. Gilman" <[email-masked]>
Date: Thu, 8 Jun 2006 08:35:57 -0700 (PDT)

Joe-
The messages refer to different ports: the "port unreachable" is a
destination port; the "port already in use" is the 'source' port,
which is to be 'replicated' by the NAT.  The two conditions
require much different responses from the host, and I didn't see
an obvious way to get both meanings out of the existing "port
unreachable" message.
-Bob
----------------------------------------------------
Bob Gilman       rrg@xxxxxxxxx      +1 303 538 3868
Joe Touch wrote:
Robert R. Gilman wrote:

IMO, that's an existing 'port unreachable' message. There is - and
should be - no reason why the port is unreachable indicated.
[rrg]: That's why I suggested a new message.

I would be very wary of an ICMP solution that suggested alternate ways
around a port that was not available. That's too much of an opportunity
for attack, IMO.
[rrg]: I'm not sure I see the attack you do.  Are you thinking of a DoS
       attack in which the attacker returns ICMP "port unavailable"
       messages to cause the host to retry on another port, ad nauseam?
       It seems to me the "port unreachable" will have the same effect
       (with only one fake message required.)
Joe

-Bob

---

From: Joe Touch <[email-masked]>
Date: Thu, 8 Jun 2006 09:06:44 -0700 (PDT)

I'm not sure it matters that this is a source port issue. According to
RFC1122:

            3    (Port Unreachable), when the designated transport
                 protocol (e.g., UDP) is unable to demultiplex the
                 datagram but has no protocol mechanism to inform the
                 sender.

"Unable to demux" is what the NAT is experiencing; IMO, it is acting as
a host at this point (it's looking at the port numbers), so port
unreachable is the best error to provide.

Another option is reply with a TCP RST to that connection - which is
what would happen if this were a conventional host and such a collision
occurred.

If either response seems cryptic, then you could also send an error
indicating that  the destination service is unreachable given the
parameters of the packet, in which case you can use pointer-error (Code
12, Type 0), and point to the source port.

There's no other fundamental "reason" to deny a particular source port
so I wouldn't suggest a new ICMP error number for this.

Joe

Robert R. Gilman wrote:
>>>> Ah, so that's what you mean by 'fixable'. -d

Attachment:
signature.asc

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 8 Jun 2006 12:29:12 -0700 (PDT)

...
> advantages rapidly diminish because there's a much higher 
> probability of port collisions but for NATs supporting small 
> numbers of hosts these advantages shouldn't be dismissed 
> (since at the worst we end up with a NAT that maintains 
> consistent mappings still).

Something worse happens on a busy NAT.  The application that
'just works' (on a quiet NAT) now no longer works, and its
intended session is going to a different host or a different
application.  That different host or different application
might misbehave.  I mean, imagine you're doing iSCSI or NAT
and all of a sudden the wrong sessions are talking to each
other.  It really calls for nonces to be exchanged, but you're
still creating a larger risk of injecting invalid data into
an application that may not have been coded well enough to
expect such invalid data and discard it.

> I would definitely agree that applications shouldn't be coded 
> to expect this sort of behaviour but there are a lot of apps 
> around that already do make these sorts of assumptions (it's 
> really dismaying how many games expect well-known ports if 
> you wish to host a game server or P2P apps if you want 
> them to work properly).

I believe we're on the cusp of defining the difference between
"current practice" and "best current practice".

-d

---

From: "Dave Hudson" <[email-masked]>
Date: Thu, 8 Jun 2006 14:30:49 -0700 (PDT)

* -----Original Message-----
* From: Dan Wing [mailto:dwing@xxxxxxxxx] 
* Sent: 08 June 2006 20:29
* To: Dave Hudson; 'Saikat Guha'
* Cc: 'Behave'; 'Joe Touch'
* Subject: RE: [Ietf-behave] RE: port preservation
* 
* ...
* > Clearly for a NAT with a large number of hosts behind it these 
* > advantages rapidly diminish because there's a much higher 
* probability 
* > of port collisions but for NATs supporting small numbers of hosts 
* > these advantages shouldn't be dismissed (since at the worst 
* we end up 
* > with a NAT that maintains consistent mappings still).
* 
* Something worse happens on a busy NAT.  The application that 
* 'just works' (on a quiet NAT) now no longer works, and its 
* intended session is going to a different host or a different 
* application.  That different host or different application 
* might misbehave.  I mean, imagine you're doing iSCSI or NAT 
* and all of a sudden the wrong sessions are talking to each 
* other.  It really calls for nonces to be exchanged, but 
* you're still creating a larger risk of injecting invalid data 
* into an application that may not have been coded well enough 
* to expect such invalid data and discard it.

Hmm - this will be true for any NAT that doesn't maintain an active
binding though.  Any new session established after an old session
existed can randomly be given the NAT'd port number of some previous
session.  In fact I've seen exactly this behaviour very frequently when
NATs are restarted if they sequentially allocate NAT port numbers to new
sessions.  Similar behaviour occurs (albeit on average less frequently)
when random ports are allocated for new sessions.  In short for UDP if a
port binding ages out then chaos can ensue no matter what.  If we have a
really busy NAT we're going to have a very high probability of such
rebinding no matter what.

In the case of the P2P apps and though these tend to have been written
to keep bindings active (in fact I've observed many apps tend to be
quite chatty in this way).

* > I would definitely agree that applications shouldn't be coded to 
* > expect this sort of behaviour but there are a lot of apps 
* around that 
* > already do make these sorts of assumptions (it's really 
* dismaying how 
* > many games expect well-known ports if you wish to host a 
* game server 
* > or P2P apps if you want them to work properly).
* 
* I believe we're on the cusp of defining the difference 
* between "current practice" and "best current practice".

This is an interesting point of course (and one that seems to come up
quite frequently).  As a NAT implementor I find the primary concern of
customers and end users is that the NATs we deliver are able to give the
best end-user experience with their existing applications.  Now equally
I would dearly love to see more applications designed in a way that
means they can traverse NATs much more effectively - my ideal solution
in fact would be to design a NAT that recognizes when a NAT traversal
scheme is in use and drops down to some nice clean mode of operation
while making best efforts in all other circumstances.  This is much like
I may not like UPnP being enabled by default (it's insecure) as a means
of setting up NAT traversals but there's no question that end-users love
the fact that things "just work".

Regards,
Dave

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 8 Jun 2006 15:07:52 -0700 (PDT)

> * > probability 
> * > we end up 
> binding though. 

Yep.  An application that allows its NAT binding to expire is also
causing a problem for itself and for whatever other application (or
host) now has that binding.

> Any new session established after an old session
> port binding ages out then chaos can ensue no matter what.  
> If we have a really busy NAT we're going to have a very high 
> probability of such rebinding no matter what.
> 

Yep.  There isn't much else they can do.

> while making best efforts in all other circumstances. 
> This is much like I may not like UPnP being enabled by default 
> (it's insecure) as a means of setting up NAT traversals but 
> there's no question that end-users love the fact that things 
> "just work".

Your guidance and contributions to the BEHAVE 'application guidance' 
document would be quite valuable.  It is one of our milestones but
we haven't yet adopted a working group document.

-d

---

From: "Robert R. Gilman" <[email-masked]>
Date: Fri, 9 Jun 2006 08:44:44 -0700 (PDT)

Joe-
Thanks for your reply.  My comments are embedded below.  Regardless of
the details, I think the only way to make a port-preserving NAT work is
to pass information back to the originator that lets it retry when there
is a conflict, while not disturbing existing connections with unexpected
traffic.
-Bob
----------------------------------------------------
Bob Gilman       rrg@xxxxxxxxx      +1 303 538 3868
Joe Touch wrote:
I'm not sure it matters that this is a source port issue. According to
RFC1122:
            3    (Port Unreachable), when the designated transport
                 protocol (e.g., UDP) is unable to demultiplex the
                 datagram but has no protocol mechanism to inform the
                 sender.
"Unable to demux" is what the NAT is experiencing; IMO, it is acting as
a host at this point (it's looking at the port numbers), so port
unreachable is the best error to provide.
[rrg]: But that error is typically expected from the destination host,
       where it refers to the destination port.  The responses to the
       two possibilities would be quite different.  The error really
       should say "the NAT will be unable to demux any reply."  Should
       we define a new code to identify the particular problem?
Another option is reply with a TCP RST to that connection - which is
what would happen if this were a conventional host and such a collision
occurred.
[rrg]: That would work for TCP, but not UDP, and it presumes the NAT
       knows the protocol procedures beyond locating the addresses and
       ports.
If either response seems cryptic, then you could also send an error
indicating that  the destination service is unreachable given the
parameters of the packet, in which case you can use pointer-error (Code
12, Type 0), and point to the source port.
[rrg]: That would make sense, I suppose, if we can't think of any other
       flaw we might have in the source port.
There's no other fundamental "reason" to deny a particular source port
so I wouldn't suggest a new ICMP error number for this.
[rrg]: I agree that a new type is not needed, but I would suggest a new
       code to specifically identify the problem, so the sender knows
       how to react.
Joe

Robert R. Gilman wrote:

Joe-

Robert R. Gilman wrote:

-Bob

True, you cannot tell until after the TCP is established and you
verify

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 26 Jun 2006 14:29:54 -0700 (PDT)

On Jun 7, 2006, at 4:54 AM, Spencer Dawkins wrote:

- I wish we could provide meaningful BCP guidance to application  
developers about this, and
We do have a goal for the WG to produce a BCP with guidance like this  
so I hope this will happen.