Subject: [BEHAVE] TURN: Long lived allocation case

From: Benny Prijono <[email-masked]>
Date: Tue, 18 Dec 2007 03:58:44 +0000

All,

 From what I've heard from the discussion so far, it sounded like 
folks doesn't consider long lived allocation as the expected use 
case for TURN. Are there any specific reasons for this? Because I 
think long live allocation is a good idea, and that's what I plan to 
do with TURN client (on SIP+ICE scenario).

For SIP+ICE+TURN client use case, the scenario for using the ICE 
media transport would be: allocate the sockets, get STUN candidate, 
do Allocate request to get relayed candidate, and keep these ICE 
media transports alive until the application is shutdown. The same 
ICE media transport will be reused for subsequent calls.

Yes there are pros and cons with long lived allocations:

PROS:
  - it mimics what we do with the SIP transport (i.e. create socket,
    find the STUN mapped address, perform registration, and keep
    this binding open all the time), so we have some sort of
    experience with using this approach.
  - it simplifies call's state machine, as the media transport will
    always be available when it's needed. No need to wait for
    media transport asynchronously, as would happen if we have to
    create the media transport before the call.
  - it speeds up call's dial delay (no need to wait for STUN and
    TURN resolutions).

CONS:
  - client need to know how many ICE transports to pre-allocate,
    but this is application and not protocol issue.
  - if clients preallocate TURN transports, and if clients support
    multiple calls in the application, this will create significant-
    ly more TURN allocations in the TURN servers. This problem could
    be mitigated with allowing BANDWIDTH=0 in TURN.
  - need to do keep alive, which raises some concerns with power
    consumption. But we need to keep-alive the SIP socket anyway,
    so this problem is not specific to STUN/TURN.
  - need to handle address change scenario (i.e. NAT is restarted)
  - there is a window of failure when NAT is restarted between
    the keep-alive interval, and a call occurs at this time. In
    this case, the client will (wrongly) advertise the old mapping
    in the SDP. But again, this problem is not specific to TURN,
    as it could happen with SIP as well.

So because of the benefits above, I think long lived allocation is a 
good idea, and some folks will use this approach anyway, so TURN 
should be made to support this use case.

This use case relates to few issues being discussed:
  - BANDWIDTH. With clients pre-allocate TURN candidates, server
    will see more bandwidth being requested even though most clients
    are idle. If we allow BANDWIDTH=0 then client can be made more
    cooperative, by requesting BANDWIDTH=0 when the TURN transport
    is idle.
  - Channel number space. Since client reuses the same TURN
    allocation for subsequent calls, it will require slightly more
    channels, as previous channels (allocated for the previous calls)
    don't get released immediately after call is released. This is
    not a major problem for this use case, and 32767 is more than
    sufficient, but perhaps we could make the situation better
    by creating ChannelUnBind mechanism to immediately release
    the channel number?

Comments?

-- 
Benny Prijono
http://www.pjsip.org

_______________________________________________
Behave mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/behave

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Tue, 18 Dec 2007 12:12:01 +0200

> do with TURN client (on SIP+ICE scenario).

There is long-lived and very long-lived.
With the huge number of channels that there is, you can always reallocate a 
new TURN allocation when you run out of channel numbers, if ever. I think 
most SIP UA will eventually be restarted before this happens, but anyway.

I expect the cost of handling running out of channel be no worse than that of 
handling channel reallocation (which is intricate due to UDP race condition 
issues).

>     experience with using this approach.

Broken. You'd need SIP outbound to handle this correctly.

>     create the media transport before the call.

Won't work.

>   - it speeds up call's dial delay (no need to wait for STUN and

Won't work. You could "cache" the TURN allocation, but caching the reflexive 
address STUN harms reliability. You would start betting against so-called 
second and third order behavior of NATs. Sorry, but quite many NATs do look 
like they are nice over a short time interval, but are turn into nasty 
symmetric ones over extended period of time.

(...)
> Comments?

Since I think your use case is a bad idea from a technical perspective, I have 
not much sympathy for dealing with it inside TURN.

-- 
Rémi Denis-Courmont

---

From: Philip Matthews <[email-masked]>
Date: Tue, 18 Dec 2007 11:35:29 -0500

Replies to both Benny's original message and Remi's response.

>    so this problem is not specific to STUN/TURN.

In some applications, for example peer protocol connections in  
P2PSIP, a node would likely always have at least one connection  
active at any time so this is not a drawback.

> symmetric ones over extended period of time.

I not seeing what the problem is here. If the client learns its  
server-reflexive address from the TURN Allocate request, then it can  
use a single keep-alive to keep both the server-reflexive and the  
relay transport addresses alive. If the client advertises them both  
in the ICE Offer/Answer exchange, then won't ICE handle whatever  
behavior the NAT is currently exhibiting?

- Philip

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 18 Dec 2007 18:56:18 -0500

Some comments inline:

Benny Prijono wrote:
> All,
> 
>  From what I've heard from the discussion so far, it sounded like folks 
> doesn't consider long lived allocation as the expected use case for 
> TURN. Are there any specific reasons for this? Because I think long live 
> allocation is a good idea, and that's what I plan to do with TURN client 
> (on SIP+ICE scenario).

I think the point is not that a client will be forbidden from this, but 
that we will not provide a channel deallocation feature. I don't think 
its needed at all for the use case you are describing.

> 
> For SIP+ICE+TURN client use case, the scenario for using the ICE media 
> transport would be: allocate the sockets, get STUN candidate, do 
> Allocate request to get relayed candidate, and keep these ICE media 
> transports alive until the application is shutdown. The same ICE media 
> transport will be reused for subsequent calls.
> 

I'm not sure what it means to 'have some experience'. What is the 
technical or operational benefit?

>    create the media transport before the call.

Whether this is simpler or not, YMMV - since it trades off some things 
(no need to grab turn allocations ahead of time) for others (complicates 
interaction with turn server).

>  - it speeds up call's dial delay (no need to wait for STUN and

Well, you'd probably want to redo STUN to make sure the server reflexive 
candidate hasn't changed. Since you can do server reflexive and relayed 
together anyway, if you do the server reflexive at call setup time, then 
you gain no improvement in call setup delay by pre-allocating TURN.

>    be mitigated with allowing BANDWIDTH=0 in TURN.

How does that help? All it means is now the client needs to signal to 
its TURN server, before call setup, that it is changing bandwidth from 
zero to non-zero. In that case, you have not saved any messaging to the 
TURN server, but you have complicated capacity management in the turn 
server, and added a lot of keepalives.

>    so this problem is not specific to STUN/TURN.

No, but we have effectively doubled the amount of keepalive traffic. 
Also the number of ports that are available on the NAT get halved or 
worse; each client now needs, permanently, 2 (and as many as 5 - 
RTP/RTCP for audio and RTP/RTCP for video and one for signaling). This 
puts a serious dent in the number of clients that can be supported 
behind a NAT with a single IP. That is a major issue.

>    as it could happen with SIP as well.

No, but this window is increased with the long-lived allocations.

In my mind the drawbacks far outweigh the benefits.

That said, a client can go ahead and do all of this if it wants. I just 
don't want to add support for a channel re-bind. Instead, the client can 
just grab a new allocation when it runs out of channels. Anyway it'll 
likely reboot before then. Lets take an example. A client makes 10 calls 
a day (which is a lot). Each of these fork to two endpoints. This means 
20 channels a day get used. With 32767 channels, thats 1638 days or 4.4 
YEARS before running out of channels. The odds of a reboot of a client 
over 4.4 years are pretty much 100%.

-Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   499 Thornall St.
Cisco Fellow                                   Edison, NJ 08837
Cisco, Voice Technology Group
[email-masked]
http://www.jdrosen.net                         PHONE: (408) 902-3084
http://www.cisco.com

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Wed, 19 Dec 2007 09:33:01 +0200

Le Tuesday 18 December 2007 18:35:29 ext Philip Matthews, vous avez écrit :
> relay transport addresses alive.

That only works for a "compliant" NAT. The probability that a NAT behave like 
a compliant one decreases over the lifetime of a binding.
That's the whole point for using ICE instead of behavior discovery.

Now, you may be assuming that all NATs are BEHAVE-compliant, but then why 
would you need TURN at all anyway?

> If the client advertises them both in the ICE Offer/Answer exchange,
> then won't ICE handle whatever behavior the NAT is currently exhibiting?

Yes, but that will increase the chance that the reflexive address candidate 
fails, and that the TURN server has to be used.

-- 
Rémi Denis-Courmont

---

From: Philip Matthews <[email-masked]>
Date: Wed, 19 Dec 2007 09:29:42 -0500

On 18-Dec-07, at 18:56 , Jonathan Rosenberg wrote:
> Well, you'd probably want to redo STUN to make sure the server  
> reflexive candidate hasn't changed. Since you can do server  
> reflexive and relayed together anyway, if you do the server  
> reflexive at call setup time, then you gain no improvement in call  
> setup delay by pre-allocating TURN.

I want to understand why you think we need to get the server- 
reflexive address at call time, rather than at the time we allocate  
the TURN relay address?

As I see it, the TURN server returns a mapped address in the Allocate  
Response, and in addition stores this mapped address as part of the 5- 
tuple associated with the allocation. At every Refresh transaction,  
the 5-tuple must be the same, since the 5-tuple is used to identify  
the allocation. Thus the receipt of the successful Refresh response  
indicates to the client that its server-reflexive address has not  
changed.

Furthermore, with a long-term allocation, the client can be pretty  
certain that it is refreshing fast enough to keep the NAT bindings  
alive, otherwise the Refresh transaction would fail. By contrast, it  
doesn't seem to necessarily know this if it just allocates a call  
setup time.

Am I missing something?

- Philip

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Wed, 19 Dec 2007 16:47:08 +0200

> doesn't seem to necessarily know this if it just allocates a call

The client can be pretty sure that it is refreshing the binding TOWARD THE 
TURN SERVER. Quite many NAT implementations look like well-behaved restricted 
port Dr.Jekyll NATs over a short period of time, and transform into monsters 
Mr.Hyde symmetric NATs if you use the binding for a long period of time.

In other word, if you have a binding with your TURN server, the chance that 
you will get the same external NAT port (à la BEHAVE) is very high within the 
few seconds after the binding got created. Then it decreases over time, 
because the NAT is not really BEHAVE compliant and start messing up things as 
it allocates new bindings (for other hosts behind the same NAT or for other 
applications on the same host), or it has some time-dependent port 
allocation.

So then, if you start sending data to anything except the TURN server, you 
will get a different port mapping, and voilà, your server-reflexive candidate 
won't work.

-- 
Rémi Denis-Courmont

---

From: Benny Prijono <[email-masked]>
Date: Thu, 20 Dec 2007 06:21:44 +0000

Jonathan Rosenberg wrote:
>> folks doesn't consider long lived allocation as the expected use case 
>> for TURN. Are there any specific reasons for this? Because I think 
>> long live allocation is a good idea, and that's what I plan to do with 
>> TURN client (on SIP+ICE scenario).
> 

I get your point, and indeed there's nothing needs to be changed for 
this use case to work. But the more basic question is, is this a 
good use case for TURN (for the benefits and drawbacks that we're 
discussing)? That's what I'm trying to find out.

> technical or operational benefit?

What I meant was we (implementers) have some sort of experience with 
using long-lived socket approach, with the SIP socket.

> you gain no improvement in call setup delay by pre-allocating TURN.

Both the STUN and TURN candidates are kept alive with periodic 
refresh, thus no need to find out the srflx candidate before the call.

> server, and added a lot of keepalives.

The benefit is client does not need to wait for the Refresh (with 
non-zero bandwidth) response before making the call. If the Refresh 
fails, client can either cancel the call or do nothing as ICE will 
detect the failure. This works under the optimistic assumption that 
Refresh most likely will succeed of course.

But true, capacity is major issue with this approach, as one TURN 
server now can only handle a maximum of 65535 online users, compared 
to 65535 active calls. And assuming that only 10% of the calls need 
to use TURN and client drops the allocation once ICE negotiation 
completes and TURN is not selected, the capacity difference will be 
much greater.

(the 65535 number is theoretical number based on available ports. 
The actual number will be less of course).

> No, but we have effectively doubled the amount of keepalive traffic. 

Yes, although I don't think bandwidth is an issue here. Not sure 
what it does to power consumption though.

> behind a NAT with a single IP. That is a major issue.

I guess it depends on the deployment use case. For deployments where 
the users are home users, I don't think this is a major issue.

> In my mind the drawbacks far outweigh the benefits.

I now tend to think the same. Thanks for bearing with me.

> over 4.4 years are pretty much 100%.

Wait a second, so does this mean a client cannot reuse a channel 
number, even after it is expired (for example when the client 
intentionally does not send any data to refresh the permission).

thanks,
  -benny

-- 
Benny Prijono
http://www.pjsip.org

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 20 Dec 2007 13:32:22 -0500

inline:

Benny Prijono wrote:

>> reflexive candidate hasn't changed. Since you can do server reflexive 
>> and relayed together anyway, if you do the server reflexive at call 
>> setup time, then you gain no improvement in call setup delay by 
>> pre-allocating TURN.
>>
> 
> Both the STUN and TURN candidates are kept alive with periodic refresh, 
> thus no need to find out the srflx candidate before the call.

The issue isn't keepalive. I think someone else noted that a very long 
lived udp binding on a nat could potentially switch from restricted to 
symmetric; so while the binding won't change, its properties will change 
and you'll end up using turn when you wouldn't have otherwise.

Now, that said, I have no idea what the frequency of such NAT behavior 
is; whether its just a few older products or more commonplace.

>> That said, a client can go ahead and do all of this if it wants. I 
>> just don't want to add support for a channel re-bind. Instead, the 
>> client can just grab a new allocation when it runs out of channels. 
>> Anyway it'll likely reboot before then. Lets take an example. A client 
>> makes 10 calls a day (which is a lot). Each of these fork to two 
>> endpoints. This means 20 channels a day get used. With 32767 channels, 
>> thats 1638 days or 4.4 YEARS before running out of channels. The odds 
>> of a reboot of a client over 4.4 years are pretty much 100%.
> 
> Wait a second, so does this mean a client cannot reuse a channel number, 
> even after it is expired (for example when the client intentionally does 
> not send any data to refresh the permission).

Correct - it cannot reuse. Channels persist even if the permission 
towards the peer has expired. There is a race condition here, identical 
to set active destination, unless you add explicit support (per my other 
note) for reusing channels.

-Jonathan R.