Subject: Re: [BEHAVE] TURN Issue: Preserving bits in the IP header

From: Rémi Denis-Courmont <[email-masked]>
Date: Thu, 3 Jan 2008 09:49:41 +0200

Le Saturday 22 December 2007 00:15:59 ext Philip Matthews, vous avez écrit :
> The thread on exceeding the MTU size has raised another issue about
> TURN: what bits (if any) in the IP header should be preserved across
> a TURN relay?
>
> My proposal is:
>
>    A TURN server MUST preserve the Don't Fragment (DF) and TOS
> (= ECN and DiffServ) bits in the IP header when relaying IPv4  
> packets. 

I don't know what ECN will do for UDP, and I suppose that would not work for 
TCP (since it works in pair with CWR).

As with TOS, your proposal makes sense to me. In fact, I think there is an RFC 
on TOS handling through tunnels. Unfortunately, from an API perspective, that 
might be challenging enough that I am willing to bet at least some TURN 
server implementations will ignore the requirement. Namely, there is no 
standard for setting TOS per outgoing IPv4 packets (usually, it can only be 
set per socket). Perhaps RFC3542 would need to be "backported" to IPv4.

By the way, does anyone know if some NATs screw up with TOS?

-- 
Rémi Denis-Courmont

_______________________________________________
Behave mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/behave

---

From: Lars Eggert <[email-masked]>
Date: Mon, 7 Jan 2008 19:04:04 +0200

> TCP (since it works in pair with CWR).

ECN is orthogonal to what transport is used. All an ECN mark is is a  
signal from a router on the path that says "I almost dropped this  
packet."

Various transports are free to define various appropriate responses to  
that signal. TCP decided to treat the ECN signal as if there had been  
a loss, but that is only one possibility. For example, a UDP sender  
that was using TFRC to compute a safe rate might decide to reduce its  
send rate when an ECN mark is received. Or a CBR UDP sender might  
decide to stop transmission after receiving ECN marks at a certain  
rate, in order to take load off the network a bit quicker when there  
is congestion.

So ECN marks do need to be preserved, in order to let any intended  
response mechanism kick in.

(Also see draft-briscoe-tsvwg-ecn-tunnel on ECN interactions with  
tunneling.)

Lars

---

From: Philip Matthews <[email-masked]>
Date: Tue, 8 Jan 2008 04:08:19 -0500

Can you say more about the implementation complexity of doing this?
It is clearly pointless to require this if many implementations are  
going to ignore it.

- Philip

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Tue, 8 Jan 2008 12:23:16 +0200

> It is clearly pointless to require this if many implementations are

As far as I know the Winsock (Windows), Linux and BSD socket API 
implementations, none of them provide per-packet TOS and ECN bit 
specification.

I know applications can get the TOS value from a received packet on Linux/BSD, 
but they set the value individually for each sent packet. This functionality 
is present in the IPv6 APIs, but not the IPv4 ones. For ECN, as far as I 
know, you cannot send nor receive the value from userland in any of these 
operating systems.

And even then, this only works for UDP. With TCP, there is no 1:1 mapping 
between packets on the wire, and read/write socket operations, so it is 
impossible to receive any bit from the IP header anyway.

I understand why TURN should preserve TOS and ECN bits, but we have to know 
that if this were required, this would require in-kernel implementation for 
most typical operating systems. And because people will anyway try to make 
userland implementations, they'll get it wrong.

-- 
Rémi Denis-Courmont

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 8 Jan 2008 20:30:03 -0800

I think you need to separate what happens here in the TCP to UDP case  
from the UDP to UDP case.

Cullen <with my individual hat on>

On Dec 21, 2007, at 2:15 PM, Philip Matthews wrote:

> TURN: what bits (if any) in the IP header should be preserved  
> across a TURN relay?
>
> My proposal is:
>
>   A TURN server MUST preserve the Don't Fragment (DF) and TOS (=  
> ECN and DiffServ) bits in the IP header when relaying IPv4 packets.
>
> Comments?
>
> - Philip
>
>

---

From: Lars Eggert <[email-masked]>
Date: Wed, 9 Jan 2008 10:46:17 +0100

> userland implementations, they'll get it wrong.

Understood, doing things correctly is hard with existing APIs. But  
having TURN break ECN and DiffServ isn't the right approach, either.

Lars

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Thu, 10 Jan 2008 10:19:16 +0200

Le Wednesday 09 January 2008 11:46:17 Lars Eggert, vous avez écrit :
> > I understand why TURN should preserve TOS and ECN bits, but we have
> > to know that if this were required, this would require in-kernel
> > implementation for most typical operating systems. And because
> > people will anyway try to make userland implementations, they'll
> > get it wrong. 
>

Of course. I am rather wondering if this would be a SHOULD, or a MUST. That 
would make some a whole class implementations non-compliant, though they'd 
still mostly work.

For UDP (and UDP-Lite if we ever define if for TURN), we certainly want TOS 
and ECN preservation. That works fine because each client-to-relay packet 
corresponds to one or zero relay-to-peer packet, and each peer-to-relay 
packet corresponds to one or zero relay-to-client packet.

I am a whole lot more suspicious when it comes to TURN-TCP though. It is 
fortunate, we have split that out of the TURN baseline spec, because I can 
already imagine the headaches. With TCP, there is no obvious matching of IP 
packets on each sides of the relay. TCP does not preserve datagram 
boundaries. Hence a single packets from one side could turn into multiples 
packets on the other side for various reasons. Worse yet, a single packet on 
one side could be "coming from" different packets from the original side.

-- 
Rémi Denis-Courmont

---

From: Lars Eggert <[email-masked]>
Date: Thu, 10 Jan 2008 10:43:40 +0200

> packet corresponds to one or zero relay-to-client packet.

Agreed.

And I hadn't even considered TURN-TCP in my earlier emails, I thought  
we were talking about TURN-UDP only. You're right in what you write  
below; what should be done for TURN-TCP in this regard is very  
different.

Lars

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 10 Jan 2008 14:25:22 -0800

> And I hadn't even considered TURN-TCP in my earlier emails, I 
> thought we were talking about TURN-UDP only. You're right in 
> what you write below; what should be done for TURN-TCP in 
> this regard is very different.

The "TURN-UDP" document, draft-ietf-behave-turn, does support TCP
between the TURN client and its TURN server.  In all cases, 
however, the connection from the TURN server to the remote TURN
peer is always UDP.  See the table in section 3.1,
http://tools.ietf.org/html/draft-ietf-behave-turn-05#section-3.1

So, I expect "TURN-UDP" does have the problems you and Remi are 
discussing:  specifically, a (sent or received) TCP packet on the 
'left' of a TURN server will not necessarily have a 1:1 
correspondence with a UDP packet (received or sent) on the 
'right'.

-d

> > fortunate, we have split that out of the TURN baseline 
> spec, because  
> > I can
> > one side could be "coming from" different packets from the 
> original  
> > side.
> 
> Lars
>

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Fri, 11 Jan 2008 15:40:47 +0200

> correspondence with a UDP packet (received or sent) on the

There might be tricks such as disabling the Nagle algorithm and playing tricks 
with TCP/PSH and such uglinesses. However, I wonder if it makes much sense. 

The TURN client (or a TURN peer in the case of TURN-TCP) IP stack would 
normally make use of the ECN bit on a TCP packet within the context of the 
corresponding TCP session. An a QoS middlebox may do the same thing with the 
TOS bits.

This is yet another argument in favor of NOT allowing multiple TCP peer 
connections through a single TCP session to the TURN server. It basically 
comes down to the same TCP congestion control issues.
As far as I understand, it has however been more or less settled that TURN 
will allow this. Then, it may do more harm than good to preserve the ECN and 
TOS bits when using TCP on either or both sides of the TURN relay.

-- 
Rémi Denis-Courmont

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 16 Jan 2008 11:18:27 -0800

Philip Matthews wrote:
> TURN: what bits (if any) in the IP header should be preserved across a
> TURN relay?
> 
> My proposal is:
> 
>   A TURN server MUST preserve the Don't Fragment (DF) and TOS (= ECN and
> DiffServ) bits in the IP header when relaying IPv4 packets.
> 
> Comments?
> 

I propose to do not try to preserve the values of this bits in the
transport between the TURN client and server, but to carry them in the
TURN protocol.

The idea is that the bit values are different for the transport between
the TURN client and the TURN server and for the channel between the
application and the peer (even if part of this transport is tunneled):

|--------- endpoint ---------|
 Application      TURN client            TURN Server             Peer
                       |=======================|
     |-------------------------------------------------------------|
                       |=======================|

The TURN spec should define the usage of this bits between the TURN
client and the TURN server, but the usage of this bits between the
application and the peer should be left to the application using TURN.
The definition of the usage this bits in the TURN spec should make sense
for any usage of this bits between the application and the peer.

The transport of the bits can be done by adding an attribute to the Send
and Data indications.  Depending if the bits are set per channel or per
packet they will have also to be carried in the ChannelBind transaction
or as an additional header of the channel.

-- 
Marc Petit-Huguenin           [                                 ]
Home: [email-masked] [RFC1855-compliant space for rent ]
Work: [email-masked]            [                                 ]
[                                                               ]

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 16 Jan 2008 20:40:47 -0800

I think we should figure out what we want to do in the TCP to UDP case  
first then worry about the UDP to UDP case. My logic is the TCP/UDP  
case is harder and once we decide there, we will probably do the same  
thing for the UDP/UDP case.

As far as requirements, I do believe we need be able to control the DF  
and TOS. I can not imagine a strong use case where we needed different  
values of DF or TOS on one side vs the other side but perhaps someone  
will think of one.  I do feel very strongly that it should be possible  
to implement whatever we do on common OS without too many  
complications. It would also be important for the P2P work that a TURN  
server could run without being "super user" sort of privileges.

I can imagine solutions like Marc is proposing or ones that just  
preserve the bits from side to side but I'm not sure if both theses  
are easily implementable. Can anyone point at details of how to do this?

Cullen <with my individual contributor hat on >

On Jan 16, 2008, at 11:18 AM, Marc Petit-Huguenin wrote:

>> across a

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Thu, 17 Jan 2008 10:21:07 +0200

Le Wednesday 16 January 2008 21:18:27 ext Marc Petit-Huguenin, vous avez 
> transport between the TURN client and server, but to carry them in the

Most probably we don't want to customize the DF bit for TCP packets. Rather we 
should let the TCP stack figure the MSS and do all the magic by itself.

I can think of two sensible options:

- One option is to replicate the bits from each sides. This only works if we 
have UDP on both sides.

- Another option is to specify the desired values for sending, during the 
binding (or allocate) requests. That works fine in all combination of TCP and 
UDP, on the client->relay->peer direction, under the assumption that the 
values are constant for a TURN binding lifetime. This does not put any extra 
requirement on the TURN client IP stack, and puts requirements that should be 
met by any server-grade operating system on the TURN relay stack.

In the peer->relay->client direction, I would think the client does not really 
care about the "incoming" DF and TOS bits. These are mostly of interest to 
the boxes on the path. As such, I don't see much point into carrying them 
into the TURN protocol. So we should try to replicate the bits in UDP-to-UDP 
case. With TCP, we may have to ignore them altogether, or use the "sending 
values" set by the TURN client.

> The transport of the bits can be done by adding an attribute to the
> Send and Data indications.  Depending if the bits are set per channel
> or per packet they will have also to be carried in the ChannelBind
> transaction or as an additional header of the channel.

I very much disagree. First this may add a big overhead, as we may end up 
using Send/Data indications all the time. Second, that hides the TOS/DF bits 
away from those who actually care about them: the middleboxes.

-- 
Rémi Denis-Courmont

---

From: Rohan Mahy <[email-masked]>
Date: Thu, 17 Jan 2008 01:02:34 -0800

If we do anything at all, then Remí's second suggestion seems the  
most sensible to me.

Of course there are those who believe that TURN is only going to get  
used in practice when sending traffic across administrative domains.  
In most of those cases, the TURN server will be reasonably close to  
the core of the Internet.  The markings between the client and the  
TURN server and between the peer and between the TURN server and the  
peer will typically be remarked to zero.  While we could influence  
the ToS in the same domain as the TURN server, this seems unlikely to  
be important.

thanks,
-rohan

On Jan 17, 2008, at 12:21 AM, Rémi Denis-Courmont wrote:

>>> ECN and

---

From: Lars Eggert <[email-masked]>
Date: Thu, 17 Jan 2008 14:16:34 +0100

On 2008-1-17, at 5:40, ext Cullen Jennings wrote:
> As far as requirements, I do believe we need be able to control the  
> DF and TOS.

The bits that should be preserved across a TURN relay are the ones  
that are important end-to-end. The DF bit isn't one of these, because  
the transport connections are terminated at the TURN relay, so it  
needs to diddle them according to what its local stack requires, for  
both relayed connections.

For the ECN bits, it matters what transport protocols are used:

For UDP-UDP relays, and assuming a 1:1 packet ratio, the ECN bits can  
be relayed, to let the applications react to them. (With different  
ratios, my feeling is that they can also be relayed, but I need to  
think about this more.)

For TCP-UDP relays, the ECN bits should not be relayed, because the  
transport connection for which they are a signal terminates at the  
TURN relay, and it should be the entity to react to the signal.  
Relaying the ECN bits doesn't make sense, because they have no meaning  
to the control loop that controls transmissions on the "other side" of  
the TURN relay.

With the DSCP, issues are similar than for ECN. For the UDP case,  
where there's a 1:1 correspondence between packets sent and received,  
the TURN relay can copy (or remark) the DSCP. For UDP-TCP, it's not  
clear how that could happen.

Lars

---

From: Philip Matthews <[email-masked]>
Date: Thu, 17 Jan 2008 09:14:33 -0500

At its heart, this issue is about whether we see one end-to-end  
connection between the client and the peer, or whether we see two  
connections (client to server, and server to peer) with some linkage  
between them.

I originally subscribed to the first view, and had the idea that the  
relay could be viewed as forwarding at the IP layer, not the UDP layer.

However, various things now make me think that the second view is the  
correct view, with the support for TCP to UDP relaying being a key  
point.

If we adopt the second view, then I think the answer is that the ECN  
and DF bits on the two connections are independent of each other.

To see why I think this is the right choice, consider the TCP to UDP  
relaying case. In this case, TCP is controlling the settings of the  
ECN and DF bits on the TCP connection, while UDP is controlling the  
setting of these bits on the UDP connection. I feel this is  
especially obvious if you consider the case of a single TCP  
connection to the server, but two or more UDP connections from the  
server to two or more peers.

Regarding the DS bits, I think the best solution is for the client to  
send the server some indication of what type of traffic is being  
transported, and let the server set the DS bits appropriately. Such  
an indication would also be useful to the server when determining  
whether to allow a new allocation.

Specifically, I propose that:

      The transport layer used between the client and the server will  
set the ECN, and DF bits as it sees fit, and similarly the transport  
layer between the client and the peer will set the ECN and DF bits as  
it sees fit. The client cannot influence or learn the ECN and DF bits  
settings between the server and the peer. The client will send the  
server an indication of the traffic type at allocation time, and the  
server will use this to set the DS bits on the server-to-peer  
connections (and perhaps on the server-to-client connection as well).

Comments?

- Philip

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 17 Jan 2008 08:29:03 -0800

On Jan 17, 2008, at 5:16 AM, Lars Eggert wrote:

> that are important end-to-end. The DF bit isn't one of these,  
> because the transport connections are terminated at the TURN relay,  
> so it needs to diddle them according to what its local stack  
> requires, for both relayed connections.
>
> For UDP-UDP relays, and assuming a 1:1 packet ratio, the ECN bits  
> can be relayed, to let the applications react to them. (With  
> different ratios, my feeling is that they can also be relayed, but I  
> need to think about this more.)
>
> Relaying the ECN bits doesn't make sense, because they have no  
> meaning to the control loop that controls transmissions on the  
> "other side" of the TURN relay.
>
> where there's a 1:1 correspondence between packets sent and  
> received, the TURN relay can copy (or remark) the DSCP. For UDP-TCP,  
> it's not clear how that could happen.
>
> Lars

works for me - I was also thinking about doing end to end PMTU.  Would  
the client need a way to tell the relay to set the DF on the traffic  
to the peer?

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 17 Jan 2008 08:33:57 -0800

On Jan 17, 2008, at 6:14 AM, Philip Matthews wrote:

> However, various things now make me think that the second view is  
> the correct view, with the support for TCP to UDP relaying being a  
> key point.
>
> Regarding the DS bits, I think the best solution is for the client  
> to send the server some indication of what type of traffic is being  

Hmm - I have no idea how I would implement "let the server set the DS  
bits appropriately". I think the DS bits are the indication of what  
type of traffic is. I'd be happier with the client told the relay the  
DS bits to use on the other side. But as long as it is clear how to  
implement this these seems like a possible path.

> layer between the client and the peer will set the ECN and DF bits  
> as it sees fit. The client cannot influence or learn the ECN and DF  
> bits settings between the server and the peer. The client will send  
> the server an indication of the traffic type at allocation time, and  
> the server will use this to set the DS bits on the server-to-peer

---

From: Philip Matthews <[email-masked]>
Date: Thu, 17 Jan 2008 11:47:14 -0500

On 17-Jan-08, at 11:33 , Cullen Jennings wrote:
>
> Hmm - I have no idea how I would implement "let the server set the  
> DS bits appropriately". I think the DS bits are the indication of  
> what type of traffic is. I'd be happier with the client told the  
> relay the DS bits to use on the other side. But as long as it is  
> clear how to implement this these seems like a possible path.

Please correct me if I am wrong, but I seem to recall that the exact  
DS codepoint for a given traffic type is network-dependent. If I am  
correct, then the client cannot say to the server "use codepoint X  
for my traffic", because the network in which the server resides may  
use codepoint Y for that traffic type. So the client needs to instead  
say "use the codepoint for VoIP traffic" and that causes the server  
to use codepoint Y.

Does this make any sense?

- Philip

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 17 Jan 2008 10:05:20 -0800

I'm assuming that the operator that configured the UA also knew about  
the TURN server and understands what DSCP would be appropriate both at  
the UA and at the TURN server. It's a bit more complicated than just  
"set the DS bits for voip" - the type of media changes things,  
signaling changes, different types of calls and different users or  
systems may get different ones. It seems like it will be hard to  
communicate enough information from the UA to the TURN server to have  
it make theses decision. It can probably be done if you want to go  
this way but there are some things to deal with.

I think what this comes down to is that if you believe TURN is only  
for RTP, then it is a much more tractable problem than if you believe  
TURN is for many things.

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Fri, 18 Jan 2008 10:29:41 +0200

> implement this these seems like a possible path.

What are the DS bits?

At least some IP stacks do allow, without any specific privileges:

* on a per-socket basis:
  - enabling/disabling path MTU discovery, which determines the DF,
  - setting the TOS bits,
  - setting the IPv4 TTL and IPv6 HL,

* on a per-datagram basis (UDP, UDP-Lite and raw only):
  - getting the TOS bits,
  - getting the IPv4 TTL or IPv6 HL,
  - setting the TOS bits for each outgoing packets (IPv6-only) [1],
  - setting the IPv6 HL [1].

I am not aware of any socket stack that provides access to ECN bits, but then 
again, I don't know that many socket stacks either.

By the way, should we preserve the TTL as well? so that you cannot make TURN 
loops??

[1] For IPv6, (informational) Advanced socket API RFC specifies it, hence 
widely implemented. Doable for IPv4 too, but no stack seemed to bother 
providing the option this far. I did have an unofficial Linux patch for this 
at some point.

And if we really want to split hairs, what about the IPv6 Traffic Class bits?

-- 
Rémi Denis-Courmont

---

From: Philip Matthews <[email-masked]>
Date: Fri, 18 Jan 2008 09:49:37 -0500

> What are the DS bits?

The DS bits are the DiffServ bits.
The old TOS field in the IP header has now been divided into a 6-bit  
DiffServ field and a 2-bit ECN field.

And thanks for the info on what can and cannot be done with standard  
APIs.

- Philip

---

From: Rohan Mahy <[email-masked]>
Date: Fri, 18 Jan 2008 07:38:43 -0800

Philip,

The DS codepoint can be network-dependent, but since DS codepoints  
are almost always reset to zero upon crossing an administrative  
domain boundary, the point is moot.   If there is no TURN server,  
peer1 says use codepoint X and the traffic arrives at peer2 marked  
with zero.

With a TURN server we could do the following things:

1) Have the TURN server explicitly use zero for all traffic.  This is  
appropriate for a TURN server in the core of the public Internet.
2) Have the TURN server copy the codepoint set by the client to the  
traffic for the peer.  In the core of the Internet, this will be  
copying zero to zero.
If the client, peer, and TURN server are all in the same  
administrative domain and all in the same access network, it could  
cause the correct codepoint to be copied.  Note that this case is  
highly unlikely.  It won't happen in any service provider network  
that I am familiar with, and any enterprise network who was trying to  
be nice enough to install their own TURN server in the access network  
would probably just configure their firewall so the client could  
communicate with the peer (in the same enterprise) using local  
addressing or STUN instead.
3) Have the client indicate what codepoint to use and have the TURN  
server honor this.  For a TURN server in the public Internet, this  
will allow the client to ask the server to mark traffic in the core  
of the network (where it is completely and utterly useless); the  
codepoint will then be remarked to zero before going to the peer's  
access network.  As we saw in #2 this approach is equally useless in  
an enterprise setting.

In summary, approach #1 is easy, approach #2 is useless, and approach  
#3 is useless.  I vote for approach #1.

thanks,
-rohan

On Jan 17, 2008, at 8:47 AM, Philip Matthews wrote:
> Please correct me if I am wrong, but I seem to recall that the  
> exact DS codepoint for a given traffic type is network-dependent.  
> If I am correct, then the client cannot say to the server "use  
> codepoint X for my traffic", because the network in which the  
> server resides may use codepoint Y for that traffic type. So the  
> client needs to instead say "use the codepoint for VoIP traffic"  
> and that causes the server to use codepoint Y.
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 18 Jan 2008 17:04:10 +0100

Rohan Mahy skrev:
> Philip,
> 
> The DS codepoint can be network-dependent, but since DS codepoints are 
> almost always reset to zero upon crossing an administrative domain 
> boundary, the point is moot.   If there is no TURN server, peer1 says 
> use codepoint X and the traffic arrives at peer2 marked with zero.
> 
> traffic for the peer.  In the core of the Internet, this will be copying 
> zero to zero.
> If the client, peer, and TURN server are all in the same administrative 
> domain and all in the same access network, it could cause the correct 
> codepoint to be copied.  Note that this case is highly unlikely.  It 
> won't happen in any service provider network that I am familiar with, 
> and any enterprise network who was trying to be nice enough to install 
> their own TURN server in the access network would probably just 
> configure their firewall so the client could communicate with the peer 
> (in the same enterprise) using local addressing or STUN instead.
> server honor this.  For a TURN server in the public Internet, this will 
> allow the client to ask the server to mark traffic in the core of the 
> network (where it is completely and utterly useless); the codepoint will 
> then be remarked to zero before going to the peer's access network.  As 
> we saw in #2 this approach is equally useless in an enterprise setting.
> 
> In summary, approach #1 is easy, approach #2 is useless, and approach #3 
> is useless.  I vote for approach #1.

As an individual contributor:

I am not agreeing with this conclusion that it is useless to preserver 
DSCP. There clearly are cases where TURN will be deployed in an operator 
or company network where it is desired to use diffserv and keep the 
mappings that has been created. This at least applies for UDP to UDP 
usage. It is likely to do so even TCP-UDP. So I don't think you get to 
take the easy way out of this.

I think approach 2 is the right one, as that will even work when TURN 
client and server are in different AS with different realization of the 
PHBs mapping to DSCP.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 18 Jan 2008 17:20:59 +0100

Lars Eggert skrev:
>> and TOS.
> The bits that should be preserved across a TURN relay are the ones that 
> are important end-to-end. The DF bit isn't one of these, because the 
> transport connections are terminated at the TURN relay, so it needs to 
> diddle them according to what its local stack requires, for both relayed 
> connections.
> 

I think we have some problem of being consistent if the relay are 
resulting in two different transport connections or not, at least for 
UDP. For UDP it would be good if it mostly was one end-to-end 
connection. Especially in regards to congestion control. All congestion 
or at least adaptation or rate limiting happens on the protocol above 
UDP. To be able to handle a split transport connection correctly you 
basically must know this protocol and be able to perform what ever 
action on that level to correctly reflect what is happening.

For UDP to UDP is is mostly copying the bits. In fact for some field 
this will be required (DSCP) as they are only possible to interpret 
within the current domain the packet is present in. A client can't know 
what DSCP to set for the domain the TURN server is present in. DF bit 
and the ICMP error resulting from fragmentation is problematic as the 
TURN server should relay the ICMP message back. And if itself is 
responsible for fragmentation due to adding the TURN encapsulation 
header also generate ICMP towards the packet sender.

For TCP-UDP this gets complicated as packet to packet mapping is not 
likely to work. Also the congestion control part is interesting as 
packets from UDP -> TCP can result in buffer overflow in the TURN 
server. In fact I think it is important to recommend the TURN server to 
not have a to large buffer here to actually cause these packet loss as 
congestion indications. I think maintaining DSCP is one of the more 
straightforward solutions. ECN is interesting, but I think a TURN server 
needs to forward these markings somehow in the UDP->TCP direction. In 
the other direction the problem doesn't exist as the client to server 
TCP will take care of this and reduce the rate. The alternative is to 
basically drop all arriving UDP packets which have the ECN field set to 
Congestion Experienced (CE).

So I don't think we have finished thinking about what to do with this one.

Cheers

Magnus Westerlund

---

From: Philip Matthews <[email-masked]>
Date: Fri, 18 Jan 2008 11:23:00 -0500

I am going to update my proposal slightly. I am now going to propose  
that the TURN server and the TURN client SHOULD always set the DF  
bit. This would allow Packetization Layer Path MTU Discovery [RFC  
4821] to work. (Note that classic Path MTU Discovery won't work  
because TURN does not currently relay ICMP packets. )

To summarize the revised proposal:
* There are two distinct connections: one from the client to the  
server, and a second from the server to the peer.
* The ECN bits are set by the transport protocol used on each  
connection. For example, if TCP is used between the client and the  
server, then TCP sets the ECN bits on that connection, and these  
setting have nothing to do with how UDP sets the ECN bits on the  
connection between the server and a peer.
* The DF bit SHOULD always be set by the client or the server when  
sending a packet.

How the DiffServ bits are set is still under discussion. I originally  
proposed that the client should send some indication of traffic type  
to the server, which then determines an appropriate DiffServ Code  
Point (DSCP), which is then used for the lifetime of the connection  
(or at least, for a period of time and not per packet). Cullen  
doesn't see how this could work, and proposes that the client just  
tell the server which DSCP to use; again a single setting for the  
lifetime of the connection. And Rohan is now proposing that the  
server just always uses '000000'.

 From Remi's recent e-mail, it seems that any one of these variations  
could be easily implemented by a user-space implementation.

Comments?
In particular, are people happy with everything but the DiffServ bits  
part?

- Philip

On 17-Jan-08, at 09:14 , Philip Matthews wrote:

> connections (client to server, and server to peer) with some  
> linkage between them.
>
> I originally subscribed to the first view, and had the idea that  
> the relay could be viewed as forwarding at the IP layer, not the  
> UDP layer.
>
> If we adopt the second view, then I think the answer is that the  
> ECN and DF bits on the two connections are independent of each other.
>
> To see why I think this is the right choice, consider the TCP to  
> UDP relaying case. In this case, TCP is controlling the settings of  
> the ECN and DF bits on the TCP connection, while UDP is controlling  
> the setting of these bits on the UDP connection. I feel this is  
>      The transport layer used between the client and the server  
> will set the ECN, and DF bits as it sees fit, and similarly the  
> transport layer between the client and the peer will set the ECN  
> and DF bits as it sees fit. The client cannot influence or learn  
> the ECN and DF bits settings between the server and the peer. The  
> client will send the server an indication of the traffic type at  
> allocation time, and the server will use this to set the DS bits on  
> the server-to-peer connections (and perhaps on the server-to-client  
> connection as well).
>
> Comments?
>
> - Philip
>
>
>

---

From: Rohan Mahy <[email-masked]>
Date: Fri, 18 Jan 2008 08:37:13 -0800

Hi Magnus,

I am still skeptical about this statement.  Can you provide an  
example use case for this?
> There clearly are cases where TURN will be deployed in an operator  
> mappings that has been created.

If there are indeed use cases for this, then maybe the TURN server  
can be configured to passthrough or to remark everything to zero.

thanks,
-rohan

On Jan 18, 2008, at 8:04 AM, Magnus Westerlund wrote:

>> 1) Have the TURN server explicitly use zero for all traffic.  This  
>> is appropriate for a TURN server in the core of the public Internet.
>> 2) Have the TURN server copy the codepoint set by the client to  
>> the traffic for the peer.  In the core of the Internet, this will  
>> be copying zero to zero.
>> that I am familiar with, and any enterprise network who was trying  
>> to be nice enough to install their own TURN server in the access  
>> network would probably just configure their firewall so the client  
>> could communicate with the peer (in the same enterprise) using  
>> local addressing or STUN instead.
>> 3) Have the client indicate what codepoint to use and have the  
>> TURN server honor this.  For a TURN server in the public Internet,  
>> this will allow the client to ask the server to mark traffic in  
>> the core of the network (where it is completely and utterly  
>> useless); the codepoint will then be remarked to zero before going  
>> to the peer's access network.  As we saw in #2 this approach is  
>> equally useless in an enterprise setting.
>> In summary, approach #1 is easy, approach #2 is useless, and  
>> approach #3 is useless.  I vote for approach #1.
>
> I am not agreeing with this conclusion that it is useless to  
> preserver DSCP. There clearly are cases where TURN will be deployed  
> in an operator or company network where it is desired to use  
> diffserv and keep the mappings that has been created. This at least  
> applies for UDP to UDP usage. It is likely to do so even TCP-UDP.  
> So I don't think you get to take the easy way out of this.
>
> I think approach 2 is the right one, as that will even work when  
> TURN client and server are in different AS with different  
> realization of the PHBs mapping to DSCP.
>
> Cheers
>
> Magnus Westerlund
>

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Mon, 21 Jan 2008 09:26:05 +0200

> DiffServ field and a 2-bit ECN field.

In that case, getting/setting the ECN bits is as easy/difficult as 
getting/setting the DSCP bits. You can use the TOS API in both cases.

-- 
Rémi Denis-Courmont

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 21 Jan 2008 18:00:28 +0100

Rohan Mahy skrev:
> Hi Magnus,
> 
> I am still skeptical about this statement.  Can you provide an example 
> use case for this?
>> There clearly are cases where TURN will be deployed in an operator or 
>> company network where it is desired to use diffserv and keep the 

To my understanding a cellular network would such a network where 
diffserv could be used in the service network where the TURN server 
would be placed. In that case you don't want to change the QoS 
properties of the relayed media traffic by having the turn server reset 
the DSCP field.

> 
> 
> If there are indeed use cases for this, then maybe the TURN server can 
> be configured to passthrough or to remark everything to zero.

Hmmm, I don't know if it is so simple. We are talking about the end to 
end behavior of relayed traffic. You are suggesting that it should be 
configurable if one like to break diffserv (and likely PCN) or not by 
having a TURN server. And the reason we having this discussion is 
because of the short comings of the OS APIs when sending and receiving 
packets. Yes, I understand that user-land implementations of TURN seems 
to become pretty hard. But I don't think sacrificing core IP 
functionality is okay.

Cheers

Magnus Westerlund

---

From: Philip Matthews <[email-masked]>
Date: Tue, 29 Jan 2008 09:45:38 -0500

Magnus:

I am replying to you under the "Preserving bits in the IP header"  
thread, since that is what we are talking about.

As a general comment, I think many of us now view this as perhaps the  
hardest open issue that we have to solve.

More inline.

- Philip

On 29-Jan-08, at 05:31 , Magnus Westerlund wrote:
> With my individual hat on!
>
> Philip Matthews skrev:
>> *) Preserving bits in the IP header. When relaying packets, how  
>> should various IP header bits be set?
>> There was general agreement that the path between the client and  
>> the peer should be considered as two distinct connections (client  
>> <--> server and server <--> peer). However, we were not able to  
>> make much progress beyond that.
>
> I personally don't agree with this characterization when it comes  
> to UDP on both legs. Because in this case unless you threat this as  
> a single end to end path that are bounced of an reflector, the TURN  
> server will be required to resolve for example the UDP congestion  
> control problems between the server and its peers. It also will  
> need QoS reservation and path MTU discovery signalling interfaces.  
> The amount of information that needs to be sent between the TURN  
> client and the Turn server for each packet will be significantly  
> increased.
>
> Even in the case of TCP -> UDP is problematic. I don't see your  
> agreement to work in the real world.

Personally, I am reluctant to treat the UDP-UDP case differently from  
the TCP-UDP case. So, unless we drop the TCP-UDP case, then I feel we  
need to focus an a good solution for that case, since it seems to be  
the harder one (as I think Cullen first pointed out).

I think we are looking for a solution with the following requirements:
a) Must work for both the UDP-UDP case and the TCP-UDP case
b) Must work (or mostly work) when the TURN server runs as a  
ordinary, unprivileged user-land application
c) Must provide a solution to the MTU exceeded problem (on either the  
client-server leg or the server-peer leg)
d) Must provide a way to have end-to-end QOS.

>
>
>> EKR proposed that, as a guiding principle, we should only mandate  
>> stuff that could be implemented in a userland application that  
>> does not have special (= root) privileges.
>
> I think this requirement may result in that no solutions exist with  
> most current operating systems.

It may be that we will have to identify what will be lost with a non- 
privileged user-land implementation. However, I think that many  
people want to implement this type of implementation, so it behooves  
us to look hard for what we can achieve with such an implementation.

- Philip

---

From: Spencer Dawkins <[email-masked]>
Date: Tue, 29 Jan 2008 09:32:36 -0600

>>> EKR proposed that, as a guiding principle, we should only mandate  stuff 
>>> that could be implemented in a userland application that  does not have 
>>> special (= root) privileges.
>>
>> I think this requirement may result in that no solutions exist with  most 
>> current operating systems.
>
> privileged user-land implementation. However, I think that many  people 
> want to implement this type of implementation, so it behooves  us to look 
> hard for what we can achieve with such an implementation.
>
> - Philip

FWIW, I agree with EKR's goal, Magnus's concern, and Philip's compromise :-)

Thanks,

Spencer

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 29 Jan 2008 22:06:31 +0100

On Tue, Jan 08, 2008 at 12:23:16PM +0200,
 Rémi Denis-Courmont <[email-masked]> wrote 

> implementations, none of them provide per-packet TOS and ECN bit

To ease the current survey of implementations, I wrote the attached
small program, that you can run on your system (tested on many Unix,
VMS - if someone wants to run a TURN server on VMS - and Windows XP),
to see what bits in the IP header an ordinary application can *set*
(wether it can *gets* them is another issue).

To compile:

make

To run:

make test

If you are root, I suggest to run it with and without root privileges,
the results may be different (for instance on Linux).

It may be a good idea to send the results only to me or to the persons
responsible for the tests (see
http://www1.ietf.org/mail-archive/web/behave/current/msg03248.html).

A big warning: the program only tests that the system accepts the
option, not that it complies. 

Executive summary of the results so far: TTL, ECN and DSCP seem to be
easily settable. precedence and DF are more questionable.

Some results:

My UID is 1000
The system is Linux 2.6.8-4-k7, the machine is i686
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
[PROBLEM] Cannot set the precedence to 160:  (Operation not permitted)
DF set through PMTU successfully set to 2
DF clear through PMTU successfully set to 0

My UID is 0
The system is Linux 2.6.8-4-k7, the machine is i686
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
DF set through PMTU successfully set to 2
DF clear through PMTU successfully set to 0

My UID is 1055
The system is OSF1 V5.1, the machine is alpha
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system

My UID is 54131
The system is FreeBSD 6.2-RELEASE, the machine is i386
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system

My UID is 1000
The system is NetBSD 3.1, the machine is sparc64
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system

My UID is 0
The system is Darwin 9.1.0, the machine is i386
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system

My UID is 1005
The system is CYGWIN_NT-5.1 1.5.24(0.156/4/2), the machine is i686
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system

My UID is 8491703
The system is OpenVMS 0, the machine is HP_rx2600__(1.40GHz/1.5MB)
TTL successfully set to 10
TOS successfully set to 8
ECN successfully set to 2
precedence successfully set to 160
No known way to set the DF bit on this system
_______________________________________________
Behave mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/behave

---

From: Philip Matthews <[email-masked]>
Date: Tue, 29 Jan 2008 18:24:20 -0500

Thanks for writing this program!

I am curious to know exactly which bits are being set by each operation.
The old Type of Service field from the IPv4 header was defined as  
(RFC 791)
          0     1     2     3     4     5     6     7
       +-----+-----+-----+-----+-----+-----+-----+-----+
       |                 |     |     |     |     |     |
       |   PRECEDENCE    |  D  |  T  |  R  |  0  |  0  |
       |                 |     |     |     |     |     |
       +-----+-----+-----+-----+-----+-----+-----+-----+

while the new definition of this field is (RFC 2474):
         0   1   2   3   4   5   6   7
       +---+---+---+---+---+---+---+---+
       |         DSCP          |  ECN  |
       +---+---+---+---+---+---+---+---+

When your program refers to the TOS field, does it mean bits 0-7, or  
just bits 3-5?

- Philip

On 29-Jan-08, at 16:06 , Stephane Bortzmeyer wrote:

> No known way to set the DF bit on this system<Makefile><ip-header- 
> set.c>_______________________________________________
> Behave mailing list
> [email-masked]

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 30 Jan 2008 11:00:29 +0100

On Tue, Jan 29, 2008 at 06:24:20PM -0500,
 Philip Matthews <[email-masked]> wrote 

> The old Type of Service field from the IPv4 header was defined as  

Actually, RFC 2474 + RFC 3168 (RFC 2474 did not mention ECN and
reserved the last two bits). Strangely, neither RFC 2474 nor 3168
claims they update RFC 791.

I admit that my vocabulary was sloppy. This is partially because Posix
did not change the name of the constants after RFC 2474. The argument
of setsockopt() is still named IP_TOS. DSCP does not seem mentioned
anywhere. On my very recent Linux machine, the comments in the header
file are still RFC 791:

#define        IP_TOS          1       /* int; IP type of service and precedence.  */

I hesitate to update the program: should I use RFC 2474 vocabulary or
Posix one?

> When your program refers to the TOS field, does it mean bits 0-7, or  

The IP_TOS option apparently allows you to change the entire byte, so
bits 0-7. Therefore, 0x08 (maximize throughput) means bit 4 set and
all other bits clear (including ECN bits). If someone wants ECN *and*
maximize throughput, he or she should set the field to 0x0A.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 31 Jan 2008 22:10:56 -0800

that is very cool - thank you for writing that. Any thoughts on being  
able to read any of these bits?

On Jan 29, 2008, at 1:06 PM, Stephane Bortzmeyer wrote:

> <Makefile.txt><ip-header- 

_______________________________________________
Behave mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/behave