Subject: Re: [Sip] Re: issue with oubound and the used transport protocol

From: Hisham Khartabil <[email-masked]>
Date: Mon, 4 Dec 2006 15:57:44 +0100

I like this proposal, although I would not use MUST. Either of these is 
optional. Both is recommended.

Hisham

On Nov 17, 2006, at 6:20 PM, Jeroen van Bemmel wrote:

> The outbound draft needs to say: "if a UA establishes a flow using an 
> unreliable transport, it MUST also establish one over a reliable 
> transport" (to support switch over). Moreover, I would suggest to add 
> "it is RECOMMENDED that the UA establishes a flow using a reliable 
> transport"
>
> Regards,
> Jeroen
>
>

_______________________________________________
Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: <[email-masked]>
Date: Tue, 5 Dec 2006 11:04:05 +0200

Hi,

I don't see the point in this proposal. If the UA can establish a
flow/connection for a reliable transport (TCP), why would it need the
unreliable transport (UDP) flow in the first place? All traffic could be
sent over TCP anyway. 

Markus  

>-----Original Message-----
>From: ext Hisham Khartabil [mailto:[email-masked]] 
>Sent: 04 December, 2006 16:58
>To: Jeroen van Bemmel
>Cc: [email-masked]; Rohan Mahy; [email-masked]; Leis,Peter
>Subject: Re: [Sip] Re: issue with oubound and the used 
>transport protocol
>
>I like this proposal, although I would not use MUST. Either of 
>these is optional. Both is recommended.
>
>Hisham
>
>This list is for NEW development of the core SIP Protocol Use 
>[email-masked] for questions on current sip

---

From: Hisham Khartabil <[email-masked]>
Date: Tue, 5 Dec 2006 10:07:32 +0100

Because some service providers don't want to use TCP.

Hisham

---

From: <[email-masked]>
Date: Tue, 5 Dec 2006 11:14:02 +0200

Hi,

The proposal says that the UA MUST open a reliable transport connection
too. So, the service provider infra would need to support TCP
connections from all UAs using outbound. So, if the service provider or
the proxy vendor would be concerned about the number of TCP connections,
this would not help. But are you saying that the issue would not the
number of connections but the traffic over those connections? Can you
elaborate this?

Markus

>Sent: 05 December, 2006 11:08
>To: Isomaki Markus (Nokia-SIR/Espoo)
>Cc: [email-masked]; [email-masked]; 
>[email-masked]; [email-masked]; [email-masked]
>> unreliable transport (UDP) flow in the first place? All 
>traffic could 
>> be sent over TCP anyway.
>>
>> Markus
>>
>>> Subject: Re: [Sip] Re: issue with oubound and the used transport 
>>> protocol
>>>
>>> I like this proposal, although I would not use MUST. Either 
>of these 
>>> is optional. Both is recommended.
>>>
>>> Hisham
>>>
>>> [email-masked] for questions on current sip Use 
>>> [email-masked] for new developments on the application of sip
>>>
>>
>
>

---

From: Fredrik Thulin <[email-masked]>
Date: Tue, 05 Dec 2006 10:21:03 +0100

> Because some service providers don't want to use TCP.

Spending IETF working group time and efforts on making standards that 
try to cope with people implementing only the bits and pieces they seem 
fit from earlier standards is both irresponsible and stupid.

I know you know that TCP+TLS is mandated for a proxy by RFC3261.

/Fredrik

---

From: Hisham Khartabil <[email-masked]>
Date: Tue, 5 Dec 2006 10:23:56 +0100

On Dec 5, 2006, at 10:14 AM, <[email-masked]> wrote:

> too.

My proposal didn't say MUST.

Hisham

> So, the service provider infra would need to support TCP

---

From: Hisham Khartabil <[email-masked]>
Date: Tue, 5 Dec 2006 11:08:32 +0100

On Dec 5, 2006, at 10:21 AM, Fredrik Thulin wrote:

> try to cope with people implementing only the bits and pieces they 
> seem fit from earlier standards is both irresponsible and stupid.

Spoken like a true academic. This has nothing to do with 
implementation, but has a lot to do with deployment. You can implement 
all the MUSTs you want, but you cannot mandate its use, TCP and TCP/TLS 
is one great example of that.

If the IETF does not want to solve real life deployment problems but 
instead wants to solve academic ones, then service providers will start 
deploying proprietary solutions to fit their immediate needs and 
vendors will implement products that no service provider cares for.

Thanks,
Hisham

---

From: "Drage, Keith \(Keith\)" <[email-masked]>
Date: Tue, 5 Dec 2006 12:45:35 +0100

While I would not necessarily have phrased it as Fredrik did, he is
right.

If you don't want to reflect the previous decisions of the SIP WG that
already exist in published deliverables, then you should be seeking to
change those published deliverables.

For security, the way to do that is by discussing and including
appropriate text in draft-ietf-sip-sips-00. I don't believe I have seen
much Hisham sourced input.

If we don't do this we end up down a compatibility discussions rathole
from which the group will never emerge.

Regards

Keith

> From: Hisham Khartabil [mailto:[email-masked]] 
> Sent: 05 December 2006 10:09
> To: Fredrik Thulin
> Cc: [email-masked]; [email-masked]; [email-masked]; 
> [email-masked]; [email-masked]
> implementation, but has a lot to do with deployment. You can 
> implement all the MUSTs you want, but you cannot mandate its 
> use, TCP and TCP/TLS is one great example of that.
> 
> If the IETF does not want to solve real life deployment 
> problems but instead wants to solve academic ones, then 
> service providers will start deploying proprietary solutions 
> to fit their immediate needs and vendors will implement 
> products that no service provider cares for.
> 
> Thanks,
> Hisham
> 
> >

---

From: Fredrik Thulin <[email-masked]>
Date: Tue, 05 Dec 2006 13:07:52 +0100

Hisham Khartabil wrote:
> Spoken like a true academic. This has nothing to do with implementation, 
> but has a lot to do with deployment. You can implement all the MUSTs you 
> want, but you cannot mandate its use, TCP and TCP/TLS is one great 
> example of that.
> 
> deploying proprietary solutions to fit their immediate needs and vendors 
> will implement products that no service provider cares for.

After sending I realized that my words were a bit harsh, for which I 
apologize. I guess you mistake the university domain name of mine with 
academic reasoning, but the truth is that I am _not_ interested in SIP 
in an academic way. I'm actually trying to deploy SIP on Stockholm 
university.

Even though I have implemented lots of things (including draft-outbound 
03) my primary concern is deployment, I can assure you. If there is 
anything I need for deployment it is simplicity. Outbound with TCP/TLS 
is a piece of cake - Outbound with UDP clearly is not. It is not TCP/TLS 
that is adding complexity, but UDP. I mentioned in another e-mail that 
my code to deframe SIP messages received over streams were less than 130 
lines of code. My STUN demuxing-responding code for UDP is about the 
double of that.

I'm assuming that you are a provider of SIP service to end users who 
might often be behind NAT. Can you please explain how you see it as the 
most deployable solution to base it all on UDP instead of offering 
TCP/TLS in your service and even advocating the use of UACs that support 
TCP/TLS?

/Fredrik

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Tue, 5 Dec 2006 13:14:36 +0100

Hi,

I still fail to see all the "complexity" that UDP adds.

Regards,

Christer 

> From: Fredrik Thulin [mailto:[email-masked]] 
> Sent: 5. joulukuuta 2006 14:08
> To: Hisham Khartabil
> > all the MUSTs you want, but you cannot mandate its use, TCP and 
> > TCP/TLS is one great example of that.
> > 
> problems but 
> > instead wants to solve academic ones, then service providers will 
> > start deploying proprietary solutions to fit their 
> immediate needs and 
> After sending I realized that my words were a bit harsh, for 
> which I apologize. I guess you mistake the university domain 
> name of mine with academic reasoning, but the truth is that I 
> am _not_ interested in SIP in an academic way. I'm actually 
> trying to deploy SIP on Stockholm university.
> 
> 03) my primary concern is deployment, I can assure you. If 
> there is anything I need for deployment it is simplicity. 
> Outbound with TCP/TLS is a piece of cake - Outbound with UDP 
> clearly is not. It is not TCP/TLS that is adding complexity, 
> but UDP. I mentioned in another e-mail that my code to 
> deframe SIP messages received over streams were less than 130 
> lines of code. My STUN demuxing-responding code for UDP is 
> about the double of that.
> 
> I'm assuming that you are a provider of SIP service to end 
> users who might often be behind NAT. Can you please explain 
> how you see it as the most deployable solution to base it all 
> on UDP instead of offering TCP/TLS in your service and even 
> advocating the use of UACs that support TCP/TLS?
> 
> /Fredrik
> 
>

---

From: Fredrik Thulin <[email-masked]>
Date: Tue, 05 Dec 2006 13:36:42 +0100

> I still fail to see all the "complexity" that UDP adds.

 From the top of my head,

   1) requiring STUN keepalives and de-muxing

   2) much more keepalives (every 29s instead of 120s)

   3) having to do resends yourself (not specific to Outbound)

   4) being insufficient in itself (having limitation of request size),
      therefor not adding any value when you _still_ have to do TCP/TLS
      as well as UDP

/Fredrik

---

From: <[email-masked]>
Date: Tue, 5 Dec 2006 16:33:23 +0200

>products that no service provider cares for.

Can you pinpont exactly the real life deployment problems you mention
with using TCP? Does it have something to do with server-side
scalability?

I personally believe going TCP-only would actually help deployment
rather than hinder it, but I'd be really interested to be convinced
otherwise.

Cheers,
Aki

---

From: [email-masked]
Date: Tue, 5 Dec 2006 10:28:32 -0500

From: Hisham Khartabil <[email-masked]>

Why?  I think adding a little more detail to this statement would
help.  Stated this plainly, my mind appends "... because they are
petulant".

But if service providers are avoiding TCP because there is some
genuine burden associated with using TCP, that must be expressible as
a requirement, one which may well have been overlooked.  So expressing
the requirement in as much detail as possible will help us progress.

Not knowing much about the subject, my guess would be that TCP has
overheads, in that one must keep significant state about each
connection and perhaps send keepalive packets periodically.  But
outbound seems to incorporate those already; I don't see how TCP could
be more burdensome than UDP-with-outbound.

Dale

---

From: Dean Willis <[email-masked]>
Date: Tue, 5 Dec 2006 10:03:03 -0600

> be more burdensome than UDP-with-outbound.

The simplest answer, which we've discussed here previously, is that a  
naive implementation of a proxy using TCP is limited by the number of  
sockets provided by the OS, which is usually something less than the  
number of file handles and tends to be less than 64k. This caps the  
"fan out" of a proxy" at a number smaller than some operators would  
like. Of course, it's possible to write a more sophisticated proxy  
using select/poll that doesn't have this limitation, but only on OS  
that support it.

There's another answer occasionally used that is far less  
supportable: many people in the mobile community have historically  
believed that TCP is less suitable than UDP for mobile networks,  
because the reliability and timing properties of those networks are  
different. These networks also don't use NAT or "outbound" per se,  
and therefore generally do not do keepalives at the IP level. Given  
this, UDP implementations are more air-efficient than naive TCP  
implementations because of TCP keepalive and retransmission. Of  
course, one can make a slightly smarter TCP implementation that does  
very little TCP keepalive, so once again we've learned how to mostly  
overcome this objection.

One other thing that UDP makes easier is fault tolerance across a  
proxy farm with replicated state. Of course, we've also learned how  
to do this with TCP (and even with TLS) at reasonable levels of  
efficiency.

In other words, to the best of my knowledge, the only good reason to  
keep using UDP is because you have it deployed already and don't want  
to change, because change is expensive and disruptive. Of course, if  
one is going to phase in "outbound", one is going to have to make  
similar changes.

We should keep this in mind -- Change isn't mandatory. Neither is  
survival. But if you're unwilling to do the first, you're unlikely to  
achieve the second.

--
Dean

---

From: Rohan Mahy <[email-masked]>
Date: Tue, 5 Dec 2006 08:30:12 -0800

Keith,

I don't believe Hisham was objecting here to MUST implement.  I think 
Hisham is objecting to adding a MUST use statement to outbound.

thanks,
-rohan

---

From: Rohan Mahy <[email-masked]>
Date: Tue, 5 Dec 2006 08:30:12 -0800

Keith,

thanks,
-rohan

---

From: Hisham Khartabil <[email-masked]>
Date: Tue, 5 Dec 2006 20:22:10 +0100

> rather than hinder it, but I'd be really interested to be convinced

I could ask you the converse. I'd really be interested to be convinced 
how TCP deployment improves scalability.

Hisham

---

From: [email-masked]
Date: Tue, 5 Dec 2006 14:28:12 -0500

From: Dean Willis <[email-masked]>

   In other words, to the best of my knowledge, the only good reason to  
   keep using UDP is because you have it deployed already and don't want  
   to change, because change is expensive and disruptive.

Which is actually a legitimate requirement, if the amount of
money/trouble is economically significant in an industry.

   Of course, if one is going to phase in "outbound", one is going to
   have to make similar changes.

It seems to me that that is the bottom line -- installing
UDP-with-outbound-keepalives is going to be nearly as much work as
installing TCP, so better to use TCP, which has better long-term
prospects.

I would, however, like to hear an analysis from someone who is
forseeing trouble from TCP.

Dale

---

From: Hisham Khartabil <[email-masked]>
Date: Tue, 5 Dec 2006 20:34:31 +0100

Dale,

Besides scalability issues, there are no technical issues. The issues 
are with having to throw all the infrastructure we currently have out 
of the window and start all over again. Having 110k+ customers live 
does not make is an easy and justifiable task.

Why do we prefer to run away from the problem and ignore it pretending 
it does not "really" exist instead of trying to fix it?

Hisham

---

From: Fredrik Thulin <[email-masked]>
Date: Tue, 05 Dec 2006 21:23:55 +0100

> On Dec 5, 2006, at 3:33 PM, <[email-masked]> wrote:
...
> how TCP deployment improves scalability.

To name two things where TCP would provide better scalability than UDP 
for a SIP service provider :

   1) Say that your services maximum capacity for non-INVITEs is 1000
      requests per second. If you get 2000 in a single second, you won't
      only have to deal with all those 2000 requests, but also with quite
      a few retransmissions sent after 500ms. With TCP, the wall isn't as
      dense.

   2) UDP sockets have a shared receive buffer, most commonly around 64k
      I believe (even though the maximum seems to be 256k on FreeBSD at
      least). With SIP messages of perhaps 700 bytes on average, you
      have to be able to move messages out of that buffer pretty fast to
      keep it from filling up. A full buffer means dropped packets. I've
      also heard that on for example Linux, every packet actually
      occupies an even 2k (or was it 4k) in the buffer, so 64k buffer is
      actually at most 32 packets at a time.

And then there are of course other aspects than scalability where UDP is 
an inferior solution, most notably security related things like for 
example spoofed CANCELs (which can't be challenged, RFC3261 #22.1).

/Fredrik

---

From: "Jeroen van Bemmel" <[email-masked]>
Date: Tue, 5 Dec 2006 22:44:16 +0100

Let's go back to the original topic - what should the draft say about the 
transport protocol in relationship with outbound

The major use case for outbound is UAs behind NAT / firewalls. Many of such 
devices have known issues with UDP fragmentation, so for many environments 
TCP will need to be used for messages larger than the path MTU (often ~1460 
bytes in ethernet environments, RFC3261 puts the limit at 1300).

If you start from the requirement "MUST only use UDP", then there is little 
room for constructive discussions. For UDP-only to work you will need to 
ensure that messages either stay below path MTUs, or disallow NAT. This 
implies significant limitations for your deployment environment. In any 
case, this is not a setup that outbound should focus on.

In the general case, there are environments that have to deal with NAT and 
messages larger than path MTU. This is what outbound was written for. This 
also implies a UA that supports TCP. Given that the UA supports TCP and the 
fact that presence of devices like stateful firewalls cannot reliably be 
detected, the UA SHOULD use TCP (or another connection oriented transport). 
If in addition the UA supports TLS, it SHOULD use TLS.

I think this is sufficient

Regards,
Jeroen

Fredrik Thulin wrote:
>> I could ask you the converse. I'd really be interested to be
>> convinced how TCP deployment improves scalability.
>
>      requests per second. If you get 2000 in a single second, you
>      won't only have to deal with all those 2000 requests, but also
>      with quite a few retransmissions sent after 500ms. With TCP, the
>      wall isn't as dense.
>
>   2) UDP sockets have a shared receive buffer, most commonly around
>      64k I believe (even though the maximum seems to be 256k on
>      FreeBSD at least). With SIP messages of perhaps 700 bytes on
>      average, you have to be able to move messages out of that buffer
>      pretty fast to keep it from filling up. A full buffer means
>      dropped packets. I've also heard that on for example Linux,
>      every packet actually occupies an even 2k (or was it 4k) in the
>      buffer, so 64k buffer is actually at most 32 packets at a time.
>
> And then there are of course other aspects than scalability where UDP
> is an inferior solution, most notably security related things like for

---

From: <[email-masked]>
Date: Tue, 5 Dec 2006 23:50:00 +0200

Hi,

On 05 Dec 2006, Dean Willis wrote: 
>The simplest answer, which we've discussed here previously, is 
>that a naive implementation of a proxy using TCP is limited by 
>the number of sockets provided by the OS, which is usually 
>something less than the number of file handles and tends to be 
>less than 64k. This caps the "fan out" of a proxy" at a number 
>smaller than some operators would like. Of course, it's 

but it's also good to notice that many/most modern operating systems
do not have these limitations. For instance, on most systems the 
64k limit you mention, only applies to outbound connections (due to the
available port range), and does not limit for instance how many incoming
TCP 
flows a SIP proxy can handle. I believe the unofficial record for a
single 
x86 box is still the 1.6M TCP connections (on FreeBSD, from 2003) [1],
but 
this of course is not the case in general. But tens of thousands of TCP 
connections is surely not a problem on any modern system.

And of course, it's true a naive implementation will have problems, but 
I can assure the limit on TCP connections is not the only place where
a naive implementation will hit the limits. ;)

There are numerous interesting papers and resources on the topic [2-4]
(and
note that most of these are quite old papers), and many indicate 
that TCP scalability is not _that_ big of an issue. Especially, in
the case of SIP, if we consider that UDP flows will require more
frequent 
keepalives, it is not obvious that a system using persistent TCP flows 
from the clients, will scale worse than a similar system utilizing UDP.

[1] http://kerneltrap.org/node/277 (search for "1.6 Million")
[2] http://www.kegel.com/c10k.html
[3]
http://bcr2.uwaterloo.ca/~brecht/papers/getpaper.php?file=ols-2004.pdf
[4] http://haproxy.1wt.eu/

>possible to write a more sophisticated proxy using select/poll 
>that doesn't have this limitation, but only on OS that support it.

Actually you shouldn't be using select/poll if you want to 
write a modern proxy -- see [2,3] above. ;) 

>One other thing that UDP makes easier is fault tolerance 
>across a proxy farm with replicated state. Of course, we've 
>also learned how to do this with TCP (and even with TLS) at 
>reasonable levels of efficiency.

Yes, agreed. This is a real concern, but also a problem that
has been solved in many existing TCP/TLS based services.

>In other words, to the best of my knowledge, the only good 
>reason to keep using UDP is because you have it deployed 
>already and don't want to change, because change is expensive 
>and disruptive. Of course, if one is going to phase in 

Yes, agreed. 

But it's also good to note that a big reason why UDP is so widely
used as the default transport, is the not-so-great state of standards
support 
for using TCP/TLS in SIP clients (how TCP/TLS flows are managed/reused
in clients,
how server elements can reuse these flows -> leading to need 
for ugly, but currently necessary, real-world hacks as described 
for instance in [5]). The sip-outbound draft may be useful for UDP, but 
it is absolutely critical for getting TCP/TLS more widely used. 

[5] sect 3.4.1 of
    http://tools.ietf.org/wg/sipping/draft-ietf-sipping-sbc-funcs/

-- 
[email-masked] (Kai Vehmanen)
Networking Technologies Laboratory, Nokia Research Center

---

From: "Jeroen van Bemmel" <[email-masked]>
Date: Tue, 5 Dec 2006 23:43:52 +0100

Not to mention the differences in transaction timers. For UDP transaction 
state (in particular non-INVITE server transactions) is retained much longer 
than for TCP. For the proxy-2-proxy case (i.e. small fan-in, few rx 
buffers ) TCP is often a better choice than UDP, especially when memory is 
your bottleneck.

Drawbacks for TCP are message boundary detection overhead and some byte 
copying overhead, which add some delay and some CPU load (though probably 
insignificant). Implementation complexity is slightly higher for TCP too.

Though interesting, IMHO all of this is not so relevant for the outbound 
discussion. Proxies must already implement TCP, the UA is better of using 
TCP if it can.

Regards,
Jeroen

[email-masked] wrote:
> 64k limit you mention, only applies to outbound connections (due to
> the available port range), and does not limit for instance how many
> incoming TCP
> this of course is not the case in general. But tens of thousands of
> TCP connections is surely not a problem on any modern system.
>
> And of course, it's true a naive implementation will have problems,
> but I can assure the limit on TCP connections is not the only place
> where
> for instance in [5]). The sip-outbound draft may be useful for UDP,
> but it is absolutely critical for getting TCP/TLS more widely used.
>
> [5] sect 3.4.1 of

---

From: "Francois Audet" <[email-masked]>
Date: Tue, 5 Dec 2006 17:09:43 -0600

Thanks for bringning back the original topic.

It seems to me that we do not have any evidence that when Outbound
keep-alive is
used because of the presence of a NAT, that there are any advantage to
using
UDP instead of TCP. It seems that all the perceived advantages of UDP
versus TCP
vanish in the presence of a NAT (in fact some have argued that TCP will
actually
scale better). Worst, all the evidence is that UDP will have 
problems with fragmentation and that it will therefore not be suitable.

Futhermore, if the most important use case for outbound keep-alive is
TCP 
and not UDP, "optimizing" the solution for UDP is not a good idea. I'd
rather
we have a solution optimized for TCP (and TLS). If we have a solution
for UDP or
not is of little interest to me: I just don't think it will be very
popular and it
will be just one more useless feature to implement.

Nit-pick: the other major use case for outbound is TLS support, but that
does
not require the keepalive. So what you really meant in the second
paragraph is
"The major use case for outbound keep-alive is ...".

> From: Jeroen van Bemmel [mailto:[email-masked]] 
> Sent: Tuesday, December 05, 2006 13:44
> To: Fredrik Thulin; Hisham Khartabil
> [email-masked]; [email-masked]; [email-masked]
> Let's go back to the original topic - what should the draft 
> say about the transport protocol in relationship with outbound
> 
> The major use case for outbound is UAs behind NAT / 
> firewalls. Many of such devices have known issues with UDP 
> fragmentation, so for many environments TCP will need to be 
> used for messages larger than the path MTU (often ~1460 bytes 
> in ethernet environments, RFC3261 puts the limit at 1300).
> 
> If you start from the requirement "MUST only use UDP", then 
> there is little room for constructive discussions. For 
> UDP-only to work you will need to ensure that messages either 
> stay below path MTUs, or disallow NAT. This implies 
> significant limitations for your deployment environment. In 
> any case, this is not a setup that outbound should focus on.
> 
> In the general case, there are environments that have to deal 
> with NAT and messages larger than path MTU. This is what 
> outbound was written for. This also implies a UA that 
> supports TCP. Given that the UA supports TCP and the fact 
> that presence of devices like stateful firewalls cannot 
> reliably be detected, the UA SHOULD use TCP (or another 
> connection oriented transport).

---

From: Dean Willis <[email-masked]>
Date: Tue, 5 Dec 2006 21:48:51 -0600

> convinced how TCP deployment improves scalability.

That's easy. Imagine that the Internet is a set of sewer pipes, down  
which someone periodically flushes whole cantaloupe. The cantaloupe  
have the side effect of blocking up other drains, causing "back up"  
problems in other people's plumbing fictures.

UDP fragmentation of large responses (or large messages, if one is so  
tacky as to send them that way) will cause bursts of non-flow- 
controlled traffic to be inserted into the network. This clogs up  
buffers, reduces efficient network flow, and generally makes SIP a  
bad network neighbor -- much like the moron upstairs who flushes the  
cantaloupe.

TCP reduces the congestion impact of large messages substantially.  
The net effect is like running the whole cantaloupe through a  
blender, flushing it a cup at a time, and not flushing the next cup  
until you've asked everyone in the building to make sure their  
plumbing is working.

Sure, we could re-invent all of the TCP flow-control using UDP, but  
why bother? We'd eventually just have TCP again.

Factor in the detail that we're well on the way towards having the  
average message large enough to throw at least one fragment (and  
possibly 3 or 4, especially on smaller MTU networks), and I think TCP- 
only SIP starts to become very attractive.

--
Dean

---

From: Sean Olson <[email-masked]>
Date: Tue, 5 Dec 2006 12:31:45 -0800

My two cents:  it's all about re-transmissions and security.  Would be curious to hear deployment experience around DTLS and whether or not that has actually helped the security of UDP deployments.

-----Original Message-----
From: Fredrik Thulin [mailto:[email-masked]]
Sent: Tuesday, December 05, 2006 12:24 PM
To: Hisham Khartabil
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Sip] Re: issue with oubound and the used transport protocol

...

/Fredrik

---

From: [email-masked]
Date: Wed, 6 Dec 2006 18:36:05 -0500

Besides scalability issues, there are no technical issues.

Naively, the scaling of UDP-with-outbound doesn't seem to be any
better than TCP-with-outbound.  Can you elucidate the scalability
issues that are causing you problems?

   The issues are with having to throw all the infrastructure we
   currently have out of the window and start all over again. Having
   110k+ customers live does not make is an easy and justifiable task.

Naively, to implement outbound will require changing the software in
the proxies and UAs, regardless of whether it uses UDP or TCP.  I
suppose this can be called "throwing infrastructure out of the
window", but it seems to be a routine matter.  Can you elucidate why
UDP-with-outbound reduces this burden compared to implementing
TCP-with-outbound?

   Why do we prefer to run away from the problem and ignore it
   pretending it does not "really" exist instead of trying to fix it?

Can you elucidate what "the problem" is that you are referring to?

Dale

---

From: Cary FitzGerald <[email-masked]>
Date: Wed, 6 Dec 2006 16:05:18 -0800 (PST)

> Sure, we could re-invent all of the TCP flow-control using UDP, but why 
> bother? We'd eventually just have TCP again.

More likely, SCTP.

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 6 Dec 2006 16:33:15 -0800

I find this whole conversation way off base. If people would like to  
deprecate UDP, go work on a document to explain that and feel free to  
bring it to the WG. In the mean time, SIP is specified over both UDP  
and TCP. Most deployments only use UDP, some only use TCP, and I  
suspect there are some that do both but I am not aware of them.  
Outbound as it is seems very consistent with 3261 in that it allows a  
system to use TCP or UDP. Keeping this in mind few comments inline ....

On Dec 5, 2006, at 1:44 PM, Jeroen van Bemmel wrote:

> Let's go back to the original topic - what should the draft say  
> about the transport protocol in relationship with outbound
>
> The major use case for outbound is UAs behind NAT / firewalls. Many  
> of such devices have known issues with UDP fragmentation, so for  
> many environments TCP will need to be used for messages larger than  
> the path MTU (often ~1460 bytes in ethernet environments, RFC3261  
> puts the limit at 1300).

These known issues with fragmentation might be less common than you  
think however, outbound does not have anything to do with MTU issues.  
It does not try and solve them and it does not make them worse.

>
> If you start from the requirement "MUST only use UDP", then there  
> is little room for constructive discussions. For UDP-only to work  
> you will need to ensure that messages either stay below path MTUs,  
> or disallow NAT. This implies significant limitations for your  
> deployment environment. In any case, this is not a setup that  
> outbound should focus on.

MTU is a problem regardless of NATs or Firewalls. Fragmented packets  
do manager to traverse most NATs and Firewalls.

>
> In the general case, there are environments that have to deal with  
> NAT and messages larger than path MTU. This is what outbound was  
> written for. This also implies a UA that supports TCP. Given that  
> the UA supports TCP and the fact that presence of devices like  
> stateful firewalls cannot reliably be detected, the UA SHOULD use  
> TCP (or another connection oriented transport). If in addition the  
> UA supports TLS, it SHOULD use TLS.

Again, this topic is not really about outbound. If you think a 3261  
UA that supports both UDP and TCP SHOULD use TCP, that is a fine  
argument to go make in the discussion about use of TCP and UDP but it  
seems to be an independent choice from anything to do with outbound.

Cullen <with my individual hat on>

---

From: "Francois Audet" <[email-masked]>
Date: Wed, 6 Dec 2006 19:12:14 -0600

I don't think this is naive at all. I think you hit the nail on the 
head very sqarely.

> From: [email-masked] [mailto:[email-masked]] 
> Sent: Wednesday, December 06, 2006 15:36
> To: [email-masked]
> Naively, the scaling of UDP-with-outbound doesn't seem to be 
> any better than TCP-with-outbound.  Can you elucidate the 
> scalability issues that are causing you problems?
> 
> Naively, to implement outbound will require changing the 
> software in the proxies and UAs, regardless of whether it 
> uses UDP or TCP.  I suppose this can be called "throwing 
> infrastructure out of the window", but it seems to be a 
> routine matter.  Can you elucidate why UDP-with-outbound 
> reduces this burden compared to implementing TCP-with-outbound?
>

---

From: [email-masked]
Date: Wed, 6 Dec 2006 20:43:09 -0500

From: "Francois Audet" <[email-masked]>

Well, I know I'm not up-to-date on all these issues.  I suspect that
Hisham knows of certain problems that I don't, but we need more
information about them to be able to discuss the situation usefully.

Dale

---

From: Rohan Mahy <[email-masked]>
Date: Wed, 6 Dec 2006 21:15:50 -0800

On Dec 6, 2006, at 16:33, Cullen Jennings wrote:
> and TCP.

To be more specific, SIP is currently specified so it can use UDP for 
requests smaller than 1300 bytes.  It is specified so it can use TCP 
for requests of any size.

Some people have suggested that user agents SHOULD be able to send and 
receive requests larger than 1300 bytes when using the outbound 
mechanism.  This seems like a logical requirement.  Are you objecting 
to this requirement Cullen?

For the sake of this discussion, lets assume that we have consensus on 
this requirement.  What are the implications?  Well, most UAs 
implementing outbound are doing so because they are behind a NAT or a 
firewall and will not be able to receive inbound TCP flows.  These UAs 
will need to establish a TCP flow if they want to meet this requirement 
of receiving requests larger than 1300 bytes. The logical normative 
statement is to say a UA which cannot (or suspects that it cannot) 
receive an inbound TCP flow from its outbound proxies SHOULD open at 
least one TCP or TLS flow to a server in its outbound-proxy-set.

A few people on this mailing list have suggested that for any user 
agent that actually bothers to open a TCP flow, there is no reason 
whatsoever to open a corresponding UDP flow. I wholeheartedly agree 
with them.

The problem is that this thread strikes right at the heart of our 
community's dilemma. We have a large body of people who have deployed 
UDP almost exclusively and effectively ignore the MUST use (and several 
even ignore the MUST implement) TCP statements in our core spec.  
Ironically, one of the problems which prevents realistic interoperable 
deployment of TCP-based SIP implementations is the lack of a 
standardized mechanism for doing outbound.  [Dean also identified some 
other problems often faced when deploying TCP with programs that use 
"obvious" (Dean actually said "naive") textbook implementation 
techniques.]

I think that a large number of the core spec contributors and seasoned 
implementors now recognize that there are many non-academic problems 
with SIP over UDP and that some day we will want to either deprecate or 
at least seriously curtail the use of UDP.  Like a very heavy drinker 
we know it will be good for us to kick the habit.  We are not quitting 
today. The question before us is are we going to do something in 
outbound that makes it harder for us to quit in the future.

My concrete proposal is the following:
- make CRLF the only keepalive mechanism for TCP and TLS over TCP flow 
(retain STUN for UDP flows)
- make outbound registrations over UDP flows optional for UAs
- include the statement that its is RECOMMENDED that the 
outbound-proxy-set is configured to result in registrations over TCP or 
TLS whenever the UA can't accept incoming TCP flows (the 3GPP 
exception).  explain that for historical purposes a lot of deployments 
still only use UDP, but that we expect/hope those deployments to 
eventually migrate to TCP when doing outbound.

What's in it for the TCP crowd?  They get to implement outbound with no 
UDP if they so desire.  They don't have to implement STUN over TCP. The 
draft clearly explains why TCP is the technically preferred solution.

What's in it for the UDP crowd? They get outbound behavior (at least 
for requests <1300 bytes) specified in the core draft for UDP.  They 
can keep doing what they are doing, but with STUN keepalives if the UA 
and edge proxy both support it.

What's not to like?
thanks,
-rohan

> Most deployments only use UDP, some only use TCP, and I suspect there 
> are some that do both but I am not aware of them. Outbound as it is 
> seems very consistent with 3261 in that it allows a system to use TCP 
> or UDP. Keeping this in mind few comments inline ....
>
>> Let's go back to the original topic - what should the draft say about 
>> the transport protocol in relationship with outbound
>>
>> of such devices have known issues with UDP fragmentation, so for many 
>> environments TCP will need to be used for messages larger than the 
>> path MTU (often ~1460 bytes in ethernet environments, RFC3261 puts 
>> the limit at 1300).
>
>> If you start from the requirement "MUST only use UDP", then there is 
>> little room for constructive discussions. For UDP-only to work you 
>> will need to ensure that messages either stay below path MTUs, or 
>> disallow NAT. This implies significant limitations for your 
>> written for. This also implies a UA that supports TCP. Given that the 
>> UA supports TCP and the fact that presence of devices like stateful 
>> firewalls cannot reliably be detected, the UA SHOULD use TCP (or 
>> another connection oriented transport). If in addition the UA 
>> supports TLS, it SHOULD use TLS.
>
> Again, this topic is not really about outbound. If you think a 3261 UA 
> that supports both UDP and TCP SHOULD use TCP, that is a fine argument 
> to go make in the discussion about use of TCP and UDP but it seems to 
> be an independent choice from anything to do with outbound.
>
>

---

From: <[email-masked]>
Date: Thu, 7 Dec 2006 10:37:05 +0200

>convinced how TCP deployment improves scalability.

That's not what I said. I said using TCP helps *deployment*, namely
gives more robust NAT/FW traversal, enables security and just generally
reduces complexity (e.g., makes transport-switching unnecessary).

My thesis here is that going TCP-only makes deployment easier without
incurring any noticeable performance penalty regarding server-side
scalability. And we've done some experiments that show this to be
accurate.

Cheers,
Aki

---

From: "Aki Niemi" <[email-masked]>
Date: Thu, 7 Dec 2006 14:46:50 +0200

>that doesn't have this limitation, but only on OS that support it.

Let me also chime in here a bit. 

We've done experiments on Linux using a proxy server built on top of our
sofia-sip stack, and have noticed no significant slowdown on an ordinary
desktop PC handling 100s of thousands of registrations simultaneously using
TCP. The stack's Linux implementation uses epoll, which is even more
efficient than select() or poll(). Now, this is not very portable, although
other OSs have similar schemes, like kqueue in BSD. 

Also, if one wanted to write their proxy in Python, the numbers would
probably look a lot different; then again high performance probably isn't
what one is looking for with Python in the first place. Similarly, the
performance of a proxy can suck due to numerous other reasons as well, e.g.,
a frequent dip in a hugely expensive database or RPC backend. We didn't
really touch up on those issues in our experiments, but I suspect many of
these other issues have fairly easy (and expensive) solutions to them. :)

So once and for all, I'd really like to bust the myth, break the urban
legend, and stop the folklore that using TCP wrecks proxy performance. On
modern OSs it really *does not*.

Cheers,
Aki

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Thu, 7 Dec 2006 15:59:17 +0100

Hi, 

>>deprecate UDP, go work on a document to explain that and 
>>feel free to bring it to the WG. In the mean time, SIP is specified
over 
>>both UDP and TCP.
> 
>To be more specific, SIP is currently specified so it can use 
>UDP for requests smaller than 1300 bytes.  It is specified so 
>it can use TCP for requests of any size.
> 
>Some people have suggested that user agents SHOULD be able to 
>send and receive requests larger than 1300 bytes when using 
>the outbound mechanism.  This seems like a logical 
>requirement.  Are you objecting to this requirement Cullen?

Isn't that an implementation issue?

What if a client "knows" that it will never send or receive requests
larger than 1300 bytes?

>For the sake of this discussion, lets assume that we have 
>consensus on this requirement.  What are the implications?  
>Well, most UAs implementing outbound are doing so because 
>they are behind a NAT or a firewall and will not be able to 
>receive inbound TCP flows. These UAs will need to establish 
>a TCP flow if they want to meet this requirement of receiving 
>requests larger than 1300 bytes. The logical normative 
>statement is to say a UA which cannot (or suspects that it 
>cannot) receive an inbound TCP flow from its outbound proxies 
>SHOULD open at least one TCP or TLS flow to a server in its 
>outbound-proxy-set.

I think it's quite clear that a client located behdind a NAT/Firewall,
wanting to be able to receive inbound TCP flows, should take actions to
allow that. But, do we need to say more than that?
 
>A few people on this mailing list have suggested that for any 
>user agent that actually bothers to open a TCP flow, there is 
>no reason whatsoever to open a corresponding UDP flow. I 
>wholeheartedly agree with them.

That's probably true. But, again, is that something we need to specify,
or is it an implementation issue?
 
>The problem is that this thread strikes right at the heart of 
>our community's dilemma. We have a large body of people who 
>have deployed UDP almost exclusively and effectively ignore 
>the MUST use (and several even ignore the MUST implement) TCP 
>statements in our core spec.  
>Ironically, one of the problems which prevents realistic 
>interoperable deployment of TCP-based SIP implementations is 
>the lack of a standardized mechanism for doing outbound.  
>[Dean also identified some other problems often faced when 
>deploying TCP with programs that use "obvious" (Dean actually 
>said "naive") textbook implementation techniques.]
> 
>I think that a large number of the core spec contributors and 
>seasoned implementors now recognize that there are many 
>non-academic problems with SIP over UDP and that some day we 
>will want to either deprecate or at least seriously curtail 
>the use of UDP.  Like a very heavy drinker we know it will be 
>good for us to kick the habit.  We are not quitting today. 
>The question before us is are we going to do something in 

First people wanted to use outbound to "promote" STUN, now to get rid of
UDP. Is that really what we are supposed to be doing?

I still fail to see what the technical problems with outbound and STUN
is. It may be more complex than TCP, but are there really any open
issues to be solved?
 
>- make CRLF the only keepalive mechanism for TCP and TLS over 
>TCP flow (retain STUN for UDP flows)
>outbound-proxy-set is configured to result in registrations 
>over TCP or TLS whenever the UA can't accept incoming TCP 
>flows (the 3GPP exception).  explain that for historical 
>purposes a lot of deployments still only use UDP, but that we 
>expect/hope those deployments to eventually migrate to TCP 
>when doing outbound.

I don't understand why a standards spec shall talk about what we hope
and expect... As Cullen said, UDP is a valid transport protocol for SIP,
so I don't understand why we need to "justify" anything...
 
>What's in it for the TCP crowd?  They get to implement 
>outbound with no UDP if they so desire.  They don't have to 
>implement STUN over TCP. The draft clearly explains why TCP 
>is the technically preferred solution.
> 
>What's in it for the UDP crowd? They get outbound behavior 
>(at least for requests <1300 bytes) specified in the core 
>draft for UDP.  They can keep doing what they are doing, but 
>with STUN keepalives if the UA and edge proxy both support it.

Just to clarify: the edge proxy is still required to implement STUN, in
case someone establishes UDP flows with it, right?

Regards,

Christer

> firewalls. Many 
> >>>>> rather than hinder it, but I'd be really interested to be 
> >>>>> convinced otherwise.
> >>>>
> >>> To name two things where TCP would provide better 
> scalability than 
> >>> UDP for a SIP service provider :
> >>>
> >>> And then there are of course other aspects than scalability where 
> >>> UDP is an inferior solution, most notably security related things 
> >>> like for example spoofed CANCELs (which can't be 
> challenged, RFC3261 
> >>> #22.1).
> >>>
> >>> /Fredrik
> >>>
> >>>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Thu, 07 Dec 2006 09:17:50 -0600

> other OSs have similar schemes, like kqueue in BSD. 

You can actually hide all of this behind canonical APIs by using
libevent (http://monkey.org/~provos/libevent/).  Your code is
extremely portable then, unless of course, your target is a
RTOS system of some sort where libevent is not compilable (and I
know a couple of the RTOS where this is the case).

Maybe this thread should now migrate to sip-implementors ;-)

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
2701 Lucent Lane, Rm. 9F-546, Lisle, Illinois 60532 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
URL:   http://www.alcatel-lucent.com/bell-labs

---

From: Rohan Mahy <[email-masked]>
Date: Thu, 7 Dec 2006 09:01:26 -0800

> Isn't that an implementation issue?

No, it is an interoperability issue which is consistent with our core 
spec.

> What if a client "knows" that it will never send or receive requests

For most UAs the client will never be able to "know" that.  In fact 
most UAs will be reasonably sure that they WILL be getting requests 
larger than this size.  The only reason this is not a MUST is that we 
have some environments (like 3GPP) that are trying hard to keep 
requests under MTU size and where requests larger than this size are 
rare.

> allow that. But, do we need to say more than that?

For most UAs we do.  Therefore a SHOULD is appropriate.

> or is it an implementation issue?

We have normative language in our specifications that recommend all 
sorts of things that are far less important and significant than this.

> UDP. Is that really what we are supposed to be doing?

No, we already have several good reasons to get rid of UDP in the 
future.  What I am proposing is that we don't make it harder for us to 
do so when the time comes.

> so I don't understand why we need to "justify" anything...

If we have a SHOULD, we need to justify why someone might not do the 
SHOULD.  In some ways the bar is even higher here since we are 
justifying why this wasn't a MUST.

> case someone establishes UDP flows with it, right?

That would be part of my plan.

thanks,
-rohan

---

From: Michael Thomas <[email-masked]>
Date: Thu, 07 Dec 2006 09:25:00 -0800

> requests under MTU size and where requests larger than this size are 

And the minute that they interconnect to something that hasn't been
trying, the assumption gets blown clean out of the water, right? That
is the nature of an internet, right? I guess I don't see the point of
special casing something for the range of the moment. It seems like
the SHOULD isn't really a SHOULD at all because we know that
those assumptions will eventually bite because the SIP wg sure isn't
putting any energy in making that true for the Internet at large.

       Mike

---

From: "Francois Audet" <[email-masked]>
Date: Thu, 7 Dec 2006 11:43:23 -0600

I really like this proposal and support it.

> From: Rohan Mahy [mailto:[email-masked]] 
> Sent: Wednesday, December 06, 2006 21:16
> To: Cullen Jennings
> Cc: Rohan Mahy; Fredrik Thulin; [email-masked];

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Dec 2006 10:22:38 -0800

Dear chairs,

Rohan has always disagreed with our TCP keep alive mechanism. He has  
brought this up over many meetings. IT has been discussed many times  
and we have taken WG hums on it. This whole argument that he is  
creating is centered around trying to kill the way TCP keep alive are  
done. He is not alone, others agree with him. The basic argument  
against it is it is a pain to implement - I've implemented and did  
not find it a big deal but agree it is more complicated that  
implementing the Double CRLF approach. We did have reasonable good  
consensus for it in the last WG hum we took on it.

I would like the chairs to make a statement about if they want reopen  
the issue of how to do TCP Keep alives. If yes, I'm glad to present  
on it at next IETF, if not, I want to know that I can just ignore it.

Cullen (with my outbound editor hat on)

On Dec 6, 2006, at 9:15 PM, Rohan Mahy wrote:

>> I find this whole conversation way off base. If people would like  
>> to deprecate UDP, go work on a document to explain that and feel  
>> free to bring it to the WG. In the mean time, SIP is specified  
>> over both UDP and TCP.
>
> To be more specific, SIP is currently specified so it can use UDP  
> for requests smaller than 1300 bytes.  It is specified so it can  
> use TCP for requests of any size.
>
> Some people have suggested that user agents SHOULD be able to send  
> and receive requests larger than 1300 bytes when using the outbound  
> mechanism.  This seems like a logical requirement.  Are you  
> objecting to this requirement Cullen?
yes.

>
> For the sake of this discussion, lets assume that we have consensus  
> on this requirement.  What are the implications?  Well, most UAs  
> implementing outbound are doing so because they are behind a NAT or  
> a firewall and will not be able to receive inbound TCP flows.   
> These UAs will need to establish a TCP flow if they want to meet  
> this requirement of receiving requests larger than 1300 bytes. The  
> logical normative statement is to say a UA which cannot (or  
> suspects that it cannot) receive an inbound TCP flow from its  
> outbound proxies SHOULD open at least one TCP or TLS flow to a  
> server in its outbound-proxy-set.
>
> community's dilemma. We have a large body of people who have  
> deployed UDP almost exclusively and effectively ignore the MUST use  
> (and several even ignore the MUST implement) TCP statements in our  
> core spec.  Ironically, one of the problems which prevents  
> realistic interoperable deployment of TCP-based SIP implementations  
> is the lack of a standardized mechanism for doing outbound.  [Dean  
> also identified some other problems often faced when deploying TCP  
> with programs that use "obvious" (Dean actually said "naive")  
> textbook implementation techniques.]
>
> seasoned implementors now recognize that there are many non- 
> academic problems with SIP over UDP and that some day we will want  
> to either deprecate or at least seriously curtail the use of UDP.   
> Like a very heavy drinker we know it will be good for us to kick  
> the habit.  We are not quitting today. The question before us is  
> are we going to do something in outbound that makes it harder for  
> us to quit in the future.
>
> - make CRLF the only keepalive mechanism for TCP and TLS over TCP  
> flow (retain STUN for UDP flows)
> - include the statement that its is RECOMMENDED that the outbound- 
> proxy-set is configured to result in registrations over TCP or TLS  
> whenever the UA can't accept incoming TCP flows (the 3GPP  
> exception).  explain that for historical purposes a lot of  
> deployments still only use UDP, but that we expect/hope those  
> deployments to eventually migrate to TCP when doing outbound.
>
> What's in it for the TCP crowd?  They get to implement outbound  
> with no UDP if they so desire.  They don't have to implement STUN  
> over TCP. The draft clearly explains why TCP is the technically  
> preferred solution.
>
> What's in it for the UDP crowd? They get outbound behavior (at  
> least for requests <1300 bytes) specified in the core draft for  
> UDP.  They can keep doing what they are doing, but with STUN  
> keepalives if the UA and edge proxy both support it.
>
> What's not to like?
> thanks,
> -rohan
>
>
>> Most deployments only use UDP, some only use TCP, and I suspect  
>> there are some that do both but I am not aware of them. Outbound  
>> as it is seems very consistent with 3261 in that it allows a  
>>> The major use case for outbound is UAs behind NAT / firewalls.  
>>> Many of such devices have known issues with UDP fragmentation, so  
>>> for many environments TCP will need to be used for messages  
>>> larger than the path MTU (often ~1460 bytes in ethernet  
>>> environments, RFC3261 puts the limit at 1300).
>>
>> These known issues with fragmentation might be less common than  
>> you think however, outbound does not have anything to do with MTU  
>> issues. It does not try and solve them and it does not make them  
>> worse.
>>
>>>
>>> you will need to ensure that messages either stay below path  
>>> MTUs, or disallow NAT. This implies significant limitations for  
>>> your deployment environment. In any case, this is not a setup  
>>> that outbound should focus on.
>>
>> MTU is a problem regardless of NATs or Firewalls. Fragmented  
>> packets do manager to traverse most NATs and Firewalls.
>>
>>>
>>> with NAT and messages larger than path MTU. This is what outbound  
>>> was written for. This also implies a UA that supports TCP. Given  
>>> that the UA supports TCP and the fact that presence of devices  
>>> like stateful firewalls cannot reliably be detected, the UA  
>>> SHOULD use TCP (or another connection oriented transport). If in  
>>> addition the UA supports TLS, it SHOULD use TLS.
>>
>> Again, this topic is not really about outbound. If you think a  
>> 3261 UA that supports both UDP and TCP SHOULD use TCP, that is a  
>> fine argument to go make in the discussion about use of TCP and  
>> UDP but it seems to be an independent choice from anything to do  
>> with outbound.
>>
>>
>>>>>> convinced

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Dec 2006 10:22:23 -0800

> use TCP for requests of any size.

Agree with above. Note that you said SIP specified a way to do this  
not that deployments must use this.

> objecting to this requirement Cullen?

yes - I object to this. I think the specification should specify how  
things work for large requests but that an UA should be allowed to  
choose to use only UDP if it wants to accept the limitations that  
this implies.

Again, I don't see why you are trying to tie up deprecating UDP with  
the outbound draft.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Dec 2006 10:22:38 -0800

Dear chairs,

yes.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Dec 2006 10:22:23 -0800

---

From: "Rohan Mahy" <[email-masked]>
Date: Thu, 7 Dec 2006 13:57:55 -0800

Hi,

- The argument for using STUN over TCP was "for consistency with UDP".
 However the code necessary to demux STUN and SIP over TCP is pretty
much orthogonal to the code to demux STUN and SIP over UDP.
- In addition to being simpler, CRLF over TCP is backwards compatible
with RFC 3261 implementations.

I believe the WG made the decision it did based on the erroneous
assumption that STUN over TCP would be easier since implementations
would have to implement STUN for UDP anyway.  I would like to reopen
this decision because I believe it was based on faulty data.  In
addition, if we allow UAs to only implement outbound over TCP, this
woudl be yet another assumption that would no longer hold true.

thanks,
-rohan

---

From: "Rohan Mahy" <[email-masked]>
Date: Thu, 7 Dec 2006 14:04:14 -0800

Hi Cullen,

Please read the following definition carefully and then tell me why
you think my proposed statement ("user agents SHOULD be able to send
and receive requests larger than 1300 bytes when using the outbound
mechanism") is inappropriate.

>From RFC 2119:
>3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
>   may exist valid reasons in particular circumstances to ignore a
>   particular item, but the full implications must be understood and
>   carefully weighed before choosing a different course.

thanks,
-rohan

---

From: "Jeroen van Bemmel" <[email-masked]>
Date: Thu, 7 Dec 2006 23:25:57 +0100

hum. This is not about deprecating UDP, it's about avoiding unnecessary 
complexity for TCP (and TLS)

Regards,
Jeroen

---

From: [email-masked] (Juha Heinanen)
Date: Fri, 8 Dec 2006 07:33:23 +0200

i support rohan's proposal to use CRLF over TCP.

-- juha

---

From: Hisham Khartabil <[email-masked]>
Date: Fri, 8 Dec 2006 14:48:46 +0100

I like this proposal and support it.

Hisham

On Dec 7, 2006, at 6:15 AM, Rohan Mahy wrote:

> will need to establish a TCP flow if they want to meet this 
> requirement of receiving requests larger than 1300 bytes. The logical 
> normative statement is to say a UA which cannot (or suspects that it 
> cannot) receive an inbound TCP flow from its outbound proxies SHOULD 
> open at least one TCP or TLS flow to a server in its 
> outbound-proxy-set.
>
> UDP almost exclusively and effectively ignore the MUST use (and 
> several even ignore the MUST implement) TCP statements in our core 
> spec.  Ironically, one of the problems which prevents realistic 
> interoperable deployment of TCP-based SIP implementations is the lack 
> of a standardized mechanism for doing outbound.  [Dean also identified 
> some other problems often faced when deploying TCP with programs that 
> use "obvious" (Dean actually said "naive") textbook implementation 
> techniques.]
>
> with SIP over UDP and that some day we will want to either deprecate 
> or at least seriously curtail the use of UDP.  Like a very heavy 
> drinker we know it will be good for us to kick the habit.  We are not 
> quitting today. The question before us is are we going to do something 
> in outbound that makes it harder for us to quit in the future.
>
> outbound-proxy-set is configured to result in registrations over TCP 
> or TLS whenever the UA can't accept incoming TCP flows (the 3GPP 
> What's in it for the TCP crowd?  They get to implement outbound with 
> no UDP if they so desire.  They don't have to implement STUN over TCP. 
> The draft clearly explains why TCP is the technically preferred 
> solution.
>
>>>> UDP

---

From: Hisham Khartabil <[email-masked]>
Date: Fri, 8 Dec 2006 14:49:42 +0100

On Dec 7, 2006, at 7:22 PM, Cullen Jennings wrote:

> against it is it is a pain to implement - I've implemented and did not 
> find it a big deal but agree it is more complicated that implementing 
> the Double CRLF approach. We did have reasonable good consensus for it 
> in the last WG hum we took on it.
>
> the issue of how to do TCP Keep alives. If yes, I'm glad to present on 
> it at next IETF

Please, lets not wait until March to discuss this. We can take a hum 
here on the mailing list. My vote for TCP keepalive is CRLF.

Hisham

> , if not, I want to know that I can just ignore it.
>
>>> deprecate UDP, go work on a document to explain that and feel free 
>>> to bring it to the WG. In the mean time, SIP is specified over both 
>>> UDP and TCP.
>>
>> firewall and will not be able to receive inbound TCP flows.  These 
>> UAs will need to establish a TCP flow if they want to meet this 
>> of a standardized mechanism for doing outbound.  [Dean also 
>> identified some other problems often faced when deploying TCP with 
>> programs that use "obvious" (Dean actually said "naive") textbook 
>> implementation techniques.]
>>
>> seasoned implementors now recognize that there are many non-academic 
>> problems with SIP over UDP and that some day we will want to either 
>> deprecate or at least seriously curtail the use of UDP.  Like a very 
>> heavy drinker we know it will be good for us to kick the habit.  We 
>> are not quitting today. The question before us is are we going to do 
>> something in outbound that makes it harder for us to quit in the 
>> future.
>>
>> no UDP if they so desire.  They don't have to implement STUN over 
>> TCP. The draft clearly explains why TCP is the technically preferred 
>> solution.
>>
>> can keep doing what they are doing, but with STUN keepalives if the 
>> UA and edge proxy both support it.
>>
>> What's not to like?
>> thanks,
>> -rohan
>>
>>
>>> there are some that do both but I am not aware of them. Outbound as 
>>> it is seems very consistent with 3261 in that it allows a system to 
>>> use TCP or UDP. Keeping this in mind few comments inline ....
>>>
>>> think however, outbound does not have anything to do with MTU 
>>> argument to go make in the discussion about use of TCP and UDP but 
>>> it seems to be an independent choice from anything to do with 
>>> outbound.
>>>
>>>

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Fri, 8 Dec 2006 14:57:50 +0100

Hi,

A couple of questions:

1. What was the reason(s) the CRLF solution is not suitable for UDP?

2. Even for UDP it can be possible to use lower layer keep-alive
mechanisms, e.g. when UDP encapsulated IPSec, is used, so I think one
should be allowed to use that instead of STUN. 

Regards,

Christer

> Sent: 8. joulukuuta 2006 15:49
> To: Rohan Mahy
> Cc: SIP WG
> feel free to 
> consensus on 
> identified 
> over TCP 
> deployments 
> outbound with 
> (at least 
> >>>>>> I personally believe going TCP-only would actually help 
> >>>>>> deployment rather than hinder it, but I'd be really 
> interested to 
> >>>>>> be convinced otherwise.
> >>>>>
> >>>>      with quite a few retransmissions sent after 500ms. 
> With TCP, 
> >>>> the
> >>>>      wall isn't as dense.
> >>>>
> >>>> like for example spoofed CANCELs (which can't be challenged, 
> >>>> RFC3261 #22.1).
> >>>>
> >>>> /Fredrik
> >>>>
> >>>>

---

From: Rohan Mahy <[email-masked]>
Date: Fri, 8 Dec 2006 07:33:39 -0800

Hi,

Well in this case, the compression they are using still makes it rare 
to receive > 1300 byte requests over the last hop even when they get 
requests from something in the public Internet that is substantially 
larger. Even over TCP it is still acceptable for a proxy or UA to 
reject a request that is too big.

thanks,
-rohan

---

From: Rohan Mahy <[email-masked]>
Date: Fri, 8 Dec 2006 07:40:06 -0800

> 1. What was the reason(s) the CRLF solution is not suitable for UDP?

You can't detect that a NAT has rebooted and your binding has changed 
over UDP unless your message contains the reflected address.

> should be allowed to use that instead of STUN.

There is not MUST USE for keepalive=stun, so I think this is tacitly 
permitted anyway.

thanks,
-rohan

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 8 Dec 2006 09:42:48 -0800

On Dec 8, 2006, at 5:49 AM, Hisham Khartabil wrote:

> Please, lets not wait until March to discuss this. We can take a  
> hum here on the mailing list

We had most of a room hum the other direction so, which did leave me  
thinking, assuming we don't get a single no on the the list, how many  
yes do think the chairs should be looking for to overturn this? I  
don't really want to know the answer to this as this is something  
that only the chairs need to figure out but it is an interesting  
question to think about when thinking about reopening this.

And just so people understand my position on this, if we reopen this  
issue, I will be voting for whichever path I think can get the draft  
to the IESG the soonest. I've implemented both and they are both  
trivial to implement. I really believe that what we need is to stop  
delaying this draft.

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 8 Dec 2006 09:44:29 -0800

On Dec 7, 2006, at 2:25 PM, Jeroen van Bemmel wrote:

> hum. This is not about deprecating UDP, it's about avoiding  
> unnecessary complexity for TCP (and TLS)

I can understand the argument that what keep alive we use in TCP is  
about avoiding unnecessary complexity for TCP and I'm fine with  
discussing that (thus my email to the chairs) however, Rohan's  
proposal contained other things that are separate issues. And how  
keep-alives work is very relevant to this draft. What protocols  
operators choose to use is an issue orthogonal to this draft (and a  
very complicated issue).

---

From: Byron Campen <[email-masked]>
Date: Fri, 8 Dec 2006 16:21:44 -0600

This looks sensible to me. But, I would like clarification on one  
point; are you proposing that server implementations of outbound  
(registrar, edge proxy) must (still) support outbound over UDP? Or  
would it be optional for these elements as well?

Best regards,
Byron Campen

---

From: "Francois Audet" <[email-masked]>
Date: Fri, 8 Dec 2006 18:31:12 -0600

Cullen,

My best guest on why this is happening is that people used to be
thinking more
about UDP than TCP and consequently expressed a preference for the STUN
approach.

It seems now that more people are paying attention to TCP because of a
bunch of issues
with UDP (including fragmentation, and TLS usage catching up since it
took us so long
to finalize this document). Many are reaching the conclusion that 
the CRLF approach makes more sense for TCP.

It seems that Rohan's last proposal of using CRLF for TCP, and STUN for
UDP to be
reasonable and logical. And it seems to have lots of support on the
list.

> From: Cullen Jennings [mailto:[email-masked]] 
> Sent: Friday, December 08, 2006 09:43
> To: Hisham Khartabil
> Cc: Rohan Mahy; Fredrik Thulin; SIP; Isomaki Markus 
> (Nokia-TP/Espoo); Peter Leis; Aki Niemi; Dean Willis; Keith Drage
> > here on the mailing list
> We had most of a room hum the other direction so, which did 
> leave me thinking, assuming we don't get a single no on the 
> the list, how many yes do think the chairs should be looking 
> for to overturn this? I don't really want to know the answer 
> to this as this is something that only the chairs need to 
> figure out but it is an interesting question to think about 
> when thinking about reopening this.
> 
> And just so people understand my position on this, if we 
> reopen this issue, I will be voting for whichever path I 
> think can get the draft to the IESG the soonest. I've 
> implemented both and they are both trivial to implement. I 
> really believe that what we need is to stop delaying this draft. 
>

---

From: Dean Willis <[email-masked]>
Date: Fri, 8 Dec 2006 20:01:31 -0600

On Dec 8, 2006, at 11:42 AM, Cullen Jennings wrote:

> We had most of a room hum the other direction so, which did leave  
> me thinking, assuming we don't get a single no on the the list, how  
> many yes do think the chairs should be looking for to overturn  
> this? I don't really want to know the answer to this as this is  
> something that only the chairs need to figure out but it is an  
> interesting question to think about when thinking about reopening  
> this.
>

As a chair, I'd be looking for "Yes" statements from a significant  
number of the implementors and folks who supported the STUN/TCP  
position in the previous hums. Continued calls for a revote that seem  
to have support only from the minority that supported CR/LF for TCP  
the first time around represent intransigence, not a change in  
working group position.

But hey, I'll Ask . . .

> And just so people understand my position on this, if we reopen  
> this issue, I will be voting for whichever path I think can get the  
> draft to the IESG the soonest. I've implemented both and they are  
> both trivial to implement. I really believe that what we need is to  
> stop delaying this draft.

Yea, verily!

--
Dean

---

From: [email-masked]
Date: Sat, 9 Dec 2006 00:38:44 -0500

From: Rohan Mahy <[email-masked]>

I suppose it's possible, but it seems to be strongly discouraged.  RFC
3261 says that an implementation must be willing to accept 64k-1 byte
UDP requests:

   18.1.1.  However, implementations MUST be able to handle messages
   up to the maximum datagram packet size.  For UDP, this size is
   65,535 bytes, including IP and UDP headers.

Dale

---

From: Rohan Mahy <[email-masked]>
Date: Fri, 8 Dec 2006 21:54:00 -0800

Edge Proxies and Registrars would still need to support outbound with 
UDP.

thanks,
-rohan

---

From: Fredrik Thulin <[email-masked]>
Date: Mon, 11 Dec 2006 09:29:30 +0100

Rohan Mahy wrote:
> Edge Proxies and Registrars would still need to support outbound with UDP.

Hi

Before answering Dean's poll, I would like to know more about this.

Have you figured out a (reasonable) way for the edge proxy/authoritative 
proxy (or the combination thereof) to switch from UDP to TCP for large 
messages destined for a UAC that uses Outbound?

I'm thinking of the problems in doing that that we discussed briefly in 
these two messages :

   http://www1.ietf.org/mail-archive/web/sip/current/msg17418.html
   http://www1.ietf.org/mail-archive/web/sip/current/msg17420.html

Thanks

/Fredrik

---

From: Fredrik Thulin <[email-masked]>
Date: Mon, 11 Dec 2006 09:29:30 +0100

Hi

Thanks

/Fredrik

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Mon, 11 Dec 2006 09:29:56 +0100

>My best guest on why this is happening is that people used to 
>be thinking more about UDP than TCP and consequently 
>expressed a preference for the STUN approach.

I think people thought more about STUN than either UDP or TCP ;)

Regards,

Christer

> (Nokia-TP/Espoo); 
> > Peter Leis; Aki Niemi; Dean Willis; Keith Drage
> leave me 
> > don't really want to know the answer to this as this is 
> something that 
> > only the chairs need to figure out but it is an interesting 
> question 
> > to think about when thinking about reopening this.
> > 
> reopen this

---

From: <[email-masked]>
Date: Mon, 11 Dec 2006 11:23:45 +0200

Hi,

I agree with Francois here. Even though there may be some performance
limitations for TCP in the current implementations, I think that almost
everyone agrees that it is the way to go in the long run. This is why it
would be good to have an optimal solution for Outbound/TCP, instead of
trying to keep consistency between UDP and TCP.

It would be great if we could do UAs supporting Outbound with TCP/TLS
only, and leave mess with message size, fragment handling, UDP-based
security, 20 second NAT keepalive timers etc. behind us. This may not be
realistic yet, if there are enough service providers who will demand UDP
(since unfortunately there are e.g. SBCs that do not even have a proper
TCP support at all), but that should be the aim.   

3GPP IMS networks, if such are deployed in the "puristic" form, may be a
different thing, but even there TCP is a better option. 

Markus

>From: ext Francois Audet [mailto:[email-masked]] 
>Sent: 09 December, 2006 02:31
>To: Cullen Jennings; Hisham Khartabil
>(Nokia-SIR/Espoo); Peter Leis; Niemi Aki (Nokia-NRC/Helsinki); 
>Dean Willis; Keith Drage
>Subject: RE: [Sip] Re: issue with oubound and the used 
>transport protocol
>
>Cullen,
>
>be thinking more about UDP than TCP and consequently expressed 
>a preference for the STUN approach.
>
>It seems now that more people are paying attention to TCP 
>because of a bunch of issues with UDP (including 
>fragmentation, and TLS usage catching up since it took us so 
>long to finalize this document). Many are reaching the 
>conclusion that the CRLF approach makes more sense for TCP.
>
>It seems that Rohan's last proposal of using CRLF for TCP, and 
>STUN for UDP to be reasonable and logical. And it seems to 
>have lots of support on the list.
>
>
>> only the chairs need to figure out but it is an interesting question

---

From: Rohan Mahy <[email-masked]>
Date: Mon, 11 Dec 2006 08:41:33 -0800

Hi Fredrik,

In the 3GPP environment I believe the specs would allow a IPsec 
protected TCP connection from the P-CSCF to the UA using a IPsec SA 
already established by a UDP flow.  If a NAT is present, I am told the 
IPsec traffic would be sent in UDP.  I am not convinced this TCP in UDP 
is an improvement and within the spirit of "sending over a congestion 
controlled transport", but I don't want to go down that rathole.

I don't think there is any desire by anyone to do switchover with 
outbound.  I'm not even convinced that anyone really uses it without 
outbound.  It appears that the folks who want to do TCP or TLS will 
only use that transport (no benefit to using UDP if you already have a 
TCP or TLS connection up), and I suspect that outside the 3GPP case the 
UDP folks are planning to only use UDP and use the "get out of jail 
free" statement in RFC 3261 that says if you tried to switched over and 
got an ICMP error, you go back to sending using UDP.

thanks,
-rohan

On Dec 11, 2006, at 0:29, Fredrik Thulin wrote:
> Have you figured out a (reasonable) way for the edge 
> proxy/authoritative proxy (or the combination thereof) to switch from 
> UDP to TCP for large messages destined for a UAC that uses Outbound?
>
> I'm thinking of the problems in doing that that we discussed briefly 
> in these two messages :
>