Subject: [Sip] draft-petithuguenin-sip-outbound-fragmentation-02

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 05 Mar 2007 05:50:05 -0800

I just submitted a new version of this I-D.  Until it appears in the
archives, a copy can be found here:

http://marc.petit-huguenin.org/draft-petithuguenin-sip-outbound-fragmentation-02.txt

Changelog:

 * Added text explaining that supporting UDP is important for
   subsequent requests (from an email from Cullen Jennings).
 * Added text to explain that the same IP address/port is used for the
   TCP connection.
 * Converted all 488 to 4XX.
 * Added text for usage of DTLS-TLS (It is in an annex, as Outbound
   does not say anything about DTLS).
 * Added text about delayed STUN transactions (Still waiting what will
   be in stunbis-06 about this issue).
 * Various nits.

-- 
Marc Petit-Huguenin           [                                 ]
Home: [email-masked] [RFC1855-compliant space for rent ]
Work: [email-masked]            [                                 ]

_______________________________________________
Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 12 Mar 2007 23:08:10 -0400

I'm with others that have commented earlier, I think this is way too 
complicated.

I'd rather have a single mechanism that is independent of outbound (say, 
by rejecting a request whose response would be too big, said response 
telling the client to try again via tcp), and then have that apply to 
sip-outbound just as well as anything else. The client would open a new 
tcp connection as a second flow, register over it, and then send its 
request.

What does the complexity of this draft add beyond that?

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Cisco Fellow                                   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 12 Mar 2007 20:18:08 -0700

> tcp connection as a second flow, register over it, and then send its

Thanks for your comments.

I am not sure to follow.  This draft is not about responses been too
big, it is about requests that cannot be sent to the endpoint, because
they are too big and the endpoint only opened an UDP "connection" to the
server.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 13 Mar 2007 12:30:16 -0400

Ah sorry, my bad. My fault for skimming to fast. I'll take another look 
and provide some useful comments....

-Jonathan R.

---

From: Dean Willis <[email-masked]>
Date: Tue, 13 Mar 2007 15:45:36 -0500

> and provide some useful comments....

It's still way too complicated. However, that level of complexity may be 
required due to the underlying complexity of SIP and multiple transport 
protocols and their inherent limitations.

But I still wish it were simpler.

Making it simpler, however, probably requires re-examining some of our 
more basic assumptions.

One approach would be to fall back to using UDP simply as a 
register-and-alert protocol.

In this model, user devices would "register" using UDP, then receive 
"imminent request" messages that would cause them to setup a TCP 
connection ala outbound. (Not too different from what Marc has now).

One could conceivably model the "imminent request" notification as an 
event package.

The difference here is that all traffic other than the registration, 
related registration-events, and "imminent request" subscription and 
notficication (the two event packages could be combined, I think) would 
go TCP. So there'd be no run-time sizing, Path-MTU discovery, or related 
complexity.

Bonus: We'd get to keep the high fan-out property of UDP edge proxies 
when we have "mostly idle" clients. Note that presence would sort of 
break this, of course.

--
dean

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 13 Mar 2007 16:50:13 -0400

>
> Bonus: We'd get to keep the high fan-out property of UDP edge  
> proxies when we have "mostly idle" clients. Note that presence  
> would sort of break this, of course.
>

I'm not sure fan-out is still such a big deal. We've run systems in  
our lab with 440,000 simultaneous TCP connections, with a stock Linux  
2.6 kernel, with some parameter tweaking (and a fair amount of RAM).

Henning

---

From: Dean Willis <[email-masked]>
Date: Tue, 13 Mar 2007 17:37:51 -0500

On Mar 13, 2007, at 3:50 PM, Henning Schulzrinne wrote:

> our lab with 440,000 simultaneous TCP connections, with a stock  
> Linux 2.6 kernel, with some parameter tweaking (and a fair amount  
> of RAM).

Well, my fall-back position is to deprecate UDP entirely from SIP.

OTOH, I've seen UDP proxies, written in java and running on stock, un- 
tweaked kernels, supporting millions of SIP registrations. Of course,  
not with "outbound" keeaalive and all that other traffic . . .

So I suspect that, from a practical perspective once we're talking  
the real world and not a rather rarified mobile singleapplication  
environment, that the scaling of TCP and UDP will be on the same order.

--
Dean

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Tue, 13 Mar 2007 16:47:02 -0700

Dean Willis wrote:
>> Ah sorry, my bad. My fault for skimming to fast. I'll take another
>> look and provide some useful comments....
> 

Yes, this can be a solution.  I choose a STUN extension because the
decision to switch or not over a TCP connection is done in the transport
layer where STUN is already used for keep-alive.

The problem is not in fact how to signal to the endpoint that it must
open a TCP connection - this can be done by a STUN message or by a SIP
message.  The problem is how to find the newly created TCP connection on
the proxy side; because the endpoint is behind a NAT and there can be
multiple endpoints behind this NAT, the source IP address can not be
used for this.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 14 Mar 2007 07:44:39 -0400

OK, I still think this is way too complicated.

My first thought is to just not solve it. We don't need to deprecate UDP 
because of this; we just need to have some tools to alert SIP 
administrators that this problem has happened. So, if an outbound proxy 
receives a request that is large, and it cannot deliver it to the UA 
over UDP, it rejects the request with a new resposne code that signals 
this case. Proxies can generate SNMP alarms or traps or whatever, and 
someone can go and buy new clients and proxies that use TCP. The alarm 
gives them the warning that the time has arrived to fix this problem.

A variant on this is to just send it over UDP anyway, and include a 
warning in the response that has the same alarming effect - there is a 
ticking time bomb here since fragmentation is going to be a problem. 
Note that even with fragmentation the request may never get delivered 
over UDP due to NAT handling of fragments; in that case the outbound 
proxy will timeout and include a warning about this case.

These two solutions at least recognize this error case but don't try to 
dynamically repair it.

An alternative to dynamically repair it is to use the reg-event package 
for the notification, since we already have that. Outbound proxy would 
reject the request, with this new response code. It would pass through 
the home proxy/registrar which will generate a notification to the 
client telling it to re-register over tcp. Registrar proxy holds onto 
the response until the new registration arrives. When it does, it can 
add a retry-after or actually just sequential fork and retry the request 
again. This approach is still complicated but doesn't introduce any 
fundamental new machinery.

I'm still inclined to just stick with a new response code. My rationale 
is that if your client can do tcp (which is required for the dynamic 
repair approach), may as well have it always connect via tcp. I don't 
buy this bit about wanting hybrid UDP TCP because of scale. The problem 
you've raised in your draft argues to me the opposite - folks whose tcp 
scale is a problem need to FIX it, since there is this issue with UDP 
outbound. The new response code gives them a warning that this is 
actually happening in their networks, and the time has come to switch on 
tcp.

Thanks,
Jonathan R.

---

From: <[email-masked]>
Date: Wed, 14 Mar 2007 14:27:04 +0200

Hi, 

>OK, I still think this is way too complicated.

I agree.

>administrators that this problem has happened. ....The alarm 

Jonathan, you must be kidding.

I find this kind of approach as horrible. Let the provider buy 
proxies and customers buy phones just to find out after some
time that they do not work and the customers are not able to
receive all the calls. Then start fixing it some way or another.
How many days/weeks would be needed for the customers to find
out that they have some random problems and stop using the service ?
I can not believe we in IETF would be happy to drive things like that.

>An alternative to dynamically repair it is to use the 
>reg-event package for the notification, since we already have that. 
>Outbound proxy would reject the request, with this new response code. 
>It would pass through the home proxy/registrar which will generate a 
>notification to the client telling it to re-register over tcp. 

The problem is that the size of the simplest reg-event notifications 
I have seen are close to 1000 bytes. The ultimate size will depend
on a number of things but there is a risk for the NOTIFY to become
bigger than MTU so that it can not be delivered over UDP. There is
a potential chicken and egg problem in this proposal.

>My rationale is that if your client can do tcp (which is required 
>for the dynamic  repair approach), may as well have it always connect 
>via tcp. I don't buy this bit about wanting hybrid UDP TCP because of 
>scale. The problem you've raised in your draft argues to me the
opposite 
>- folks whose tcp scale is a problem need to FIX it, since there is
this 
>issue with UDP outbound. 

I agree. Said that, it seems to me once again that the idea to 
keep UDP within the Outbound draft _is_ a bad idea.

Regards,

Erkki

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 14 Mar 2007 22:58:57 -0400

inline.

> Jonathan, you must be kidding.

I am not.

Listen, if folks are running a real SIP network and they are using 
outbound, and they elect to use UDP knowing that this problem can 
happen, then the consequence is that things won't work for large 
messages. Thats the consequence of making this choice, and we are 
providing them a way to know that it has actually happened.

I expect in reality, providers will know exactly how big each message 
will be since their clients are controlled by them and they have tested 
them with exactly the messages they will emit. So, this error code is 
really likely to show up in pre-deployment testing, at which point you 
either switch to tcp, or live with the risk.

> I can not believe we in IETF would be happy to drive things like that.

See above.

> a potential chicken and egg problem in this proposal.

Well, close to 1000 is still under the MTU by plenty, so it works fine.

> keep UDP within the Outbound draft _is_ a bad idea.

I'm trying to be pragmatic.

THe *reality* is that there IS still a lot of UDP out there. And IETF 
producing specifications that ignore this is just ignoring reality. 
It'll just mean that folks ignore sip-outbound. The best we can do is 
clearly document the ongoing and increasing troubles of UDP, so that a 
provider or enterprise can make an educated choice and make the move to 
tcp based on the evidence we've provided.

-Jonathan R.

---

From: <[email-masked]>
Date: Thu, 15 Mar 2007 12:44:09 +0200

Hi, 

>I expect in reality, providers will know exactly how big each 
>message will be since their clients are controlled by them and 
>they have tested them with exactly the messages they will emit. 
>So, this error code is really likely to show up in pre-deployment 
>testing, at which point you either switch to tcp, or live with the 
>risk.

You are making quite strict assumptions in what you are saying.
In practice this model would have the following limitations:

- As ICE is likely to increase the size of the SDP body
  of INVITE by 200-400 bytes, when applying the full set 
  of NAT traversal method the probability grows that the
  size of the INVITE request ultimately hits the MTU limit.

- UDP would only apply to closed environments (walled gardens)
  where the service provider is able to certify all the 
  connected clients and control the number of proxies in 
  any possible route within the network (to limit the number
  of Via and Record-Route headers). Furthermore there can 
  not be any direct SIP connectivity to rest of world. 
  I admit that this may apply to how SIP is used e.g. in 
  enterprises with their IT departments testing and validating 
  all pieces of equipment or VoIP operators exclusively providing 
  the UAs for their customers and having their PSTN gateways 
  towards the rest of the world. 

- It would preclude consumers connecting the service with 
  their favourite new softphone just installed to their PC.

- Only the services certified by the service provider could
  ever be applied. 

- Or in best case if the service provider is able to control
  the types of NATs applied within the network so that those
  would be able to cope with IP fragmentation, in this case
  UDP could be used even for messages larger than MTU.

If the provider is happy with all these limitations, yes
then Outbound with UDP could make sense for the deployment.
I still have the feeling that there are two many 'if's 
preventing this scheme to be robust and reliable. Too
many things might simply go wrong.

>>>- folks whose tcp scale is a problem need to FIX it, 
>>> since there is this issue with UDP outbound. 
>> 
>It'll just mean that folks ignore sip-outbound. 

It is hard for me to understand that while all the UAs and proxies
have to be modified in order to support Outbound, upgrading for
TCP support would be as difficult as it appears to many. For the
UA point of view, having a IP stack with TCP support integrated
to SIP does not appear much more difficult than supporting STUN 
keepalives. For the proxy point of view I am hearing evidence
from multiple independent research projects that scaling TCP
would not be impossible to solve (at least for Linux servers).
TCP could even help some of the scalability problems since it
uses even more lighweight keepalive schemes as STUN and the
frequency of keepalives is lower in TCP compared to UDP.

>The best we can do is clearly document the ongoing and 
>increasing troubles of UDP, so that a provider or enterprise 
>can make an educated choice and make 
>the move to tcp based on the evidence we've provided.

What I am worried about is that the message IETF makes in 
SIP Outbound draft (only one paragraph in the spec giving
a hint that there may be a problem) is very vague. Yes it
has been discussed in the list, but still providing extensive
UDP machinery in Outbound (STUN keepalives etc.) would create a
mixed message and delay the whole proxy industry to face the 
issue and start actions to fix the TCP scalability problem.

Also it would call the vendors to provide more complex
solutions as necessary. Here I mostly refer to STUN 
keepalives needed for UDP vs the CRLF/TCP keepalives 
only needed for TCP. This also can delay how soon
Outbound really hits the market in large scale.

About the dynamic fix with reg-event:

>>>..... The home proxy/registrar would generate a 

I agree that typical reg-event NOTIFYs are smaller than MTU,
but there is not really any upper limit so there is a chance
of fatal failure. As an example consider a case where a person 
called Gordon had registered his wireless PDA arrives to office 
and registers now also from his PC using Outbound and UDP:

-> REGISTER
<- 200 OK
-> SUBSCRIBE
<- 200 OK
<- NOTIFY

In this case the full reg-event notification could be over
1300 bytes e.g. like this one:

Request-Line: NOTIFY sip:[email-masked]:12363
SIP/2.0
Call-ID: VZ8iKyeJoIc2TwYRTcSB5j_bBPdQ3x
Contact: <sip:registrar.yourserviceprovider.com>
Content-Type: application/reginfo+xml
CSeq: 1 NOTIFY
From: "Gordon Maverick"
<sip:[email-masked]>;tag=poqw089q23oijf0+as9e
Max-Forwards: 69
To:  "Gordon Maverick"
<sip:[email-masked]>;tag=kh0h5eib81hc7r1i1sqt
Via: SIP/2.0/UDP
proxy.yourserviceprovider.com:5060;branch=z9hG4bK2147483647
Via: SIP/2.0/UDP
registrar.yourserviceprovider.com:5060;branch=z9hG4bK2134765492
Event:  reg
Subscription-State: active;expires=3600
Content-Length: 649
Message body
<?xml version="1.0" encoding="UTF-8"?>
<reginfo xmlns="urn:ietf:params:xml:ns:reginfo" version="0"
state="full">
<registration aor="sip:[email-masked]"
id="a100" state="active">
<contact id="980" state="active" event="created"
duration-registered="0">
<uri>sip:[email-masked]:12363</uri>
<display-name>Gordon Maverick</display-name>
</contact>
<contact id="977" state="active" event="registered"
duration-registered="1204" call-id="Vqpwoidf034i0_jv0wj090vasdfj">
<uri>sip:[email-masked]:14568</uri>
<display-name>Gordon Maverick</display-name>
</contact>
</registration>
</reginfo>

Do you believe we should impose some limitations about how
the reg-event notifications can look like if sent for UAs
using Outbound with UDP ? Or introduce some exceptional
practices to send partial notification right after SUBSCRIBE
asking the client to register with TCP if the full notification 
would be bigger than MTU ? Sorry to say but to me this mechanism 
appears as broken.

Regards,

Erkki

---

From: Spencer Dawkins <[email-masked]>
Date: Thu, 15 Mar 2007 07:04:01 -0500

If I understand correctly, Erkki is saying that we're driving off a cliff, 
and Jonathan is saying that we haven't given anyone outside the IETF an 
indication that we're going to change direction, so suddenly deprecating UDP 
isn't realistic.

I do see what both Jonathan and Erkki are saying, and I think both have 
valid points.

Perhaps it would be worth putting together a short I-D, either as 
Informational or (probably better) as BCP, to say:

- we know 3261 requires TCP support,

- we know many SIP devices were deployed without it and work today,

- we know that we are in the process of developing new standards-track 
specifications that will significantly increase the size of SIP messages,

- we know that IP-level fragmentation will increase, and is likely to be a 
problem,

- so if you have already deployed UDP-only SIP, this would be a good time to 
start planning for TCP support".

I can name some specifications that qualify here (Erkki mentioned ICE, and 
GRUU was discussed earlier this week), but is there a complete list that 
could be included, just off the top of anyone's head?

This is probably less than two pages of non-boilerplate text, even including 
the list. Could it help the current situation?

Spencer

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Thursday, March 15, 2007 5:44 AM
Subject: RE: [Sip] draft-petithuguenin-sip-outbound-fragmentation-02

Hi,

-> REGISTER
<- 200 OK
-> SUBSCRIBE
<- 200 OK
<- NOTIFY

Regards,

Erkki

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Thu, 15 Mar 2007 13:50:55 +0100

Hi,

What I am worried about is the assumption that the size of SIP messages
will keep on growing, growing and growing, and nobody seems to care.
Maybe THAT is something we should look at also, and not only focus on
the transport protocol.

And, just because we define hundreds of new message increasing
extensions, it doesn't mean that people are going to implement them - at
least not in the near future. I am pretty sure there are lots of
deployments out there, for which UDP works just fine. When people start
to implement/deploy new message-increasing features, they are probably
going to look into the transport issues also.

Also, even if we remove UDP from outbound, what will prevent people from
implement it anyway - either using STUN keep-alive, or some other
mechanism for the keep-alive??? Isn't it better to have a standardized
way of doing it? 

Remember that STUN keep-alive is not part of outbound as such - it's a
STUN usage defined in 3489bis. The outbound spec defines how to use the
STUN keep-alive in SIP environments, no matter if you implement the rest
of outbound or not, so that should be there (or somewhere else) even if
outbound support for UDP is removed.

Regards,

Christer 

> -----Original Message-----
> From: Spencer Dawkins [mailto:[email-masked]] 
> Sent: 15. maaliskuuta 2007 14:04
> To: [email-masked]
> Subject: Re: [Sip] draft-petithuguenin-sip-outbound-fragmentation-02
> 
> If I understand correctly, Erkki is saying that we're driving 
> off a cliff, and Jonathan is saying that we haven't given 
> anyone outside the IETF an indication that we're going to 
> change direction, so suddenly deprecating UDP isn't realistic.
> 
> I do see what both Jonathan and Erkki are saying, and I think 
> both have valid points.
> 
> Perhaps it would be worth putting together a short I-D, 
> either as Informational or (probably better) as BCP, to say:
> 
> - we know that we are in the process of developing new 
> standards-track specifications that will significantly 
> increase the size of SIP messages,
> 
> - we know that IP-level fragmentation will increase, and is 
> likely to be a problem,
> 
> - so if you have already deployed UDP-only SIP, this would be 
> a good time to start planning for TCP support".
> 
> I can name some specifications that qualify here (Erkki 
> mentioned ICE, and GRUU was discussed earlier this week), but 
> is there a complete list that could be included, just off the 
> top of anyone's head?
> 
> This is probably less than two pages of non-boilerplate text, 
> even including the list. Could it help the current situation?
> 
> Spencer
> 
> Request-Line: NOTIFY 
> sip:[email-masked]:12363
> SIP/2.0

---

From: [email-masked]
Date: Thu, 15 Mar 2007 09:22:00 -0500

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>

   What I am worried about is the assumption that the size of SIP
   messages will keep on growing, growing and growing,

It seems a safe assumption.

   and nobody seems to care.

What do you propose to do about it?

Dale

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 15 Mar 2007 11:26:50 -0400

In case you hadn't noticed, the amount of memory and cpu required to run 
applications keeps growing and nobody seems to care. And the amount of 
network bandwidth needed to successfully browse on the web also keeps 
growing and nobody seems to care.

The main reason nobody seems to care is probably that the amount of cpu, 
memory, and bandwidth available, and the unit price of it, also keeps 
growing.

	Paul

---

From: [email-masked]
Date: Thu, 15 Mar 2007 12:44:35 -0500

From: Paul Kyzivat <[email-masked]>

   The main reason nobody seems to care is probably that the amount of
   cpu, memory, and bandwidth available, and the [amount of it per
   dollar], also keeps growing.

And perversely, despite that SIP is hardly designed to be efficient
with each bit it sends over the wire, a SIP-based phone system is
cheaper than a TDM-based phone system.

Dale

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Thu, 15 Mar 2007 19:52:28 +0100

Hi, 

>In case you hadn't noticed, the amount of memory and cpu required to
run applications keeps growing and nobody seems 
>to care. 

Memory and cpu is probably not a problem for a SIP client running on a
PC. But, there are other types of nodes where memory is not "freely
available" in the same sense.

Also, what is my grand mother supposed to do when she is trying to make
an important phone call, and her SIP phone crashes, and all she sees on
the "screen" is a core dump or a message saying "Out of memory"...

>And the amount of network bandwidth needed to successfully browse on
the web also keeps growing and nobody 
>seems to care.

Web browsing is not as real-time sensitive, and does not have the same
setup restrictions, as phone calls. Also, I believe we are expected to
design something which is more effective and efficient than the
protocols used today.

>The main reason nobody seems to care is probably that the amount of
cpu, memory, and bandwidth available, and the unit 
>price of it, also keeps growing.

That is probably why we have sigcomp, why people are working on more and
more efficient codecs etc etc...

Regards,

Christer

> > This list is for NEW development of the core SIP Protocol Use 
> > [email-masked] for questions on current sip Use 
> > [email-masked] for new developments on the application of sip
> > 
> 
> This list is for NEW development of the core SIP Protocol Use 
> [email-masked] for questions on current sip

---

From: "Christer Holmberg \(JO/LMF\)" <[email-masked]>
Date: Thu, 15 Mar 2007 20:16:06 +0100

Hi, 

>And perversely, despite that SIP is hardly designed to be 
>efficient with each bit it sends over the wire, a SIP-based 
>phone system is cheaper than a TDM-based phone system.

So, our aim is only to make something which is cheaper (SIP phones I've
seen aren't that cheap, though...), it doesn't have to be better? :)

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 15 Mar 2007 15:49:00 -0400

Everything is relative. But I'm not seeing anyone screaming to make sip 
work over 300 baud modems on a .5mhz device with 700kb of memory.

	Paul