Subject: [Sipping] Re: Early media: a summary

From: Paul Kyzivat <[email-masked]>
Date: Mon, 02 Dec 2002 15:20:52 -0500

Gonzalo,

I don't see how the changes you have made help solve the problem. As far as I can see, adding another attribute simply makes the offer/answer handshake more complex without benefit.

The fundamental problem with the offer/answer protocol is that each side explicitly controls where it will receive with each offer/answer exchange. There is never any guarantee that the answerer will provide the same media address that was provided in a prior exchange. So when an endpoint wants to switch from not-sending to sending, if it sends an offer and then begins sending media to the previously agreed to address/port, there is no guarantee that the port in the answer will be the same, or that the media stream might not forbid sending entirely.

Also, if there was previously a negotiation to permit sending but not intend to send, a new offer to change the intention to permit sending may still be overtaken by media packets. To use this technique to avoid clipping, the receiver would have to receive and present media even when it has been negotiated that the sending doesn't intend to send. Hence the a:intention attribute doesn't seem to have any value.

If you assume that an endpoint will concurrently learn of the need to send and begin getting media, then there is no solution to clipping other than establishing a media stream with sender and receiver prior to that event. If the endpoint can separate those two events, and control when media will be available for sending based on signalling feedback, then there are many other options available.

The typical telephone seems to fall somewhere between the two extremes. There is often some delay between the indication that a call should be answered and the time when meaningful media becomes available to send, but the delay is not coupled to signalling feedback. Whether the delay is long enough for signalling to complete will vary based on network delays, the human factors of the device design, and human response times. (How long does it take from off-hook until the phone is in position to speak into?) This should *only* apply to devices directly associated with humans. Automated devices that *control* their media should not enable it until signalling is complete.

In the case of human controlled devices, this problem could be mitigated to some extent by providing feedback to the user about when it is acceptable to talk, perhaps by generating feedback, such as switching noises that indicate the call isn't yet complete.

	Paul 

Gonzalo Camarillo wrote:
> 
> Folks,
> 
> I believe we have had interesting exchanges of ideas about early media
> in the mailing list so far. We are getting a better understanding of the
> problem.
> 
> I have submitted version 00 of the draft with a couple of modifications
> from the pre-released version I made available last week.
> 
> I have added a section about media clipping avoidance summarizing all
> the discussions we have had so far (Sections 2.3., 2.4 and 2.5). I would
> like to know your opinion about the SDP media-level attribute that the
> draft defines. It is backwards compatible and it can be used only by
> systems with tough requirements regarding media clipping... do people
> agree that the difference between a receiver not accepting media and a
> sender not intending to send media is the issue we have to resolve?
> 
> Until it appears in the archives, you can fetch it from:
> 
> http://standards.ericsson.net/gonzalo/papers/draft-camarillo-sipping-early-media-00.txt
> 
> Let's try to resolve all the issues one by one. Let's focus now on media
> clipping and then we will move on to the other issues we have. Otherwise
> the mailing list discussions can easily get out of control.
> 
> Feedback is appreciated.
> 
> Thanks,
> 
> Gonzalo
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Gonzalo Camarillo <[email-masked]>
Date: Tue, 03 Dec 2002 11:12:41 +0200

Paul,

> Also, if there was previously a negotiation to permit sending but not 
>intend to send, a new offer to change the intention to permit sending 
>may still be overtaken by media packets. To use this technique to avoid 
>clipping, the receiver would have to receive and present media even when 
>it has been negotiated that the sending doesn't intend to send. Hence 
>the a:intention attribute doesn't seem to have any value.

The fact that any SIP signal can be overtaken my the media is something
unavoidable we have to live with. However, the a:intention attribute
would be use to drive the GUI, not the media tools. Therefore, you do
not get media clipping. You would get an unsynchronized GUI for a
typically very short period of time.

And to me, being able to signal to the other end that you will not be
sending media for a while seems useful. The video example I wrote in the
draft illustrates this point.

Thanks,

Gonzalo

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 03 Dec 2002 09:57:57 -0500

Gonzalo,

Following this logic, a recipient that has negotiated 'inactive' for the media stream, and who has also received a=intention:nosend, is still expected to receive media over the connection and render it.

In your video example, if the video window is minimized when no video is expected, you will still get 'video clipping. 

An analogous audio example would be if you normally play music on your pc via a media player application. When your softphone has negotiated a media stream that expects to receive audio, it then grabs the audio device from the media player and uses it to render incoming audio over the media stream. Whenever the stream is not expected to receive audio, the device is handed back to the media player. This also would result in clipping if audio is received over a media stream where incoming audio has not been negotiated.

As someone else mentioned on this thread, fixing clipping by handling received media on a stream that isn't expected to receive media may also be inconsistent with the intended semantics of 'inactive' in the midst of a media session. 

If these changes are made, what is the purpose of the directionality attributes? Are they simply a request of the form "could you help me out by being quiet for awhile?", that is not in any way binding? And then, is the intention attribute simply a way of saying "No, I intend to talk even though you don't want me to?", which is itself optional because you can talk without saying it and the recipient is still expected to render what you send?

This seems very wrong.

It is important to get this clarified. It makes a big difference in the way I implement a media stream. Am I *required* to read from a stream that is inactive, to ensure that unexpected media doesn't queue up in it? This isn't such a problem with UDP, but could be a serious problem with media streams sent over TCP. If so, some may find it preferable to go back to c=0.0.0.0 for inactive streams.

	Paul

---

From: Gonzalo Camarillo <[email-masked]>
Date: Tue, 03 Dec 2002 17:17:11 +0200

Paul,

there are two transitions in the GUI when a sender stops sending media
for a while:

1) The sender stops sending media: this can be signalled with the
intention attribute. This way the receiver can distinguish this
situation from a network dropping packets.

2) The sender resumes sending media: this time SIP will not help you,
because your media packets may arrive before your SIP signalling to the
receiver. Therefore, the receiver must listen to the media.

Therefore, the intention attribute resolves the first bullet. The second
bullet cannot be resolved with SIP. Now, we should see if it is
worthwhile resolving the first bullet or not.

I claim that it is worthwhile resolving it, because in many situations I
want the other end to know that I am not sending media for a while
(intention attribute), but I want him to know that media may resume at
any point in time (SDP direction attribute). In the video example, the
receiver will have to be ready to maximize the window as soon as media
resumes.

What is your opinion on this matter? Do you find useful resolving bullet
1? The other option is let the receiver think that either the sender is
not sending anything or that the network is dropping packets. This may
not usually be a problem with audio, but with video the receiver will
typically freeze the last image received -> bad user experience.

Thanks for the comments,

Gonzalo

-- 
Gonzalo Camarillo         Phone :  +358  9 299 33 71
Oy L M Ericsson Ab        Mobile:  +358 40 702 35 35
Telecom R&D               Fax   :  +358  9 299 30 52
FIN-02420 Jorvas          Email :  [email-masked]
Finland                   http://www.hut.fi/~gonzalo
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP