Subject: RE: [MMUSIC] Video phone and offer/answer model

From: "Christer Holmberg (JO/LMF)" <[email-masked]>
Date: Wed, 5 Jan 2005 07:36:08 +0100

Hi,

>Is there anything which bundle a number of m= lines of SDP?  I would
>like a session to be established only when the offerer and 
>the answerer
>agree in the codec of both video and audio. Otherwise, the answerer
>would reject the offer. (see the following sequence)
> 
> video phone#1                       video phone#2
>  (offerer)                            (answerer)
>      | (offer)                            |
>      | m=audio ...                        |
>      | m=video ...                        |
>      |----------------------------------->|
>      |                                    |
>      |                           (answer) |
>      |                        m=audio ... |
>      |                        m=video ... |
>      |<-----------------------------------|
> 
>But according to examples in RFC3264 (offer/answer model), 
>each m= line (video/audio/etc...) seems like being dealt with independently. So a
>session would be established when they agree in the codec of ether audio,
>video or etc. If the offerer and the answerer agreed in the codec of
>video and they didn't agreed in that of audio, they establish 
>a session and there becomes a video phone service without voice.

Correct. Of course, your application can choose to reject also the video stream if no common audio codecs are found. That's an implementation issue. You don't have to set up a video only service just because you happen to have common video codecs.

>I would like to avoid this situation. And I'm looking for the means that enables offerers to desire the video/audio 
>communication at a time.

I guess you're talking about some indicator saying "please accept this call if you find common codecs for both audio and video"? I don't think SDP supports that, so you would have to handle it on the application level. The easiest way, without using any protocol extensions etc, would be that the offerer simply terminates the session setup when it receveis the answer indicating video only.

Regards,

Christer Holmberg
Ericsson Finland

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: "James M. Polk" <[email-masked]>
Date: Wed, 05 Jan 2005 12:40:20 -0600

comments below

At 03:52 PM 1/5/2005 +0100, Belling Thomas wrote:
>Hi folks,
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> > Sent: Wednesday, January 05, 2005 3:14 PM
> > To: Jun KOSHIKO
> > Cc: [email-masked]
> > Subject: Re: [MMUSIC] Video phone and offer/answer model
> >
>...
> >
> > But I think your issue is that the *caller* doesn't want the call to
> > succeed unless there is both voice and video. I don't know of
> > any *good*
> > way to do this currently.
> >
>A clean way to resolve that would probably bee a new type of grouping 
>within the grouping framework. As a minimum, the grouping would express 
>that all media for the grouped m-lines need to be established together, or 
>none of them.

ah, but don't we really then need to address the scenario of the offer 
containing more than 1 audio codec and more than 1 video codec (i.e. "MUST 
choose one codec from each M line"), and only wishing to accept the call if 
one from each (1 audio and 1 video) is chosen - and not one from either 
(audio *or* video)?

For example, I might offer you (0 4 18) in audio and (31 34) in video in my 
SDP1 to you, and I care that you choose one each from both of these groups, 
not one total from either.

>I do not know if this is worth the effort, and there would certainly be a 
>backward compatibilty issue with hosts not understnding the extension.

If an indication is created requiring one codec from each M line to be 
chosen - and the answerer does not comply - this leaves it up to the 
offerer to decide if it wants to mandate both audio and video MUST be 
supported by the answerer in order to complete the call. If the offerer 
believes this is true, I do not believe there is a backwards compatibility 
issue, is there? The answer is simply rejected. This rejection should 
indicate to the offerer that the called party either doesn't support 
mandating audio and video, or doesn't support one of the M lines.

>For those kind of hosts as callee, the caller would still need to 
>terminate the call if undesirable media combinations are selected.
>
>Best regards,
>
>Th Belling
>
>
>----------------------------------
>Dr. Thomas Belling
>Siemens AG
>Sankt-Martin-Str. 76
>D-81541 München Germany
>Com MN PG NT MN 1
>MCH M 34 307
>Tel     +49 89 636 75207
>Fax     +49 89 636 75577
>Mobile  +49 172 2974678
>Email   [email-masked]
>
>
>

cheers,
James

                                *******************
                 Truth is not to be argued... it is to be presented

---

From: Belling Thomas <[email-masked]>
Date: Mon, 10 Jan 2005 18:49:52 +0100

Hi James,

comments below.

Thomas

> [mailto:[email-masked]] On Behalf Of James M. Polk
> Sent: Wednesday, January 05, 2005 7:40 PM
> To: [email-masked]
> Subject: RE: [MMUSIC] Video phone and offer/answer model
> > >

In my understanding each m-line talks about a single media stream, with different encoding possibilities. Requiring a selection of only 1 codec for each media stream is different from the problem beeing discussed so far, i.e. expressing that several media streams are required in a combination.

RFC 3264 states:
"The offer will contain zero or more media streams (each media stream
   is described by an "m=" line and its associated attributes). 
...
The list of media formats for each media stream conveys two pieces of
   information, namely the set of formats (codecs and any parameters
   associated with the codec, in the case of RTP) that the offerer is
   capable of sending and/or receiving (depending on the direction
   attributes), and, in the case of RTP, the RTP payload type numbers
   used to identify those formats.  If multiple formats are listed, it
   means that the offerer is capable of making use of any of those
   formats during the session.  In other words, the answerer MAY change
   formats in the middle of the session, making use of any of the
   formats listed, without sending a new offer.
...
10.2 One of N Codec Selection

   A common occurrence in embedded phones is that the Digital Signal
   Processor (DSP) used for compression can support multiple codecs at a
   time, but once that codec is selected, it cannot be readily changed
   on the fly.  This example shows how a session can be set up using an
   initial offer/answer exchange, followed immediately by a second one
   to lock down the set of codecs."

> not one total from either.

This is not what I had in mind in my proposal. The grouping would rather indicate that the answerer MUST either accept all the grouped media streams (each described by a separate m-line) or reject them all.
Accepting or Rejecting would be done for each media stream according to RFC 3264, i.e. the answerer needs to select at least one of the offered codecs for each m-line to accept the corresponding media stream.

---

From: "Even, Roni" <[email-masked]>
Date: Thu, 13 Jan 2005 19:43:54 +0200

Hi,
I would like to add that the caller can use caller prefs (RFC3841) to
say that he prefers to get the called party video phone if he has one
Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Brian Rosen
Sent: Thursday, January 13, 2005 4:46 PM
To: 'Jun KOSHIKO'; [email-masked]
Cc: 'Belling Thomas'
Subject: RE: [MMUSIC] Video phone and offer/answer model

I think this is a poor system design.  This is scenario 2 where the
caller
offers audio and video, but the called party only has audio, and you
want
refuse the answer because the caller insists on video.  Your problem is
that
the called party doesn't know it's going to be refused, so it begins
alerting.  The call is cancelled after the alerting begins, and you
don't
like that.

I'd prefer your caller to accept the call audio only.  That's what all
the
systems I've ever seen would do.  I've never seen a system refuse a call
if
it didn't offer/answer video.  In the systems I'm most familiar with,
this
scenario occurs even though the called party has video capability.  It
occurs when the called user disables the video, and it also occurs when
the
local environment can't handle the bandwidth implied.  The caller should
not
have an expectation that everyone he calls will accept video, even if he
knows they have the capability to.  

I think we have so much work to do, and this requirement is of so little
interest, that we should not spend any time on it.  If there are lots of
systems that need to be able to refuse a call if the caller can't handle
video, I'd like to hear about them.

Brian

> From: Jun KOSHIKO [mailto:[email-masked]]
> Sent: Thursday, January 13, 2005 5:37 AM
> To: [email-masked]
> Cc: 'Belling Thomas'; Brian Rosen
> Thank you for your reply.
> 
> Brian Rosen wrote:
> > I think this is bordering on silliness.
> >
> > Read the thread.
> >
> > You have the four cases of two variables:
> > 1) The caller offers audio and video, the called answers with both
> > 	This works with multiple sections, choices for each from the
> > 	offer, a single choice for each from the answer.
> > 2) The caller offers audio and video, the called answers audio only
> > 	99.99% of the time, the caller will accept the call audio only.
> > 	This is the normal case of a video phone calling an audio phone.
> > 	You almost always want to accept the call.  If you DON'T want to
> > 	let the call complete, the caller can reject accept the answer.
> > 	Done.
> > 3) The caller offers audio, the called would like video.  This is
the
> > 	audio phone calling the video phone.  99.99% of the time, the
> > 	right thing is to just make an audio choice and be happy.
> > 	If you want to reject the call, the called party can do that.
> > 4) The caller offers audio, the called is audio only.  No problem,
> > 	right?
> >
> > Where is the problem?
> 
> I can grasp the understanding for what you are saying. Now I would
like
> to consider about human interface. For example, in the following case,
> if the caller reject the offer which includes audio and doesn't
include
> video, although the callee's phone ring and the it's hung up, the call
> will be cancelled. This way seems unfriendly interface, I think.
> 
> If the caller can group m lines of video and audio and the callee
> understand it, the callee can reject the offer to avoid such a
situation.
> Definitely, whether the situation can be avoided or not depends on the
> implementation of the device of callee.  If the callee which doesn't
> understand the grouping attribute of m lines will accept the offer,
and
> then the call will be cancelled after the callee's phone rang and the
> user got the phone up, naturally.
> 
>    Caller                                  Callee
> (Video phone)                          (Audio Phone)
>       |                                      |
>       | INVITE with SDP(video/audio)         |
>       |------------------------------------->|
>       | 180 Ringing without SDP              |
>       |<-------------------------------------| Phone Ring!!
>       | 200 OK with SDP(audio)               |
>       |<-------------------------------------| Hung up!!
>       | ACK                                  |
>       |------------------------------------->|
>       | BYE                                  |
>       |------------------------------------->| Call cancelled!!
>       | 200 OK                               |
>       |<-------------------------------------|
>       | ACK                                  |
>       |------------------------------------->|
>       |                                      |
> 
> Best Regards,
> Jun Koshiko
> 
> > I say this as someone who has done this.  I'm happy to show you a
SIP
> video
> > phone that places and receives calls from other video phones and
audio
> > phones.  It works very nicely, as anyone who has seen it can attest.
> >
> > Lip Sync is a completely separate issue from offer/answer.  The
> videophone
> > mentioned above has perfect lip sync with no problems without any
> > proprietary anything.   If you want to make lip sync better, the
only
> > thing I think you would like is to understand the absolute time
> difference
> > between the audio clock and the video clock.  In every system I've
seen,
> > if you maintain the sync implied by the first packets, you're fine.
> > It would be nice to not have to assume this.  Just for reference,
the
> above
> > system maintains perfect sync on calls lasting 6-8 hours.  It
probably
> works
> > for longer, but I haven't looked to see if it does.
> >
> > If we don't absolutely need another mechanism, please, let's not
make
> more
> > work.
> > Brian
> > > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> > > Of Belling Thomas
> > > Sent: Monday, January 10, 2005 1:15 PM
> > > To: 'Jun KOSHIKO'; [email-masked]
> > > Cc: Belling Thomas
> > > Dear Jon,
> > > BR, Thomas
> > > > From: Jun KOSHIKO [mailto:[email-masked]]
> > > > Sent: Thursday, January 06, 2005 10:41 AM
> > > > To: [email-masked]
> > > > Cc: Belling Thomas
> > > > Hello Folks;
> > > > Thank you for your replies.
> > > >
> > > > > Belling Thomas wrote:
> > > > ...
> > > >
> > > > > that all media for the grouped m-lines need to be
> > > > established together,
> > > > > or none of them. As the media belong together in the sense
that
> they
> > > > > need to be handled by a combined application, the more far-
> reaching
> > > > > idea would be to indicate the application in the new grouping
> type.
> > > > > This obviously is more complicated, but also more powerful,
> > > > and issues
> > > > > like how to define and/or register such applications come up.
> > > >
> > > > RFC3388, grouping framework of m line, seems like a good
> > > > resolution for
> > > > the problem. Thanks for the information.
> > > >
> > > > I imagine that your idea of indication of the application like
the
> > > > following example. A new semantic like "VIDEO-PHONE" will be
> defined,
> > > > and lines of two m lines (audio/video) will be grouped. And some
> > > > attributes will be newly defined for video-phone specific use.
> > > > Is my understanding true?
> > > >
> > > > ...
> > > > a=group:VIDEO-PHONE 1 2
> > > > m=audio 30000 RTP/AVP 0 4 18
> > > > a=mid:1
> > > > a=xxx:xxx  (<- newly defined attribute)
> > > > m=video 30002 RTP/AVP 31 34
> > > > a=mid:2
> > > > a=xxx:xxx  (<- newly defined attribute)
> > > >
> > > >
> > >
> > > As I stated, I see two options:
> > >
> > > 1. Having a general new semantics, e.g. "COMBINED-APPLICATION" to
> express
> > > that the answerer shall either accept or reject all grouped media
> streams
> > > together.
> > >
> > > 2. Having specific semantics for each combined application, e.g.
> VIDEO-
> > > PHONE as you propose. For each application, this may imply
different
> > > additional things compared to 1, e.g. lipsynch.
> > >
> > > > As a minimum, I guess LS(Lip Synchronization) semantic defined
in
> > > > RFC3388 could be applicable without difining a new type of
grouping.
> > > >
> > > > I think that video and audio stream need to be synchronized for
the
> > > > video phone service. And synchronization of video and audio
> > > > implies the
> > > > existance of common codecs both of video and of audio.
Considering
> the
> > > > following case, I guess that a callee usually rejects the offer
if
> it
> > > > doesn't support either (0) of audio codec or (31) of video
codec.
> > > >
> > > > I feel slightly uncomfortable about using LS semantic because
> > > > the usage
> > > > of LS semantic seems like different from the problem.
> > > >
> > > >  Caller                                 Callee
> > > >    |                                       |
> > > >    | (offer)                               |
> > > >    | a=group:LS 1 2                        |
> > > >    | m=audio 30000 RTP/AVP 0               |
> > > >    | .                                     |
> > > >    | .                                     |
> > > >    | a=mid:1                               |
> > > >    | m=video 30002 RTP/AVP 31              |
> > > >    | .                                     |
> > > >    | .                                     |
> > > >    | a=mid:2                               |
> > > >    |-------------------------------------->|
> > > >    |                                       |
> > > >    |                               (answer)|
> > > >    |<--------------------------------------|
> > >
> > > If you are only intrested in a combination of audio and video, LS
> might
> > > do, although it is not exactly what you had in mind. An answerer
will
> > > probably accept or reject both media streams if lipsynch is
required,
> > > because this implies the media streams are closely linked
together.
> One
> > > the other hand, you might only want a combined acceptance or
rejection
> of
> > > media streams without lipsynch, and then LS would not be
appropriate -
> > > perhaps a bit theoretical for an audio and video combination.
> > >
> > > >
> > > > > I do not know if this is worth the effort, and there would
> > > > certainly be
> > > > > a backward compatibilty issue with hosts not understnding
> > > > the extension.
> > > > I think something like SIP require header may be needed to avoid
> > > > undesirable media combinations considering a backward
compatibility.
> > >
> > > The drawback of this solution is that an answering host supporting
a
> > > combined audio-video application, but not the new semantics, would
> reject
> > > the offer, although it might well behave in the desired way if no
SIP
> > > require header is used. It is more graceful for the offerer to
> terminate
> > > the session only if such an answerer indeed did an undesirable
> selection.
> > >
> > > >
> > > > Best regards,
> > > > Jun Koshiko
> > > >
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Th Belling
> > > > >
> > > > >
> > > > > D-81541 M|nchen Germany
> > > > ---------------------------------------------------------
> > > > Jun KOSHIKO
> > > > Next Generation Strategy
> > > > Research and Development Center
> > > > NTT-east Corporation
> > > >
> > > > Telephone  +81 3 5359 5104
> > > > Facsimile  +81 3 5359 4768
> > > > E-mail: [email-masked]
> > > > 19-2 Nishi-shinjuku 3-chome Shinjuku-ku Tokyo 163-8019 Japan
> > > > ---------------------------------------------------------
> > > >
> > >
> ---------------------------------------------------------
> Jun KOSHIKO
> Next Generation Strategy
> Research and Development Center
> NTT-east Corporation
> 
> Telephone  +81 3 5359 5104
> Facsimile  +81 3 5359 4768
> E-mail: [email-masked]
> 19-2 Nishi-shinjuku 3-chome Shinjuku-ku Tokyo 163-8019 Japan
> ---------------------------------------------------------
> 
>