Subject: Re: [splices] Using Two Separate Devices to Start a Conversation proposal

From: Peter Musgrave <[email-masked]>
Date: Sun, 5 Jun 2011 15:15:10 -0400

HI Rifaat,

I have concerns about this approach. Fundamentally the RTP stream is not
symmetric (in the sense of RFC4961). This has consequences for NAT
traversal, general interoperability and media path security schemes like
ZRTP.

I think an approach in which one of the loosely coupled endpoints acts as a
media relay would eliminate all these issues. It can then source and sink
audio on the same port to the external device and provide the appearance of
symmetric RTP. (This approach would also have merit in the case where one LC
endpoint does audio and the other does video since there is now an RTP/RTCP
path between the endpoints for exchanging data for the purposes of
lipsync).

I recognize this imposes an additional role of relaying media in the
implementation of a LC device - but if the objective is to be interoperable
with non LC devices "in the wild" - I see no other alternative.

Regards,

Peter

On Thu, May 26, 2011 at 8:37 PM, Shekh-Yusef, Rifaat (Rifaat) <
[email-masked]> wrote:

> Hi,
>
> The following proposal is to cover the first use case in the disaggregated
> media draft.
>
> Regards,
>  Rifaat
>
>
>   Alice                 Alice                Proxy                  Bob
>    PC                 Desk Phone
>    |                     |                     |                     |
>    |                     |                     |                     |
> Both Alice's devices subscribe to the reg event package, which allows each
> device to
> discover the capabilities of the other device based on the feature tags
> provided by each device.
> The Desk Phone knows that the PC supports Video, while the PC knows that
> the Desk Phone only supports audio.
>    |                     |                     |                     |
>    |                     | SUBSCRIBE reg       |                     |
>    |                     |-------------------->|                     |
>    |                     | 200 OK              |                     |
>    |                     |<--------------------|                     |
>    |                     |                     |                     |
>    | SUBSCRIBE reg       |                     |                     |
>    |------------------------------------------>|                     |
>    | 200 OK              |                     |                     |
>    |<------------------------------------------|                     |
>    |                     |                     |                     |
>    |                     |                     |                     |
> The two devices also subscribe to the dialog of each other.
>    |                     |                     |                     |
>    |                     |                     |                     |
>    | SUBSCRIBE dialog    |                     |                     |
>    |-------------------->|                     |                     |
>    | 200 OK              |                     |                     |
>    |<--------------------|                     |                     |
>    |                     |                     |                     |
>    | SUBSCRIBE dialog    |                     |                     |
>    |<--------------------|                     |                     |
>    | 200 OK              |                     |                     |
>    |-------------------->|                     |                     |
>    |                     |                     |                     |
>    |                     |                     |                     |
>
>
>
> Alice wants to initiate a call to Bob, but wants to use her Desk Phone to
> receive the audio from Bob and use her PC to send audio to Bob.
> The PC instructs the Desk Phone to initiate a call to Bob in a receive only
> mode.
>
>    |                     |                     |                     |
>    | INVOKE Action:
> urn:invoke:call:initiate;media=audio;direction=recvonly;target=bob@whatever
>    |-------------------->|                     |                     |
>    | OK                  |                     |                     |
>    |<--------------------|                     |                     |
>    |                     | INVITE [recvonly]   |                     |
>    |                     |-------------------->|                     |
>    |                     |                     | INVITE [recvonly]   |
>    |                     |                     |-------------------->|
>    |                     |                     | 180                 |
>    |                     | 180                 |<--------------------|
>    |                     |<--------------------|                     |
>    |                     |                     | 200 OK              |
>    |                     | 200 OK              |<--------------------|
>    |                     |<--------------------|                     |
>    |                     |                     |                     |
> As soon as Bob answers the call, a NOTIFY is sent to Alice's PC (dialog
> event package)
>    |                     |                     |                     |
>    | NOTIFY              |                     |                     |
>    |<--------------------|                     |                     |
>    | OK                  |                     |                     |
>    |-------------------->|                     |                     |
>    |                     |                     |                     |
> PC sends a new INVITE with Join with a sendonly SDP offer
>    |                     |                     |                     |
>    | INVITE Join [sendonly]                    |                     |
>    |-------------------->|                     |                     |
>    | 100 Trying          |                     |                     |
>    |<--------------------|                     |                     |
>    |                     |                     |                     |
>    |                     | re-INVITE [sendrecv]|                     |
>    |                     |-------------------->|                     |
>    |                     |                     | re-INVITE [sendrecv]|
>    |                     |                     |-------------------->|
>    |                     |                     | 200 OK              |
>    |                     | 200 OK              |<--------------------|
>    |                     |<--------------------|                     |
>    | 200 OK              |                     |                     |
>    |<--------------------|                     |                     |
>    |                     |                     |                     |
>    |                     |                     |                     |
>    |<----dialog2-------->|<---dialog1------------------------------->|
>    |                     |                     |                     |
>    |                     |<======audio===============================|
>    |=============================audio==============================>|
>    |                     |                     |                     |
>    |                     |                     |                     |
>
> _______________________________________________
> splices mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/splices
>

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 6 Jun 2011 15:07:29 -0400

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Peter Musgrave [[email-masked]]

I have concerns about this approach. Fundamentally the RTP stream is not symmetric (in the sense of RFC4961). This has consequences for NAT traversal, general interoperability and media path security schemes like ZRTP.
_______________________________________________

I have to agree and disagree...  As Rifaat diagrammed it, the call should work, as it uses already-defined SIP facilities.  Indeed, it's rather clever, I'd never thought of joining a sendonly dialog with a recvonly dialog.  There are some limitations, in that we depend on the far-end UA to execute the join in the way we want.

The problems you raise, those of asymmetric RTP, already exist -- One can make a sendonly audio call, or a recvonly audio call.  We need to ensure that NAT traversal and ZRTP work correctly when  media stream is set up to be one-way.

Dale

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 6 Jun 2011 15:19:49 -0400

I think an approach in which one of the loosely coupled endpoints acts as a media relay would eliminate all these issues. It can then source and sink audio on the same port to the external device and provide the appearance of symmetric RTP.
_______________________________________________

I like the approach of having an aggregation controller, probably incorporated into one of the UAs, that coordinates the other aggregated UAs using 3PCC techniques.  (Either relaying the RTP, or manipulating the SDP to connect each source and sink.)

Of course, the aggregation controller has to handle all the "macro" call-control actions, such as receiving and acting on REFER, sending REFER, receiving and acting on INVITE-Replaces and INVITE-Join, etc.  Interestingly, all of the proposals require that one UA or the other handle all of these circumstances, although with a 3PCC aggregation controller, the need for designing how to handle these actions is more obvious.

The one additional operation is how does an aggregation controller hand over a call to another aggregation controller?  The aggregation process contains quite a bit of state (although some of it is reported in the dialog events of the aggregation controller).

BTW, here is a case which shows why it is better to have  the aggregation process done entirely at one end of the call:  Consider two aggregated clusters of UAs maintaining a dialog between themselves.  If a UA had to be aware of the aggregation at the far end of the call, along with the aggregation it was participating in on the near end, the logic would be incredibly complicated.

Dale

---

From: Alan Johnston <[email-masked]>
Date: Mon, 6 Jun 2011 14:35:56 -0500

If I understand correctly, there will be two separate RTP streams, two m= lines?

If this is the case, then two separate NAT traversal mechanisms will be used (i.e. ICE rubs twice or two relays used). As far as ZRTP or other media path keying protocols, each session will be keyed separately. With ZRTP the endpoint will see two different ZIDs.  This does unfortunately mean two Diffie Hellman calculations. 

Since these are separate sessions, each will have an RTCP session as well, and these may need NAT traversal as well.

So this will work, but things like logging and quality reporting will be quite complicated. 

- Alan -

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 06 Jun 2011 15:36:44 -0400

IMO it is *undesirable* to require that there be a common point through 
which all the media to/from that end must flow. Its especially 
problematic if the "intelligent" device that would have the capability 
to do this is bandwidth limited - as could be the case with mobile phones.

Maybe this can be tolerated if we can find nothing better. Or maybe it 
can be a fallback approach if something else doesn't work. But I'd hate 
to see defining that as *the* approach without trying to find a way to 
avoid it.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 06 Jun 2011 15:48:10 -0400

> So this will work, but things like logging and quality reporting will be quite complicated.

As Dale mentioned, this also depends on the other end doing the "right 
thing" with this case. It is a leap of faith to assume it will realize 
it should accept both streams and use one for input and the other for 
output.

ISTM its at least as likely that it would accept the first stream, 
reject the second, and then have only a one-way "conversation".

	Thanks,
	Paul

> - Alan -
>
>
>
> On Jun 6, 2011, at 2:07 PM, "Worley, Dale R (Dale)"<[email-masked]>  wrote:
>

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 6 Jun 2011 15:45:49 -0400

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Paul Kyzivat [[email-masked]]

A good point -- We've been assuming that all of the aggregated UAs are near
each other in network terms.  But this use-case shows that to be false:  the controller
is my smartphone, but the video source/sink is my PC.

Of course we can accommodate this within a 3PCC environment by having the controller
shuffle SDP information to connect sources/sinks, rather than relaying RTP.

Dale

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 6 Jun 2011 15:49:12 -0400

> If I understand correctly, there will be two separate RTP streams,

There will be two m= lines, but it is one m= line in one dialog, and
another m= line in the other dialog.

> If this is the case, then two separate NAT traversal mechanisms will
> be used (i.e. ICE rubs twice or two relays used). As far as ZRTP or
> other media path keying protocols, each session will be keyed
> separately. With ZRTP the endpoint will see two different ZIDs.
> This does unfortunately mean two Diffie Hellman calculations.

I think the envisioned problem is with simpler NAT traversal
mechanisms which do not use ICE, but rather depend on the RTP packets
traveling "out" through the NAT to establish and maintain an inward
mapping "pinhole".  That won't work with one-way media, or rather,
inward-only media.

> Since these are separate sessions, each will have an RTCP session as
> well, and these may need NAT traversal as well.

> will be quite complicated.

Hmmm...  In this case, they won't be any harder than they would be in
a three-way ad-hoc conference call.

Dale

---

From: Alan Johnston <[email-masked]>
Date: Mon, 6 Jun 2011 15:34:29 -0500

On Mon, Jun 6, 2011 at 2:48 PM, Paul Kyzivat <[email-masked]> wrote:
>> If this is the case, then two separate NAT traversal mechanisms will be
>> used (i.e. ICE rubs twice or two relays used). As far as ZRTP or other media
>> path keying protocols, each session will be keyed separately. With ZRTP the
>> endpoint will see two different ZIDs.  This does unfortunately mean two
>> Diffie Hellman calculations.
>>
>> Since these are separate sessions, each will have an RTCP session as well,
>> and these may need NAT traversal as well.
>>
> thing" with this case. It is a leap of faith to assume it will realize it
> should accept both streams and use one for input and the other for output.
>
> ISTM its at least as likely that it would accept the first stream, reject
> the second, and then have only a one-way "conversation".

OK, I understand the problem right now.  The problem is that this is a
special kind of "Join" operation.  It is kind of a join, but not
quite.

The operation we are after is perhaps similar to 3.3.8 Far-Fork in RFC 5850:

I suspect instead of Join, we need a new primitive, lets call it
Splice or Merge which requests the UAS to add the media in the INVITE
into the dialog that is referenced.

- Alan -

>>> I have to agree and disagree...  As Rifaat diagrammed it, the call should
>>> work, as it uses already-defined SIP facilities.  Indeed, it's rather
>>> clever, I'd never thought of joining a sendonly dialog with a recvonly
>>> dialog.  There are some limitations, in that we depend on the far-end UA to
>>> execute the join in the way we want.
>>>
>>> The problems you raise, those of asymmetric RTP, already exist -- One can
>>> make a sendonly audio call, or a recvonly audio call.  We need to ensure
>>> that NAT traversal and ZRTP work correctly when  media stream is set up to
>>> be one-way.
>>>
>>> Dale
>>>

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Mon, 6 Jun 2011 17:15:57 -0400

No, surely it is an ordinary join.  At the far UA, there are two incoming dialogs, each with one audio m= line.  The only oddity is that one dialog is recvonly (at the far UA), and so is never sent media.  The other dialog is sendonly (at the far UA) and so never provides audio to the mixing, but is only sent the result of the mixing.  The effect is that the far UA's input audio is sent only to the second dialog, and the far UA's output audio is driven only by the first dialog.

Dale

---

From: Alan Johnston <[email-masked]>
Date: Mon, 6 Jun 2011 16:45:40 -0500

But isn't the INVITE Join received at Alice's Desk Phone?  If so, then
a normal Join will result in Alice's Desk Phone performing mixing of
audio streams from Alice's PC, Alice's Desk Phone, and Bob.  I thought
what was desired (and shown by Rifaat's flow) was that Alice's Desk
phone did not mix the media, but instead inserted/spliced the media
from Alice's PC into the dialog with Bob, resulting in a re-INVITE to
Bob with two m= lines.

- Alan -

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 06 Jun 2011 18:09:13 -0400

OK, forget what I said about this. I didn't read the details and made a 
bad assumption about them.

	Sorry,
	Paul

---

From: Peter Musgrave <[email-masked]>
Date: Mon, 6 Jun 2011 19:27:57 -0400

Hi all,

My assumptions were that:
- Alice's desk phone is doing the media aggregation and creating a reINVITE
for Bob with a=sendrecv and asymmetric media:
  * media is sent from Bob to Alice desk phone
  * media is sent from Alice Computer to Bob

As far as Bob knows this is a dialog only with Alice's deskphone. Bob does
not have to support SPLICES or do anything special.

Bob must handle asymmetric RTP from Alice.

Ok?

I agree that requiring media relay does become a bad idea when the relaying
element is e.g a cellphone.

Do we think that having asymmetric RTP will be ok? A lot of implementation
have worked toward symmetric RTP (and some will only play RTP arriving from
the place they are sending to! [1 implementation at SIPIT28, 3 at SIPIT27 -
so maybe not worth worrying about]). Are there any issues with ZRTP or use
of ICE? [I am not an ICE expert so I can't recall if symmetric RTP was baked
into ICE, umm "frozen in" to ICE maybe?]

Regards,

Peter

On Mon, Jun 6, 2011 at 5:45 PM, Alan Johnston <[email-masked]>wrote:

> > No, surely it is an ordinary join.  At the far UA, there are two incoming
> dialogs, each with one audio m= line.  The only oddity is that one dialog is
> recvonly (at the far UA), and so is never sent media.  The other dialog is
> sendonly (at the far UA) and so never provides audio to the mixing, but is
> only sent the result of the mixing.  The effect is that the far UA's input
> audio is sent only to the second dialog, and the far UA's output audio is
> driven only by the first dialog.
> >
> > Dale
> >

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 06 Jun 2011 19:45:49 -0400

I suspect asymmetric rtp *won't* work a lot of the time, for many 
reasons, whether we think they are reasonable or not.

Maybe this is just an unreasonable use case. I can't really imagine 
wanting to do this. If I have a handset, or headset, with a mic, it 
probably has a "speaker" too, and it would seem weird using the mic and 
not its speaker too. And when they are separate there is likely to be 
problems with echos.

	Thanks,
	Paul

On 6/6/2011 7:27 PM, Peter Musgrave wrote:
> - Alice's desk phone is doing the media aggregation and creating a
> reINVITE for Bob with a=sendrecv and asymmetric media:
> As far as Bob knows this is a dialog only with Alice's deskphone. Bob
> does not have to support SPLICES or do anything special.
>
> I agree that requiring media relay does become a bad idea when the
> relaying element is e.g a cellphone.
>
> Do we think that having asymmetric RTP will be ok? A lot of
> implementation have worked toward symmetric RTP (and some will only play
> RTP arriving from the place they are sending to! [1 implementation at
> SIPIT28, 3 at SIPIT27 - so maybe not worth worrying about]). Are there
> any issues with ZRTP or use of ICE? [I am not an ICE expert so I can't
> recall if symmetric RTP was baked into ICE, umm "frozen in" to ICE maybe?]
>
> Regards,
>
> Peter
>
>
>
>
>
> On Mon, Jun 6, 2011 at 5:45 PM, Alan Johnston <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     On Mon, Jun 6, 2011 at 4:15 PM, Worley, Dale R (Dale)
>     <[email-masked] <mailto:[email-masked]>> wrote:
>      > No, surely it is an ordinary join.  At the far UA, there are two
>     incoming dialogs, each with one audio m= line.  The only oddity is
>     that one dialog is recvonly (at the far UA), and so is never sent
>     media.  The other dialog is sendonly (at the far UA) and so never
>     provides audio to the mixing, but is only sent the result of the
>     mixing.  The effect is that the far UA's input audio is sent only to
>     the second dialog, and the far UA's output audio is driven only by
>     the first dialog.
>      >
>      > Dale
>      >
>     [email-masked] <mailto:[email-masked]>

---

From: "Shekh-Yusef, Rifaat (Rifaat)" <[email-masked]>
Date: Mon, 6 Jun 2011 22:02:52 -0400

Hi All,

Very interesting discussion.

What I had in mind is what Peter described below, but you guys came up with very interesting new ideas that can be used to address this scenario.
The scenario is technically an interesting and challenging one, but I tend to agree with Paul that it is not really a use case that will likely be used and maybe we should spend our time addressing other more likely to be used uses case.

Regards,
 Rifaat

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Paul Kyzivat [[email-masked]]
Sent: Monday, June 06, 2011 7:45 PM
To: [email-masked]
Subject: Re: [splices] Using Two Separate Devices to Start a Conversation proposal

        Thanks,
        Paul

---

From: Peter Musgrave <[email-masked]>
Date: Tue, 7 Jun 2011 06:19:10 -0400

I agree this can be dropped.

Peter

Sent from my iPad

---

From: Gonzalo Camarillo <[email-masked]>
Date: Tue, 21 Jun 2011 11:09:21 +0300

Hi Alan,

> into the dialog that is referenced.

Yes... this is exactly what we proposed 5 years ago in one of the drafts
that, eventually, led to the creation of the SPLICES WG:

http://tools.ietf.org/html/draft-loreto-sipping-dialog-correlation-01

The idea was that the media streams of a given session could be
established using more than one SIP dialog.

Note that most SPLICES-related discussions up to a couple of months ago
were based on the fact that such a mechanism (i.e., using different
dialogs to establish a single session) would require support from the
remote end point. That is also the point Paul made in his comment above.
I personally would be OK with a mechanism along these lines, though.

Cheers,

Gonzalo

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 21 Jun 2011 08:32:15 -0400

IMO there is nothing wrong with using Join to bring together one dialog 
with audio and another with video. I don't see why we would create 
another mechanism when that one is there to be used. Certainly it is a 
degenerate case of Join, but there is nothing wrong with degenerate cases.

Of course there might be UAs that are capable of supporting the case 
where the media in the dialogs is disjoint, but not the case where there 
is common media in the dialogs that must be mixed. But surely we can 
find an error code to cover that case.

The key issue here is what UA we expect to support this capability. IMO 
a user that wants to "splice" some of its devices together should take 
the responsibility of finding a UA that can handle the coordination, 
rather than depending upon the far end to do it.

	Thanks,
	Paul

---

From: Gonzalo Camarillo <[email-masked]>
Date: Thu, 30 Jun 2011 09:06:47 +0300

Hi Paul,

yes, Join was considered at that point. As you say, the semantics are
similar to Join but not identical (specially with respect of how things
are presented to the user). Overloading Join may not be the best way to
proceed.

If you are concerned with supporting another primitive in addition to
Join, the code base for Join and the new primitive would be almost
identical.

Cheers,

Gonzalo

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 30 Jun 2011 18:06:38 -0400

> into the dialog that is referenced.

OK, now I see the problem with using Join.

UA A1 sends an INVITE (requesting only audio) to UA B (which supports
both audio and video).

UA A2 sends an INVITE (requesting only video) to UA B (which supports
both audio and video), with Join to the first dialog.

Then we should expect that UA B will send its audio to A1 and its
video to A2, within the semantics of Join.

The difficulty is that UA B still sees the situation as two different
dialogs.  E.g., if A1 sends a REFER/Refer-To: C to B, the A1-B dialog
will be replaced by the C-B dialog, but the A2-B dialog will be
undisturbed.

Which is not the behavior we want if we are trying to logically bond
A1 and A2 to act as a single UA.

Using Join would have been convenient as it is already defined, and
theoretically, UAs already support it.

Dale