Subject: [clue] Review of draft-ietf-clue-framework-03

From: "Roni Even" <[email-masked]>
Date: Wed, 8 Feb 2012 15:48:58 +0200

Hi,

I read 03 and it is a bit clear than 01, 02 was not a real version.

These are my initial comments. I will try also to start separate thread on
the major issues from this long list

1.       In section 3 the definition of media capture say "A Media Capture
(MC) may be the source of one or more Media streams."  What is the "more" in
this case. Can you explain since my understanding of a media capture is that
is one stream, can you give an example.

2.       In section 4, "This constitutes a significant change from previous
video conferencing  systems in which transmission of content was determined
primarily by  the sender." I am not sure what is "content" here, is it the
actual data or the semantics of the stream. I also do not believe that this
statement is true.

3.       The issue of capture scene / set / entry is not fully described
yet. In section 4 the text says "A provider organizes its media captures
that represent the same scene into capture sets.  A consumer chooses which
media captures it wants to receive according to the capture sets sent by the
provider."  I think that it will be good to clarify that there can be more
than one capture set per capture scene and to add the term capture set entry
to the terminology section. This is also relevant to the text in section
6.2.

4.       The coordinate units are first discussed in section 4. The currents
units which I am OK with are the mm and no unknown. The no scale as far as I
understand provide units in the coordinates system that allows the provider
to define a spatial order that the consumer will use to render, this option
can be used for example by a MCU. I think that we can add two more units.
The first is "no spatial information" this will allow the provider to leave
it to the consumer to decide what order to use. The second is "priority"
this is still no spatial relation between the media captures but the
provider can use it to specify which media captures are more important
allowing the consumer to select between media captures.

5.       In section 6.1.1 "Media Capture (MC) Attributes describe static
information about the captures that can be used by the consumer to help
decide which Media  Captures should be requested". I think that this
sentence is using a passive language and I suggest rephrasing it to "Media
Capture Attributes describe static information about the captures. A
Provider use the MC attributes to propose multiple MC choices to the
consumer. The consumer will select the MCs that he wants to render"

6.       In section 6.1.1 I suggest adding one more purpose "SL" for sign
language indicating that this is the sign language representation of the
audio stream, see RFC4796.

7.       In 6.1.1 the composed attribute is discussed elsewhere and is still
an open issue.

8.       In 6.1.1 I noticed that the audio channel format attribute values
changed from previous version. I do not recall that it was discussed and
that there was a consensus to remove the other values like 3.0, 3.1, 5.1
surround sound and binaural. As for the stereo value, does it represent one
stereo or two mono channels.

9.       In 6.1.1 for the switched attribute "What is 'most appropriate' is
up to the producer and could be the active speaker, a lecturer or a VIP." I
do not agree with this statement. The producer must be able to offer options
and let the consumer decide what is appropriate.

10.   In section 6.1.1 I suggest adding an attribute that will correlate
between the MC and the SDP m-line or SSRC of this media stream.

11.   When talking about capture set entry, my understanding is that a
consumer can select part of the MCs in the entry and not all of them since
they represent simultaneous streams. For example if the provider offers
(VC0, VC1, VC2) the consumer can request (VC2) or (VC0, VC2). I think that
this should be clear in the text. Maybe in add some text in section 6.3
fourth paragraph.

12.   In 6.2 example bullet 2 "(VC3) - video capture associated with loudest
room segment." My view is that using the current attribute values all you
can say is that it is one of video captures based on some switching
algorithm as decided but the provider. The consumer has no way of knowing
that this is the loudest speaker.

13.   In 6.2.1 the text says that the scale attribute is optional, I
suggested in my comment 4 to have a value for the case where there is no
scale.

14.   In section 6.3 I think that there is a third type of constrain based
on what is called concept in section 6.1. The provider can send a voice
activated video switch content or switch between the MCs every 10 seconds.

15.   In section 6.3 the text say that " The second type of limitation
reflects the encoding resources available - bandwidth and
macroblocks/second". This information is available in the SDP description
and the CLUE protocol should not duplicate information.

16.   The fourth paragraph of section 6.3 "Simultaneous transmission sets
are expressed as sets of the MCs" I suggest changing "sets" to "entries of
capture sets".

17.   Section 7 talks about encoding. My view is that these constrains are
already defined in the respective m-lines in the SDP. For example the first
paragraph of section 7.1 describes the SDP and codec parameters so why do we
need to duplicate the individual encoding which is also codec specific, see
section 10 talking about new encoding parameters for new codecs. My
suggestion is to add a correlation between the SDP information and the media
capture that is sending the specific media stream.  The same goes for the
group information in section 7.2, at least to the bandwidth parameter which
just repeats the logic defined in RFC4566. 

18.   Figure 1 in section 7.2 is not clear, what is the relation of the
proposed encoding groups to the media captures.

19.   The last sentence in 7.2 "There is no requirement for all encodings
within an encoding group to be instantiated at once" is not clear to me with
respect to the rest of the section.

20.   In section 8 "Every media capture is associated with an encoding
group" is it a media capture or a capture set entry?

21.   I read section 8 and most of it is not clear. The last sentence is the
only sentence that is needed.

22.   The first paragraph of section 9 demonstrate the need to associate the
MCs with the SDP since by choosing a MC the consumer must also receive the
RTP media associated with it.

23.   In section 9 I suggest changing the "has" in the last sentence of the
first paragraph to a "may" . A provider may be able to support the capture
set entries without any limitations on the encoding beside the ones
specified in the SDP.

24.   Section 9 fourth paragraph describes an exchange similar to offer
answer. I think that even if the consumer current configuration is not
affected by the new advertisement, the consumer still need to acknowledge
that he received the new advertisement and understands it.

25.    Section 9.4 discusses the message flow. I think that two messages are
enough preferably the last two. This will enable the CLUE protocol to work
with offer/answer for example for codec parameters like H.264 RFC6184.

26.   In the example in section 11.1 there is a text definition for VC3 and
VC4 but the current attribute do not make it explicit to the consumer for
example without changing the attribute I can write VC3 (switch between the
cameras every 10 second) and VC4 (a side by side composition of two out of
the three cameras with the last two speakers)

27.   In section 11.1 should AC3 also have composed attribute otherwise it
may be a mix or a switch.

28.   In section 11.1 there are two simultaneous information sets.

The physical simultaneity information is:

      {VC0, VC1, VC2, VC3, VC4, VC6}

      {VC0, VC2, VC5, VC6}

And the capture sets

                            +--------------------+

                            | Capture Set #1 |

                            +---------------------+

                            | VC0, VC1, VC2  |

                            | VC3                     |

                            | VC4                    |

                            | VC5                    |

                            | AC0, AC1, AC2  |

                            | AC3                     |

                            +--------------------+

                            +--------------------+

                            | Capture Set #2 |

                            +--------------------+

                            | VC6                    |

                            | AC4                    |

                            +--------------------+

Are both being specified in the data model? The Capture sets are subset of
the physical simultaneity information. So why need both.

Thanks

Roni Even

---

From: "Duckworth, Mark" <[email-masked]>
Date: Mon, 13 Feb 2012 11:30:44 -0800

Hi Roni,

I think your example is this:
A provider wants to advertise that it can provide a capture with voice activated switching, or it can provide a capture that switches every 10 seconds, but not both at the same time, is that right?

This is related to the ticket #7 discussion of how to distinguish between these two different switching policies.  That discussion leads to two possibilities of how a provider can advertise this:

Possibility 1: advertise a single media capture with a "switch policy" attribute with (loudest talker, round robin) values, and the consumer can pick which one it wants.

Possibility 2: advertise a media capture with a "switch policy" attribute with (loudest talker); and advertise another media capture with a "switch policy" attribute with (round robin).  Then to indicate it can't send them both at once, the provider must also advertise simultaneous sets that do not have these two media captures in the same set, or it can advertise them both with the same encoding group that has only a single individual encoding in it.

Possibility 1 seems a lot simpler for both provider and consumer to deal with.  Does this address your idea about this constraint?

Mark

===========================================
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Wednesday, February 08, 2012 8:49 AM
To: [email-masked]
Subject: [clue] Review of draft-ietf-clue-framework-03

14. In section 6.3 I think that there is a third type of constrain based on what is called concept in section 6.1. The provider can send a voice activated video switch content or switch between the MCs every 10 seconds.

Thanks
Roni Even

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Tue, 14 Feb 2012 21:18:35 -0800

Hi Roni, 

Content meant what the audio or video is focused on. I suggest removing
this sentence; it was a side comment which, if controversial, is not
interesting.

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Roni Even
Sent: Wednesday, February 08, 2012 5:49 AM
To: [email-masked]
Subject: [clue] Review of draft-ietf-clue-framework-03

1.       In section 4, "This constitutes a significant change from
previous video conferencing  systems in which transmission of content
was determined primarily by  the sender." I am not sure what is
"content" here, is it the actual data or the semantics of the stream. I
also do not believe that this statement is true.

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Wed, 15 Feb 2012 08:06:07 -0800

Hi Roni,

There is already an attribute that correlates the VC with the encoding
group, so that the consumer can know which Encodings it can potentially
use for this VC.  If the Encodings are carried in SDP, I believe the
connection you are asking for will be made.

If there were no encoding, as you suggest in another comment, then there
would need to be a link between the capture and the media attribute, as
you here suggest. I don't think the SSRC is appropriate because it may
change, the mline in SDP would seem more appropriate.

Regards,

Allyn

Hi,

1.       In section 6.1.1 I suggest adding an attribute that will
correlate between the MC and the SDP m-line or SSRC of this media
stream.

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Tue, 21 Feb 2012 17:41:22 -0800

Hi Roni,

Some responses below on 3 of your comments about encodings and encoding
groups.

Best regards,

Allyn

Hi Roni,

The constraints expressed by the encoding group aren't duplicating
what's already in the SDP - true, the parameters may be superficially
the same, but the m lines in the SDP refer to the whole call, whereas
the CLUE constraints break down this total capability into what can be
done in multiple, separate chunks.  This is not already expressed in
SDP.

The text could be changed to:

" The second type of limitation reflects the encoding resources
available - bandwidth and macroblocks/second. This type of constraint
refines the existing call-level SDP parameters, and gives the consumer
information on how those overall limitations can be subdivided when
instantiating multiple individual encodings."

15.   In section 6.3 the text say that " The second type of limitation
reflects the encoding resources available - bandwidth and
macroblocks/second". This information is available in the SDP
description and the CLUE protocol should not duplicate information.

With reference to your comment below, I do not believe that the
encodings described in CLUE are already in SDP. When you say, 

My suggestion is to add a correlation between the SDP information and
the media capture that is sending the specific media stream.

That is fine, once the consumer has selected one or more of the possible
media streams for each capture that it wants.

But currently SDP does not allow expression of groups of potential
encodings from the sender in which each group is subject to a group
constraint (the encoding group parameters), and from which the consumer
can select. I am not saying it could not be said in SDP,though it would
probably require some extensions, but such groups of individual
encodings are not currently expressed in SDP. So, I don't think that
there is duplication between the constraints currently expressed in SDP
and the encodings and encoding groups in the CLUE framework.

17.   Section 7 talks about encoding. My view is that these constrains
are already defined in the respective m-lines in the SDP. For example
the first paragraph of section 7.1 describes the SDP and codec
parameters so why do we need to duplicate the individual encoding which
is also codec specific, see section 10 talking about new encoding
parameters for new codecs. My suggestion is to add a correlation between
the SDP information and the media capture that is sending the specific
media stream.  The same goes for the group information in section 7.2,
at least to the bandwidth parameter which just repeats the logic defined
in RFC4566. 

I think there always needs to be an Encoding Group ID in the media
capture so that the consumer knows from which pool to pull streams for
the capture.  So I guess I don't see the "may", or how a provider can
support capture set entries without encoding group limitations bounding
the resources on the streams with which the captures are instantiated.
The way the framework is written now, there is always a need for the
encoding group and encoding to instantiate a potential stream with a
capture

23.   In section 9 I suggest changing the "has" in the last sentence of
the first paragraph to a "may" . A provider may be able to support the
capture set entries without any limitations on the encoding beside the
ones specified in the SDP.

---

From: "Roni Even" <[email-masked]>
Date: Thu, 23 Feb 2012 17:04:19 +0200

Hi Allyn,

I will try to explain why it works and what is the problem in the multipoint
case using the solution in the multistream conference  draft.

I am basing my description on the first signaling alternative discussed in
the interim meeting with the mapping described in
http://tools.ietf.org/html/draft-even-clue-rtp-mapping-00 .

In the point to point  for a 3 camera/monitor system case the CLUE proposal
may include three individual streams each from each camera (VC1, VC2 and
VC3) and a VC4 which is the loudest speaker switched. The SDP will include
the SDP parameters for the VC1, VC2 , VC3, VC4 and if using SSRC multiplex
with BUNDLE  and including the SSRC attribute for each stream, the receiver
can identify based on the SSRC which stream it is currently receiving.

In the multipoint case, the question will be if the MCU will use his own
ssrc when sending streams to receiving acting similar to an RTP mixer. (note
that an RTP mixer can switch source but use its own ssrc). In this case the
conference event package can be used to know whose media are received. Again
a full SDP will be exchanged between the consumer and the provider. I
believe that your view is based on the RTP translator model where the SSRC
will be the original ones and sending partial codec information as specified
by the individual encodes to all parties. I am not sure that this method
will work without having an offer answer (RFC3264) between the parties
unless there is just one codec in fixed configuration defined for CLUE . The
consumer need to know which codec it is receiving (h.264, H.263, H.265,
VP8,..) in order to know that it can support it including a specific
configuration like for H.264 which profile. A provider may be able to send
H.264 and H.263 and if there are some consumers that support H.264 and all
support H.263 how will he know to send H.263 and with which annexes. This
will be worse when we will start using H.265. This is without addressing the
issue of SBCs in the middle.

Roni Even

From: Allyn Romanow (allyn) [mailto:[email-masked]] 
Sent: Wednesday, February 22, 2012 3:41 AM
To: Roni Even; [email-masked]
Subject: RE: [clue] 3 comments on encodings and encoding groups --Review of
draft-ietf-clue-framework-03

Hi Roni,

Best regards,

Allyn

From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni
Even
Sent: Wednesday, February 08, 2012 5:49 AM
To: [email-masked]
Subject: [clue] Review of draft-ietf-clue-framework-03

Hi Roni,

The constraints expressed by the encoding group aren't duplicating what's
already in the SDP - true, the parameters may be superficially the same, but
the m lines in the SDP refer to the whole call, whereas the CLUE constraints
break down this total capability into what can be done in multiple, separate
chunks.  This is not already expressed in SDP.

" The second type of limitation reflects the encoding resources available -
bandwidth and macroblocks/second. This type of constraint refines the
existing call-level SDP parameters, and gives the consumer information on
how those overall limitations can be subdivided when instantiating multiple
individual encodings."

 

With reference to your comment below, I do not believe that the encodings
described in CLUE are already in SDP. When you say, 

My suggestion is to add a correlation between the SDP information and the
media capture that is sending the specific media stream.

But currently SDP does not allow expression of groups of potential encodings
from the sender in which each group is subject to a group constraint (the
encoding group parameters), and from which the consumer can select. I am not
saying it could not be said in SDP,though it would probably require some
extensions, but such groups of individual encodings are not currently
expressed in SDP. So, I don't think that there is duplication between the
constraints currently expressed in SDP and the encodings and encoding groups
in the CLUE framework.

 

I think there always needs to be an Encoding Group ID in the media capture
so that the consumer knows from which pool to pull streams for the capture.
So I guess I don't see the "may", or how a provider can support capture set
entries without encoding group limitations bounding the resources on the
streams with which the captures are instantiated. The way the framework is
written now, there is always a need for the encoding group and encoding to
instantiate a potential stream with a capture