Subject: [clue] Comment on requirements-03 - aspect ratio

From: "Elwell, John" <[email-masked]>
Date: Wed, 15 Jun 2011 18:30:24 +0200

We have:
"REQMT-1:   The solution MUST support a description of the spatial
              arrangement of source video images sent in video streams
              which enables a satisfactory reproduction at the receiver
              of the original scene.  This applies to each site in a
              point to point or a multipoint meeting and refers to the
              spatial ordering within a site, not to the ordering of
              images between sites."
and below that:
"REQMT-1c:  The solution MUST support a means to
                         communicate the aspect ratio."
Is this the aspect ratio of a particular camera? If so, what is the relevance of this to REQMT-1, which is concerned with the spatial ordering of video images? Also REQMT-6 addresses aspect ratios, so how does REQMT-1c relate to REQMT-6?

John

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Sun, 19 Jun 2011 16:45:18 -0700

Hi John,
Thanks for your comments. 
I agree with them.

Aspect ratio is not part of ordering, and should be a separate
requirement, not part of Reqmt 1. Reqmt 6 which says The solution MUST
support means of enabling
              interoperability between telepresence endpoints where
              cameras are of different aspect ratios.
Certainly includes supporting communicating information about aspect
ratio, which is what 1c says.

Also, 1d, on supporting multi-view, as described in use cases, could be
pulled out of 1 and made a separate requirement as well. 

I propose making these changes. How is that?

Regards,
Allyn

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
Of
> Elwell, John
> Sent: Wednesday, June 15, 2011 9:30 AM
> To: [email-masked]
> Subject: [clue] Comment on requirements-03 - aspect ratio
> 
> We have:
> Is this the aspect ratio of a particular camera? If so, what is the
> relevance of this to REQMT-1, which is concerned with the spatial
> ordering of video images? Also REQMT-6 addresses aspect ratios, so how
> does REQMT-1c relate to REQMT-6?
> 
> John
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue

---

From: "Elwell, John" <[email-masked]>
Date: Mon, 20 Jun 2011 09:53:05 +0200

Allyn,

Removal of 1c is fine.

Concerning 1d, I searched for multi-view in the rest of the document and in the use cases, but couldn't find it, so if we retain it in the form of a separate requirement, it would certainly need more explanation.

John 

> From: Allyn Romanow (allyn) [mailto:[email-masked]] 
> Sent: 20 June 2011 00:45
> To: Elwell, John; [email-masked]
> Subject: RE: [clue] Comment on requirements-03 - aspect ratio
> 
> Hi John,

---

From: Stephen Botzko <[email-masked]>
Date: Mon, 20 Jun 2011 05:58:45 -0400

in-line

Stephen Botzko

On Mon, Jun 20, 2011 at 3:53 AM, Elwell, John <
[email-masked]> wrote:

> Allyn,
>
> Removal of 1c is fine.
>
> I think the proposal is to move it, not remove it altogether.

> Concerning 1d, I searched for multi-view in the rest of the document and in
> the use cases, but couldn't find it, so if we retain it in the form of a
> separate requirement, it would certainly need more explanation.
>
> There was a use case put on the list about a month ago by Mark Gorzynski.
I think it would be useful to add it to the use cases draft.
1. Virtual Space Multipoint

This use case describes a virtual space multipoint video meeting with good
eye contact and spatial layout of participants.  The use case was proposed
very early in the development of video conferencing systems as described in
1983 by Allardyce and Randal (http://handle.dtic.mil/100.2/ADA127738  ).
The use case is illustrated in figure 2-5 of their report.  Today, the use
case is implemented in multiple telepresence type video conferencing systems
on the market.  The term “virtual space” was used in their report.  The main
difference between the result obtained with modern systems and those from
1983 are larger display sizes.

Virtual space multipoint as defined here assumes endpoints with multiple
cameras and displays.  Usually there are the same number of cameras and
displays at a given endpoint.  A camera is positioned above each display.  A
key aspect of virtual space multipoint is the details of how the cameras are
aimed.  The cameras are each aimed on the same area of view of the
participants at the site.  Thus each camera takes a picture of the same set
of people but from a different angle.  Each endpoint sender in the virtual
space multipoint meeting therefore offers a choice of video streams to
remote receivers, each stream representing a different view point.  For
example a camera positioned above a display to a participant’s left may take
video pictures of the participant’s left ear while at the same time, a
camera positioned above a display to the participant’s right may take video
pictures of the participant’s right ear.

Since a sending endpoint has a camera associated with each display, an
association is made between the receiving stream output on a particular
display and the corresponding sending stream from the camera associated with
that display.  These associations are repeated for each display/camera pair
in a meeting.  The result of this system is a horizontal arrangement of
video images from remote sites, one per display.  The image from each
display is paired with the camera output from the camera above that display
resulting in excellent eye contact.

---

From: "Espen Berger (espeberg)" <[email-masked]>
Date: Mon, 20 Jun 2011 14:04:23 +0200

I did not fully understand the reason for REQ-6, REQ-7 and REQ-8. These
requirements are already covered in current decode rules for H.264
video. A decoder: 
 - should decode any aspect-ratio below the frame-size it can receive 
 - should decode any resolution below the max frame size 
 - The renderer is free to chose display size

In addition to frame-size and aspects-ratio there are a large number of
other parameters that could be negotiated in SDP and I do not think we
should duplicate those in the CLUE-requirement documentation. 

Should we drop REQ 6, 7, 8 and instead refer to SDP mechanisms for
negotiating video and audio parameters?

Cheers 

-Espen 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Allyn Romanow (allyn)
Sent: 20. juni 2011 01:45
To: Elwell, John; [email-masked]
Subject: Re: [clue] Comment on requirements-03 - aspect ratio

Hi John,

Regards,
Allyn

Of

---

From: "Elwell, John" <[email-masked]>
Date: Mon, 20 Jun 2011 14:55:30 +0200

> From: Stephen Botzko [mailto:[email-masked]] 
> Sent: 20 June 2011 10:59
> To: Elwell, John
> Cc: Allyn Romanow (allyn); [email-masked]
> On Mon, Jun 20, 2011 at 3:53 AM, Elwell, John 
> <[email-masked]> wrote:
> 
> 
> 	Allyn,
> 	
> 	Removal of 1c is fine.
> 	
> 	
> 
[JRE] I thought the proposal was that the existing REQMT-6 already covers it, and hence 1c can just be deleted.

John

>  
> 
> 
> 	Concerning 1d, I searched for multi-view in the rest of 
> the document and in the use cases, but couldn't find it, so 
> if we retain it in the form of a separate requirement, it 
> would certainly need more explanation.
> 	
> 	
> 
> There was a use case put on the list about a month ago by 
> Mark Gorzynski.  I think it would be useful to add it to the 
> use cases draft.
> >>>
> This use case describes a virtual space multipoint video 
> meeting with good eye contact and spatial layout of 
> participants.  The use case was proposed very early in the 
> development of video conferencing systems as described in 
> 1983 by Allardyce and Randal 
> (http://handle.dtic.mil/100.2/ADA127738  ).  The use case is 
> illustrated in figure 2-5 of their report.  Today, the use 
> case is implemented in multiple telepresence type video 
> conferencing systems on the market.  The term "virtual space" 
> was used in their report.  The main difference between the 
> result obtained with modern systems and those from 1983 are 
> larger display sizes.
> 
>  
> 
> Virtual space multipoint as defined here assumes endpoints 
> with multiple cameras and displays.  Usually there are the 
> same number of cameras and displays at a given endpoint.  A 
> camera is positioned above each display.  A key aspect of 
> virtual space multipoint is the details of how the cameras 
> are aimed.  The cameras are each aimed on the same area of 
> view of the participants at the site.  Thus each camera takes 
> a picture of the same set of people but from a different 
> angle.  Each endpoint sender in the virtual space multipoint 
> meeting therefore offers a choice of video streams to remote 
> receivers, each stream representing a different view point.  
> For example a camera positioned above a display to a 
> participant's left may take video pictures of the 
> participant's left ear while at the same time, a camera 
> positioned above a display to the participant's right may 
> take video pictures of the participant's right ear. 
> 
>  
> 
> Since a sending endpoint has a camera associated with each 
> display, an association is made between the receiving stream 
> output on a particular display and the corresponding sending 
> stream from the camera associated with that display.  These 
> associations are repeated for each display/camera pair in a 
> meeting.  The result of this system is a horizontal 
> arrangement of video images from remote sites, one per 
> display.  The image from each display is paired with the 
> camera output from the camera above that display resulting in 
> excellent eye contact.
> >>>
>  
> 
> 	John
> 	
> 
> 	> >               which enables a satisfactory 
> reproduction at the
> 	> receiver

---

From: "Allyn Romanow (allyn)" <[email-masked]>
Date: Mon, 20 Jun 2011 06:46:10 -0700

Yes, I believe it is covered by 6

-----Original Message-----
From: Elwell, John [mailto:[email-masked]] 
Sent: Monday, June 20, 2011 5:56 AM
To: Stephen Botzko
Cc: Allyn Romanow (allyn); [email-masked]
Subject: RE: [clue] Comment on requirements-03 - aspect ratio

> I think the proposal is to move it, not remove it altogether.
[JRE] I thought the proposal was that the existing REQMT-6 already
covers it, and hence 1c can just be deleted.

John

---

From: Stephen Botzko <[email-masked]>
Date: Mon, 20 Jun 2011 10:10:59 -0400

Aspect Ratio creates some unique issues with multiple displays, especially
if the aspect ratio of the received pictures do not match the physical
displays.  This is shown in the use cases.  So the H.264 decoder rules are
not sufficient to get the behavior we want here.

Similarly, the solution needs to enable display of far-end participants at
the proper apparent size.  Again, H.264 decoder rules do not cover this.
Requirements 6-8  were intended to fill the gap.  However, we seem to have
lost the "actual size" display (called "full-sized display" in the use cases
draft) in the current formulation.  This needs to be fixed, as the ability
to match the visual scale of the incoming video is an important aspect of
the solution.

Regards,
Stephen Botzko

On Mon, Jun 20, 2011 at 8:04 AM, Espen Berger (espeberg) <[email-masked]
> wrote:

---

From: "Espen Berger (espeberg)" <[email-masked]>
Date: Tue, 21 Jun 2011 14:00:57 +0200

We can add support for full-size display a sub-point to REQ-1. 

-Espen 

From: Stephen Botzko [mailto:[email-masked]] 
Sent: 20. juni 2011 16:11
To: Espen Berger (espeberg)
Cc: Allyn Romanow (allyn); Elwell, John; [email-masked]
Subject: Re: [clue] Comment on requirements-03 - aspect ratio

Aspect Ratio creates some unique issues with multiple displays,
especially if the aspect ratio of the received pictures do not match the
physical displays.  This is shown in the use cases.  So the H.264
decoder rules are not sufficient to get the behavior we want here.

Similarly, the solution needs to enable display of far-end participants
at the proper apparent size.  Again, H.264 decoder rules do not cover
this.  Requirements 6-8  were intended to fill the gap.  However, we
seem to have lost the "actual size" display (called "full-sized display"
in the use cases draft) in the current formulation.  This needs to be
fixed, as the ability to match the visual scale of the incoming video is
an important aspect of the solution.

Regards,
Stephen Botzko

On Mon, Jun 20, 2011 at 8:04 AM, Espen Berger (espeberg)
<[email-masked]> wrote:

Cheers

-Espen

-----Original Message-----

Allyn Romanow (allyn)

Hi John,

Regards,
Allyn

Of