Subject: [AVTCORE] Comments on draft-westerlund-avtcore-multistream-and-simulcast-00

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Tue, 26 Jul 2011 22:31:51 +0000

Hi Magnus,

Reading your draft 'RTP Multiple Stream Sessions and Simulcast', I was wondering whether the proposed simulcast extensions could allow for RTP to be used for the delivery of spatially segmented content.

Spatial segmentation is the concept of spatially dividing a video up into multiple segments, each being independently decodable. It can for example be used to efficiently deliver video to clients who are only interested in specific areas of a video. Such bandwidth-saving are especially important when dealing with extremely high resolutions.

Depending on the application, the recombination of segments into a single video might take place on the client or on a network node (such as an RTP mixer). Your simulcast extensions might be a good way to more effectively deal with the large number of segment streams being received by such a node.

I think it would be interesting to see whether we can expand the scope of your draft to include the specific properties of spatially segmented content.

Cheers,

Ray van Brandenburg
This e-mail and its contents are subject to the DISCLAIMER at http://www.tno.nl/emaildisclaimer

---

From: Stephen Botzko <[email-masked]>
Date: Tue, 26 Jul 2011 22:19:25 -0400

Such spatial relationships are CLUE's scope.

On Tue, Jul 26, 2011 at 6:31 PM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

>  Hi Magnus,****
>
> ** **
>
> Reading your draft 'RTP Multiple Stream Sessions and Simulcast', I was
> wondering whether the proposed simulcast extensions could allow for RTP to
> be used for the delivery of spatially segmented content. ****
>
> ** **
>
> Spatial segmentation is the concept of spatially dividing a video up into
> multiple segments, each being independently decodable. It can for example be
> used to efficiently deliver video to clients who are only interested in
> specific areas of a video. Such bandwidth-saving are especially important
> when dealing with extremely high resolutions. ****
>
> ** **
>
> Depending on the application, the recombination of segments into a single
> video might take place on the client or on a network node (such as an RTP
> mixer). Your simulcast extensions might be a good way to more effectively
> deal with the large number of segment streams being received by such a node.
> ****
>
> ** **
>
> I think it would be interesting to see whether we can expand the scope of
> your draft to include the specific properties of spatially segmented
> content.****
>
> ** **
>
> Cheers,****
>
> ** **
>
> Ray van Brandenburg****
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>
>

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Wed, 27 Jul 2011 14:38:27 +0000

Let me further clarify what I mean by spatial segmentation.

In the most general case, you would have a single camera source that records in a very high resolution (e.g. 4K or higher). Instead of encoding the recorded images as a single video stream, the camera spatially segments each video frame and encodes the results into multiple video streams (e.g. 4 or 16) each representing a specific area of the recorded video. The result would be an RTP sender that makes available 16 separate streams.

Depending on the area a user is interested in (the region-of-interest), an RTP receiver (or mixer) would set up one or more sessions to receive the one or more streams that are necessary to reconstruct the desired region-of-interest. The result would be that different RTP receivers would simultaneously receive different subsets of the available segment streams.

While some of the desired functionality might lie in the CLUE area (which I must admit I am not that familiar with), I think there is a considerable overlap with the use cases described in the Multistream/Simulcast draft.

Cheers,

Ray van Brandenburg

From: Stephen Botzko [mailto:[email-masked]]
Sent: woensdag 27 juli 2011 4:19
To: Brandenburg, R. (Ray) van
Cc: [email-masked]; [email-masked]
Subject: Re: [AVTCORE] Comments on draft-westerlund-avtcore-multistream-and-simulcast-00

Such spatial relationships are CLUE's scope.
On Tue, Jul 26, 2011 at 6:31 PM, Brandenburg, R. (Ray) van <[email-masked]<mailto:[email-masked]>> wrote:
Hi Magnus,

Cheers,

Ray van Brandenburg

_______________________________________________
Audio/Video Transport Core Maintenance
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/avt

---

From: Stephen Botzko <[email-masked]>
Date: Wed, 27 Jul 2011 13:14:44 -0400

Defining spatial relationships between RTP video sources is one reason CLUE
was formed.

Although Magnus explicitly used a telepresence system as one of his use
cases this morning, he made it clear that the side information needed to
define the spatial relationships was not part of his intended scope.

Stephen Botzko

On Wed, Jul 27, 2011 at 10:38 AM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

>  Let me further clarify what I mean by spatial segmentation. ****
>
> ** **
>
> In the most general case, you would have a single camera source that
> records in a very high resolution (e.g. 4K or higher). Instead of encoding
> the recorded images as a single video stream, the camera spatially segments
> each video frame and encodes the results into multiple video streams (e.g. 4
> or 16) each representing a specific area of the recorded video. The result
> would be an RTP sender that makes available 16 separate streams.****
>
> ** **
>
> Depending on the area a user is interested in (the region-of-interest), an
> RTP receiver (or mixer) would set up one or more sessions to receive the one
> or more streams that are necessary to reconstruct the desired
> region-of-interest. The result would be that different RTP receivers would
> simultaneously receive different subsets of the available segment streams.
> ****
>
> ** **
>
> While some of the desired functionality might lie in the CLUE area (which I
> must admit I am not that familiar with), I think there is a considerable
> overlap with the use cases described in the Multistream/Simulcast draft. *
> ***
>
> ** **
>
> Cheers,****
>
> ** **
>
> Ray van Brandenburg****
>
> ** **
>
> ** **
>
> *From:* Stephen Botzko [mailto:[email-masked]]
> *Sent:* woensdag 27 juli 2011 4:19
> *To:* Brandenburg, R. (Ray) van
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [AVTCORE] Comments on
> draft-westerlund-avtcore-multistream-and-simulcast-00****
>
> ** **
>
> Such spatial relationships are CLUE's scope.****
>
> [email-masked]> wrote:****
>
> Hi Magnus,****
>
>  ****
>
> This e-mail and its contents are subject to the DISCLAIMER at
> http://www.tno.nl/emaildisclaimer****
>
>
> https://www.ietf.org/mailman/listinfo/avt****
>
> ** **
>

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 27 Jul 2011 13:23:38 -0400

Hi Ray,

As you describe this case, I would say that it would be very similar to
having for example 3 cameras covering adjacent fields of view. The
spatial relation between the streams are clearly a CLUE question, not a
AVTCORE one.

Cheers

Magnus

On 2011-07-27 10:38, Brandenburg, R. (Ray) van wrote:
> records in a very high resolution (e.g. 4K or higher). Instead of
> encoding the recorded images as a single video stream, the camera
> spatially segments each video frame and encodes the results into
> multiple video streams (e.g. 4 or 16) each representing a specific area
> of the recorded video. The result would be an RTP sender that makes
> available 16 separate streams.
> 
>  
> 
> Depending on the area a user is interested in (the region-of-interest),
> an RTP receiver (or mixer) would set up one or more sessions to receive
> the one or more streams that are necessary to reconstruct the desired
> region-of-interest. The result would be that different RTP receivers
> would simultaneously receive different subsets of the available segment
> streams.
> 
>  
> 
> While some of the desired functionality might lie in the CLUE area
> (which I must admit I am not that familiar with), I think there is a
> considerable overlap with the use cases described in the
> Multistream/Simulcast draft.
> 

-- 

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Wed, 27 Jul 2011 17:53:01 +0000

I agree.

What I meant was that in the case of spatially segmented content and thus receivers receiving multiple streams, you run into the same kind of difficulties as you described in your draft: when to use multiple sessions, when to multiplex multiple streams into one session. 

Furthermore, for spatially segmented content it might be especially useful to have multiple versions of the same stream available (in different resolutions for example) and some form of RTP Simulcast might thus be useful.

Ray

-----Original Message-----
From: Magnus Westerlund [mailto:[email-masked]] 
Sent: woensdag 27 juli 2011 19:24
To: Brandenburg, R. (Ray) van
Cc: Stephen Botzko; [email-masked]
Subject: Re: [AVTCORE] Comments on draft-westerlund-avtcore-multistream-and-simulcast-00

Hi Ray,

As you describe this case, I would say that it would be very similar to having for example 3 cameras covering adjacent fields of view. The spatial relation between the streams are clearly a CLUE question, not a AVTCORE one.

Cheers

Magnus

On 2011-07-27 10:38, Brandenburg, R. (Ray) van wrote:
> multiple video streams (e.g. 4 or 16) each representing a specific 
> area of the recorded video. The result would be an RTP sender that 
> makes available 16 separate streams.
> 
>  
> 
> Depending on the area a user is interested in (the 
> region-of-interest), an RTP receiver (or mixer) would set up one or 
> more sessions to receive the one or more streams that are necessary to 
> reconstruct the desired region-of-interest. The result would be that 
> different RTP receivers would simultaneously receive different subsets 
> of the available segment streams.
> 
>  
> 

-- 

Magnus Westerlund