Subject: [MMUSIC] Remarks from today

From: Bert Greevenbosch <[email-masked]>
Date: Fri, 01 Apr 2011 11:34:22 +0000

Hi all,

Thank you for your remarks during today's session. Here are the remarks I received concerning the 3D Format and Parallax Signalling drafts:

Parallax Signalling draft

  *   Why use unit "pixels"? Maybe should think of format that is more general for different display sizes.
  *   Is the timed 3D text meant to overlay video?

3D Format draft

  *   What is the relationship between the 3D Format draft and the RTP transport of MVC draft?
  *   What is the relationship to the adjacent-grouping draft (and CLUE)?
  *   Why signal it in SDP, when it can also be signalled in the codec?
  *   Should there be a central registry for this kind of signalling?

Please let me know if I missed some questions/comments, or if you have any new questions/comments on the drafts.

I plan to come back with detailed responses on the list next week.

Best regards,
Bert

---
Bert Greevenbosch M.Sc.
Senior Research and Standardisation Engineer

email: [email-masked]
telephone: +86-755-28978088

HUAWEI TECHNOLOGIES CO.,LTD.
Building F1 / Floor 8
Huawei Industrial Base
Bantian, Longgang District
Shenzhen  518129
P.R. China

*******************************************************************************
This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it!
*******************************************************************************

---

From: [email-masked]
Date: Fri, 1 Apr 2011 08:40:32 -0400

> Thank you for your remarks during today's session. Here are the remarks I received concerning the 3D Format and Parallax Signalling drafts:
[...]
> Please let me know if I missed some questions/comments, or if you have any new questions/comments on the drafts.

I had one other question I was going to ask, but the line had been closed.

What is the backward-compability story here?  It seems like a non-3D-aware
endpoint receiving this SDP message is going to have a lot of problems
understanding what it means.  (Remember that in SDP, unknown attributes are
ignored.)

Packed frames are obviously the most problematic case here, but even the
simulcast and auxiliary modes are potentially unappealing.

-- 
Jonathan Lennox
[email-masked]

---

From: "Roni Even" <[email-masked]>
Date: Sat, 2 Apr 2011 18:41:05 +0300

Hi,
This is why I suggested to create a payload format for it where the payload
header will be the directory for the separate streams. This works good if
the proposal is to let the answerer select all streams or none at all. My
assumption is that all the 3D streams are using the same codec which
simplifies the solution.
If there is one of the streams that must be used by systems not supporting
the multiple m-lines than need to define the offer using media capability
negotiation.

Roni Even

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: Friday, April 01, 2011 3:41 PM
> To: Bert Greevenbosch
> Cc: [email-masked]
> Subject: Re: [MMUSIC] Remarks from today
> 
> > Thank you for your remarks during today's session. Here are the
> remarks I received concerning the 3D Format and Parallax Signalling
> drafts:
> >
> [...]
> >
> > Please let me know if I missed some questions/comments, or if you
> have any new questions/comments on the drafts.
> 
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Bert Greevenbosch <[email-masked]>
Date: Fri, 08 Apr 2011 03:08:59 +0000

Hi Jonathan, all,

Interesting question! Here are some considerations:

For frame compatible 3D video, the backwards compatibility issue consists mainly in that the receiving legacy device will display the both L/R views, in the way they were packed into the video stream. This is not a very pleasing for the eye, but it should not lead to big technical problems.

However, in the simulcast and 2D+aux transmitted as two streams case, the problem is bigger, as there are two "m=video ..." lines, and a choice needs to be made which video to show. In the simulcast case, any of the L- and R-streams can do. But in the 2D+aux case, it would be best if the receiver showed the 2D stream, rather than the auxiliary stream.

In case of SDP offer/answer, the problem is similar to the problem described in the SDP grouping spec (RFC 5888). Section 9.4.2 gives an overview on what will happen when the answerer does not understand the "group" attribute, in which case client behaviour is not clear. According to that text, the client might refuse the offer, in which case it is easy for the offerer to offer a 2D session instead. But if the client does not refuse the offer, it is up to the client to decide how to process the multiple video lines.

Related is the fact, that the "3DS" grouping semantics will not be understood by legacy devices. This is not a new problem, for example, RFC 4756 about Forward Error Correction (FEC) grouping, also describes the problem. In that RFC, there are listed two possible responses:
(1) The answerer returns an answer that ignores the grouping attribute.
(2) The answerer sends a refusal to the request (e.g., 488 Not acceptable here or 606 Not acceptable in SIP).

In the first case, the offerer understands "3DS" grouping is not supported and can send a new offer for only a 2D connection.
In the second case, the offerer can also try to send a new offer advertising for only a 2D connection, although it cannot be sure that the failure was caused by the "3DS" grouping beforehand.

RFC 5583, about the Media Decoding Dependency "MDD" grouping, lists similar behaviour for SDP offer/answer (section 6.1). Section 6.2 addresses the case in which a session is advertised using RTSP. and the legacy device does not understand "MDD" grouping. They differentiate the following cases:
(1) If at least one of the media types included in the SDP is within the receiver's capabilities, it selects among those candidates according to implementation specific criteria for setup, as usual.
(2) If none of the media types included in the SDP can be processed, then obviously no setup can occur.

This would mean, that in case of simulcast L/R streams the receiver will most likely display one of the two streams, which is OK.

But in the 2D+aux case, the receiver might either display the 2D stream or the auxiliary stream.

Best regards,
Bert

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 01 April 2011 20:41
To: Bert Greevenbosch
Cc: [email-masked]
Subject: Re: [MMUSIC] Remarks from today

[...]

-- 
Jonathan Lennox
[email-masked]

---

From: Ted Hardie <[email-masked]>
Date: Fri, 8 Apr 2011 10:51:04 -0700

On Thu, Apr 7, 2011 at 8:08 PM, Bert Greevenbosch <
[email-masked]> wrote:

> For frame compatible 3D video, the backwards compatibility issue consists
> mainly in that the receiving legacy device will display the both L/R views,
> in the way they were packed into the video stream. This is not a very
> pleasing for the eye, but it should not lead to big technical problems.
>
>
Can you clarify this further, or provide a reference? I'm curious in
particular about what you think a legacy device will use as a display
framerate when it's getting signal originally intended for use with shutter
glasses.

regards,

Ted

> However, in the simulcast and 2D+aux transmitted as two streams case, the
> problem is bigger, as there are two "m=video ..." lines, and a choice needs
> to be made which video to show. In the simulcast case, any of the L- and
> R-streams can do. But in the 2D+aux case, it would be best if the receiver
> showed the 2D stream, rather than the auxiliary stream.
>
> In case of SDP offer/answer, the problem is similar to the problem
> described in the SDP grouping spec (RFC 5888). Section 9.4.2 gives an
> overview on what will happen when the answerer does not understand the
> "group" attribute, in which case client behaviour is not clear. According to
> that text, the client might refuse the offer, in which case it is easy for
> the offerer to offer a 2D session instead. But if the client does not refuse
> the offer, it is up to the client to decide how to process the multiple
> video lines.
>
> Related is the fact, that the "3DS" grouping semantics will not be
> understood by legacy devices. This is not a new problem, for example, RFC
> 4756 about Forward Error Correction (FEC) grouping, also describes the
> problem. In that RFC, there are listed two possible responses:
> In the first case, the offerer understands "3DS" grouping is not supported
> and can send a new offer for only a 2D connection.
> In the second case, the offerer can also try to send a new offer
> advertising for only a 2D connection, although it cannot be sure that the
> failure was caused by the "3DS" grouping beforehand.
>
> RFC 5583, about the Media Decoding Dependency "MDD" grouping, lists similar
> behaviour for SDP offer/answer (section 6.1). Section 6.2 addresses the case
> in which a session is advertised using RTSP. and the legacy device does not
> understand "MDD" grouping. They differentiate the following cases:
> (1) If at least one of the media types included in the SDP is within the
> receiver's capabilities, it selects among those candidates according to
> implementation specific criteria for setup, as usual.
> This would mean, that in case of simulcast L/R streams the receiver will
> most likely display one of the two streams, which is OK.
>
> > Thank you for your remarks during today's session. Here are the remarks I
> received concerning the 3D Format and Parallax Signalling drafts:
> >
> [...]
> >
> > Please let me know if I missed some questions/comments, or if you have
> any new questions/comments on the drafts.
>

---

From: Bert Greevenbosch <[email-masked]>
Date: Tue, 12 Apr 2011 02:45:56 +0000

Hi Ted, Jonathan, all,

I see your point. It depends on the parameters that are chosen for the 3D stream, and how the legacy device responds to these parameters. The parameters are frame width, frame height and frame rate.

I think the main advantage of frame packing is that the 3D video can be carried as a regular 2D video stream. Therefore, we can distinguish the following cases:

1.       The 3D frame packed video is transmitted through a 2D video stream with fixed (prescribed) resolution/frame rate.

2.       The 3D frame packed video is transmitted through a 2D video stream with flexible resolution/frame rate.

In case the 3D frame packed video is transmitted through a 2D video stream with a fixed resolution/frame rate, the streaming server has to obey the restrictions of the 2D streaming system. For example, if the 2D streaming system has 1280x720 pixel^2 frames, and a frequency of 60Hz, the 3D video stream can have a resolution of 640x720 pixel^2, 60 Hz in case of side-by-side packing, or 1280x720 pixel^2, 30 Hz in case of frame sequential packing. In either case, there is no problem for a legacy 2D device receiving and displaying the stream, although it will not look good to the spectator.

If the 3D frame packed video is transmitted through a 2D video stream with flexible resolution/frame rate, there is more freedom in choosing the resolution/frame rate of the 3D stream too. The 3D streaming server could choose a higher resolution or frame rate of the 2D video stream, such that the corresponding 3D video has better quality. For example, if a resolution of 1280x720 pixel^2, 60 Hz is required for the 3D stream, the server could give the carrying 2D stream a resolution of 2560x720 pixel^2, 60Hz or 1280x720 pixel^2, 120Hz.

In either case, the legacy 2D receiver needs to understand the parameters of the carrying 2D video stream. It will go wrong when the parameters of the carrying 2D stream do not match the legacy 2D device's capabilities. But this is normal in the 2D case too; for example, if the system has fixed 1280x720 pixel^2, 60Hz, the server will have a problem attempting to the stream with any other resolution or frame rate. I think this issue is independent on whether the signal contains frame packed 3D video or not.

With a flexible resolution/frame rate, problems can still occur, for example when the server trespasses some upper limits in what the system can process. But this, too, I see as independent from the frame packing.

To come back to your question on shutter glasses: I think you mean a situation where a 3D video is frame sequential packed, with a frame rate that matches the frequency of the shutter glasses. Here the above two cases still hold: if the carrying 2D video stream has a fixed frame rate, the 3D video frame rate and shutter glass frequency can only be half the allowed 2D video stream frame rate. Otherwise the 3D stream cannot be carried in the 2D video stream. But if the restrictions on the 2D video stream are met, the 2D device will have no problem with displaying it.

What's your opinion?

Best regards,
Bert

From: Ted Hardie [mailto:[email-masked]]
Sent: 09 April 2011 01:51
To: Bert Greevenbosch
Cc: [email-masked]; [email-masked]
Subject: Re: [MMUSIC] Remarks from today

On Thu, Apr 7, 2011 at 8:08 PM, Bert Greevenbosch <[email-masked]<mailto:[email-masked]>> wrote:
Hi Jonathan, all,

Can you clarify this further, or provide a reference? I'm curious in particular about what you think a legacy device will use as a display framerate when it's getting signal originally intended for use with shutter glasses.

regards,

Ted

Best regards,
Bert

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>]
Sent: 01 April 2011 20:41
To: Bert Greevenbosch
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [MMUSIC] Remarks from today

[...]

--
Jonathan Lennox
[email-masked]<mailto:[email-masked]>
_______________________________________________
mmusic mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mmusic

---

From: Ted Hardie <[email-masked]>
Date: Tue, 12 Apr 2011 08:53:49 -0700

Hi Bert,

Thanks for taking the time to craft such a detailed response.  Some further
questions and comments in-line.

On Mon, Apr 11, 2011 at 7:45 PM, Bert Greevenbosch <
[email-masked]> wrote:

>  Hi Ted, Jonathan, all,
>
>
>
> I see your point. It depends on the parameters that are chosen for the 3D
> stream, and how the legacy device responds to these parameters. The
> parameters are frame width, frame height and frame rate.
>
>
>
> I think the main advantage of frame packing is that the 3D video can be
> carried as a regular 2D video stream. Therefore, we can distinguish the
> following cases:
>
>
>
> 1.       The 3D frame packed video is transmitted through a 2D video
> stream with fixed (prescribed) resolution/frame rate.
>
> 2.       The 3D frame packed video is transmitted through a 2D video
> stream with flexible resolution/frame rate.
>
>
>
> In case the 3D frame packed video is transmitted through a 2D video stream
> with a fixed resolution/frame rate, the streaming server has to obey the
> restrictions of the 2D streaming system. For example, if the 2D streaming
> system has 1280x720 pixel^2 frames, and a frequency of 60Hz, the 3D video
> stream can have a resolution of 640x720 pixel^2, 60 Hz in case of
> side-by-side packing, or 1280x720 pixel^2, 30 Hz in case of frame sequential
> packing. In either case, there is no problem for a legacy 2D device
> receiving and displaying the stream, although it will not look good to the
> spectator.
>
>
>

It seems  like the 30Hz case is pretty unlikely, at least in systems
designed for use with shutter-glasses, since they will halve that rate to
show a different frame to each eye.  In the usual case, the 3D systems use
double the frame rate of the 2D system.  I'm wondering whether even in the
60 Hz case this will look substantially lower quality to the 3D viewer.

> If the 3D frame packed video is transmitted through a 2D video stream with
> flexible resolution/frame rate, there is more freedom in choosing the
> resolution/frame rate of the 3D stream too. The 3D streaming server could
> choose a higher resolution or frame rate of the 2D video stream, such that
> the corresponding 3D video has better quality. For example, if a resolution
> of 1280x720 pixel^2, 60 Hz is required for the 3D stream, the server could
> give the carrying 2D stream a resolution of 2560x720 pixel^2, 60Hz or
> 1280x720 pixel^2, 120Hz.
>
>
>
> In either case, the legacy 2D receiver needs to understand the parameters
> of the carrying 2D video stream. It will go wrong when the parameters of the
> carrying 2D stream do not match the legacy 2D device's capabilities. But
> this is normal in the 2D case too; for example, if the system has fixed
> 1280x720 pixel^2, 60Hz, the server will have a problem attempting to the
> stream with any other resolution or frame rate. I think this issue is
> independent on whether the signal contains frame packed 3D video or not.
>
>
>
> With a flexible resolution/frame rate, problems can still occur, for
> example when the server trespasses some upper limits in what the system can
> process. But this, too, I see as independent from the frame packing.
>
>
>
> To come back to your question on shutter glasses: I think you mean a
> situation where a 3D video is frame sequential packed, with a frame rate
> that matches the frequency of the shutter glasses. Here the above two cases
> still hold: if the carrying 2D video stream has a fixed frame rate, the 3D
> video frame rate and shutter glass frequency can only be half the allowed 2D
> video stream frame rate. Otherwise the 3D stream cannot be carried in the 2D
> video stream. But if the restrictions on the 2D video stream are met, the 2D
> device will have no problem with displaying it.
>
>
>

Do you know what the minimum shutter glass speed is?  I know only know one
manufacturer's system at all well, and my understanding is that the timing
signals which synchronize them are really matching the frame start times to
the shuttering, but don't adjust the frame rates.  Are there glasses which
can modify the rate substantially as well?

regards,

Ted Hardie

> What's your opinion?
>
>
>
> Best regards,
>
> Bert
>
>
>
>
>
> *From:* Ted Hardie [mailto:[email-masked]]
> *Sent:* 09 April 2011 01:51
> *To:* Bert Greevenbosch
> *Cc:* [email-masked]; [email-masked]
>
> *Subject:* Re: [MMUSIC] Remarks from today
>
>
>

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 14 Apr 2011 05:44:58 +0000

Hi Ted, all,

Thank you for your feedback. Here are my thoughts:

>It seems  like the 30Hz case is pretty unlikely, at least in systems designed for use with shutter-glasses, since they will halve that rate to show a different frame to each eye.  In the usual case, the 3D systems use double the frame rate of the 2D system.  I'm wondering whether even in the 60 Hz case this will look substantially lower quality to the 3D viewer.

>Do you know what the minimum shutter glass speed is?

I agree, the 30Hz (per view) case would give a flickering effect. Since normal TVs have a frequency of 60Hz or more, for a similar quality with shutter glasses the frame rate should be 120Hz (=60 Hz/view) or more.

But the receiver can also upsample the received signal to a higher display rate. For example, if it receives the following frame sequence at 30Hz/view:

L1,R1,L2,R2,L3,R3,...,

then it could display at 60Hz/view:

L1,R1,L1,R1,L2,R2,L2,R2,L3,R3,L3,R3,....

The receiver can also use some interpolation technique to generate the frames between Lk and L(k+1).

I think that in praxis the receiving device will process the received stream to make it fit the display system. This allows different types/standards of shutter glasses to be used, independent on the format in which the 3D stream was received. It also allows other display types such as polarisation or autostereoscopic displays.

>I know only know one manufacturer's system at all well, and my understanding is that the timing signals which synchronize them are really matching the frame start times to the shuttering, but don't adjust the frame rates.  Are there glasses which can modify the rate substantially as well?

Can you please explain a bit more about the system you are referring too, especially about not modifying the frame rate? Do you mean the frame rate between the TV screen and the signalling device (that transmits the signal to the shutter glasses) or the frame rate of the video stream as it is received before it is decoded (by the codec)?

Do you think we need to add some text about this to the draft?

Best regards,
Bert

From: Ted Hardie [mailto:[email-masked]]
Sent: 12 April 2011 23:54
To: Bert Greevenbosch
Cc: [email-masked]; [email-masked]
Subject: Re: [MMUSIC] Remarks from today

Hi Bert,

Thanks for taking the time to craft such a detailed response.  Some further questions and comments in-line.
On Mon, Apr 11, 2011 at 7:45 PM, Bert Greevenbosch <[email-masked]<mailto:[email-masked]>> wrote:
Hi Ted, Jonathan, all,

It seems  like the 30Hz case is pretty unlikely, at least in systems designed for use with shutter-glasses, since they will halve that rate to show a different frame to each eye.  In the usual case, the 3D systems use double the frame rate of the 2D system.  I'm wondering whether even in the 60 Hz case this will look substantially lower quality to the 3D viewer.

Do you know what the minimum shutter glass speed is?  I know only know one manufacturer's system at all well, and my understanding is that the timing signals which synchronize them are really matching the frame start times to the shuttering, but don't adjust the frame rates.  Are there glasses which can modify the rate substantially as well?

regards,

Ted Hardie
What's your opinion?

Best regards,
Bert

From: Ted Hardie [mailto:[email-masked]<mailto:[email-masked]>]
Sent: 09 April 2011 01:51
To: Bert Greevenbosch
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

regards,

Ted

Best regards,
Bert

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>]
Sent: 01 April 2011 20:41
To: Bert Greevenbosch
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [MMUSIC] Remarks from today

[...]

---

From: Ted Hardie <[email-masked]>
Date: Thu, 14 Apr 2011 11:48:59 -0700

On Wed, Apr 13, 2011 at 10:44 PM, Bert Greevenbosch <
[email-masked]> wrote:

>  I agree, the 30Hz (per view) case would give a flickering effect. Since
> normal TVs have a frequency of 60Hz or more, for a similar quality with
> shutter glasses the frame rate should be 120Hz (=60 Hz/view) or more.
>
>
>
> But the receiver can also upsample the received signal to a higher display
> rate. For example, if it receives the following frame sequence at 30Hz/view:
>
>
>
> L1,R1,L2,R2,L3,R3,...,
>
>
>
> The receiver can also use some interpolation technique to generate the
> frames between L*k* and L(*k*+1).
>
>
>
> I think that in praxis the receiving device will process the received
> stream to make it fit the display system. This allows different
> types/standards of shutter glasses to be used, independent on the format in
> which the 3D stream was received. It also allows other display types such as
> polarisation or autostereoscopic displays.
>
>
>

My suspicion is that there are probably practical limits to some system's
ability to either upsample or interpolate.  Would it make sense to add
something to the 3dFormat line to indicate what the resulting per-view rate
will be?  So that

a=3dFormat:FP SbS

had an additional parameter:

a=3dFormat:FP SbS 30Hz

*>I know only know one manufacturer's system at all well, and my
> understanding is that the timing signals which synchronize them are really
> matching the frame start times to the shuttering, but don't adjust the frame
> rates.  Are there glasses which can modify the rate substantially as well?
> *
>
>
>
> Can you please explain a bit more about the system you are referring too,
> especially about not modifying the frame rate? Do you mean the frame rate
> between the TV screen and the signalling device (that transmits the signal
> to the shutter glasses) or the frame rate of the video stream as it is
> received before it is decoded (by the codec)?
>
>
>

I probably should have said "shuttering rate", sorry.  I mean that it
matches the shutters to the alternating left and right view, but that the
shuttering rate is consistent (resulting in 60Hz per view, in this case).

regards,

Ted Hardie

---

From: Bert Greevenbosch <[email-masked]>
Date: Fri, 15 Apr 2011 06:21:24 +0000

Hi Ted,

My suspicion is that there are probably practical limits to some system's ability to either upsample or interpolate.  Would it make sense to add something to the 3dFormat line to indicate what the resulting per-view rate will be?  So that

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz

We could also consider using the SDP attribute "framerate" (RFC 4566). This attribute would give the frame rate of the 2D channel stream, but we can write in the draft that in the frame sequential case the frame rate per view is half of it.  Or do you want to indicate the target shutter glass frequency?

Best regards,
Bert

From: Ted Hardie [mailto:[email-masked]]
Sent: 15 April 2011 02:49
To: Bert Greevenbosch
Cc: [email-masked]; [email-masked]
Subject: Re: [MMUSIC] Remarks from today

On Wed, Apr 13, 2011 at 10:44 PM, Bert Greevenbosch <[email-masked]<mailto:[email-masked]>> wrote:
I agree, the 30Hz (per view) case would give a flickering effect. Since normal TVs have a frequency of 60Hz or more, for a similar quality with shutter glasses the frame rate should be 120Hz (=60 Hz/view) or more.

L1,R1,L2,R2,L3,R3,...,

My suspicion is that there are probably practical limits to some system's ability to either upsample or interpolate.  Would it make sense to add something to the 3dFormat line to indicate what the resulting per-view rate will be?  So that

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz

I probably should have said "shuttering rate", sorry.  I mean that it matches the shutters to the alternating left and right view, but that the shuttering rate is consistent (resulting in 60Hz per view, in this case).

regards,

Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Fri, 15 Apr 2011 17:35:47 -0700

Hi,

Some further comments in-line.

On Thu, Apr 14, 2011 at 11:21 PM, Bert Greevenbosch <
[email-masked]> wrote:

>  We could also consider using the SDP attribute "framerate" (RFC 4566).
> This attribute would give the frame rate of the 2D channel stream, but we
> can write in the draft that in the frame sequential case the frame rate per
> view is half of it.
>

The relevant text from RFC 4566 is:

a=framerate:<frame rate>

         This gives the maximum video frame rate in frames/sec.  It is
         intended as a recommendation for the encoding of video data.
         Decimal representations of fractional values using the notation
         "<integer>.<fraction>" are allowed.  It is a media-level
         attribute, defined only for video media, and it is not
         dependent on charset.

While I think it could be re-used here, it seems to me to make more sense to
use a separate indicator.  This is partly because it is more for the
decoder/upsampler/interpolation check (Can I upsample this to something
reasonable?  If not, don't select this choice).  And it's partly because I
worry a bit about the interaction with existing code.

>  Or do you want to indicate the target shutter glass frequency?
>
>
>

No, I think you're right that it's more likely that things will get adjusted
to a standard shutter speed by upsampling than they will change the shutter
speed.

regards,

Ted Hardie

> *Sent:* 15 April 2011 02:49
>
> *To:* Bert Greevenbosch

---

From: Bert Greevenbosch <[email-masked]>
Date: Tue, 19 Apr 2011 02:52:40 +0000

Hi Ted,

If I understand you well, you would like to be able to negotiate the video frequency, such that it can easily be converted to the shutter glass frequency.

So for example, if the shutter glass frequency is 120Hz (60Hz/view), then reasonable video frequencies would be 60Hz, 40Hz, 30Hz, i.e. divisors of 60Hz (or in the Frame Sequential case, 120Hz). In this case it is easy to upsample the signal by repeating each frame sequence {Lk Rk} an integer number of times.

I have a preference to use the "framerate" attribute for this, as adding another rather similar parameter to the "3dFormat" attribute seems somehow double. However, I also see your point in that the "framerate" attribute only indicates a maximum frequency, not a guaranteed frequency. We could solve this in the draft by mandating that in case of 3D streaming, the frequency the video MUST equal the value of the "framerate" attribute, in case the "framerate" attribute is included.

What do you think?

Best regards,
Bert

From: Ted Hardie [mailto:[email-masked]]
Sent: 16 April 2011 08:36
To: Bert Greevenbosch
Cc: [email-masked]; [email-masked]
Subject: Re: [MMUSIC] Remarks from today

Hi,

Some further comments in-line.
On Thu, Apr 14, 2011 at 11:21 PM, Bert Greevenbosch <[email-masked]<mailto:[email-masked]>> wrote:
Hi Ted,

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz
We could also consider using the SDP attribute "framerate" (RFC 4566). This attribute would give the frame rate of the 2D channel stream, but we can write in the draft that in the frame sequential case the frame rate per view is half of it.

a=framerate:<frame rate>

         dependent on charset.
While I think it could be re-used here, it seems to me to make more sense to use a separate indicator.  This is partly because it is more for the decoder/upsampler/interpolation check (Can I upsample this to something reasonable?  If not, don't select this choice).  And it's partly because I worry a bit about the interaction with existing code.

No, I think you're right that it's more likely that things will get adjusted to a standard shutter speed by upsampling than they will change the shutter speed.

regards,

Ted Hardie

Best regards,
Bert

To: Bert Greevenbosch

L1,R1,L2,R2,L3,R3,...,

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz

regards,

Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Thu, 21 Apr 2011 09:35:43 -0700

On Mon, Apr 18, 2011 at 7:52 PM, Bert Greevenbosch <
[email-masked]> wrote:

>  Hi Ted,
>
>
>
> If I understand you well, you would like to be able to negotiate the video
> frequency, such that it can easily be converted to the shutter glass
> frequency.
>
>
>
> So for example, if the shutter glass frequency is 120Hz (60Hz/view), then
> reasonable video frequencies would be 60Hz, 40Hz, 30Hz, i.e. divisors of
> 60Hz (or in the Frame Sequential case, 120Hz). In this case it is easy to
> upsample the signal by repeating each frame sequence {Lk Rk} an integer
> number of times.
>
>
>
> I have a preference to use the "framerate" attribute for this, as adding
> another rather similar parameter to the "3dFormat" attribute seems somehow
> double. However, I also see your point in that the "framerate" attribute
> only indicates a maximum frequency, not a guaranteed frequency. We could
> solve this in the draft by mandating that in case of 3D streaming, the
> frequency the video MUST equal the value of the "framerate" attribute, in
> case the "framerate" attribute is included.
>
>
Well, I agree that this solves the problem.  I'm a little leary of having
the same parameter have two related but distinct meanings, depending on the
context.  But if no one else objects to this, I'm certainly willing to go
along.  I think you are correct that the context is easy enough to
determine.

regards,

Ted Hardie

> *Sent:* 16 April 2011 08:36
>
> *To:* Bert Greevenbosch
> We could also consider using the SDP attribute "framerate" (RFC 4566). This
> attribute would give the frame rate of the 2D channel stream, but we can
> write in the draft that in the frame sequential case the frame rate per view
> is half of it.
>
>
> While I think it could be re-used here, it seems to me to make more sense
> to use a separate indicator.  This is partly because it is more for the
> No, I think you're right that it's more likely that things will get
> adjusted to a standard shutter speed by upsampling than they will change the
> shutter speed.
>
> regards,
>
> Ted Hardie
>
>
>  Best regards,
>
> Bert
>
>
>
>
>

---

From: Stephen Botzko <[email-masked]>
Date: Thu, 21 Apr 2011 13:30:06 -0400

in-line
Stephen Botzko

> along.  I think you are correct that the context is easy enough to

So this means that this video frequency negotiation can't be done for 3D
video conferencing?  Providing application-specific definitions for a
parameter seems like a problem to me, as many stacks / systems support
multiple applications, and good definitions of the application might be hard
to craft.

Are we sure we really need this negotiation to achieve interoperability?
I'm not convinced myself.  If we do, I think a new parameter is a better
approach.

---

From: Bert Greevenbosch <[email-masked]>
Date: Sat, 23 Apr 2011 02:55:56 +0000

Hi Ted, Stephen, all,

I have thought a bit more about the frame rate issue, especially why it is originally defined as a maximum frame rate rather than an exact frame rate.

The main problem with the negotiation of the frame rate is, that it is likely that the server can only provide a limited number of fixed frame rates. It could have the coded video ready for these frame rates, but it would be hard to provide any other frame rate. With the negotiation of a max frame rate, the source can choose a lower frame rate, as it already has the video stream with that frame rate available.

So a fixed frame rate negotiation would only work if there is only a limited number of common shutter glass frequencies and associated video frame rates. In that case the server can have video representations with matching frame rates available.

I think a common shutter glass frequency is 120 Hz (60 Hz/view). Common frame rates are 24 fps, 30 fps, 50 fps, 60 fps. If this is the case in general, negotiation with an exact frame rate can work. Notice that the 24 fps and 50 fps are not divisors of 60 Hz, but by displaying some of the frames one time more than the other frames, it is still relatively easy to convert the frame rates to shutter glass frequencies.

@Stephen: do you want to keep the meaning of the maximum frame rate because you want to be able to choose a lower frame rate, for example because the video is to be multiplexed with other video in the same RTP stream?

@Ted: given the common frame rates in practise, do you think they still need to be negotiated or will the shutter glasses be able to accept all of these? Then we could also add some informative text about this in the draft.

Best regards,
Bert

From: Stephen Botzko [mailto:[email-masked]] 
Sent: 22 April 2011 01:30
To: Ted Hardie
Cc: Bert Greevenbosch; [email-masked]; [email-masked]
Subject: Re: [MMUSIC] Remarks from today

in-line
Stephen Botzko
On Thu, Apr 21, 2011 at 12:35 PM, Ted Hardie <[email-masked]> wrote:
On Mon, Apr 18, 2011 at 7:52 PM, Bert Greevenbosch <[email-masked]> wrote:
Hi Ted,

 

Well, I agree that this solves the problem.  I'm a little leary of having the same parameter have two related but distinct meanings, depending on the context.  But if no one else objects to this, I'm certainly willing to go along.  I think you are correct that the context is easy enough to determine.

regards,
 
So this means that this video frequency negotiation can't be done for 3D video conferencing?  Providing application-specific definitions for a parameter seems like a problem to me, as many stacks / systems support multiple applications, and good definitions of the application might be hard to craft.  

Are we sure we really need this negotiation to achieve interoperability?  I'm not convinced myself.  If we do, I think a new parameter is a better approach.

Ted Hardie

What do you think?
 
Best regards,
Bert

To: Bert Greevenbosch
 
Hi,

Some further comments in-line.
On Thu, Apr 14, 2011 at 11:21 PM, Bert Greevenbosch <[email-masked]> wrote:
Hi Ted,

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz

         dependent on charset.

regards,

Ted Hardie
 
Best regards,
Bert

To: Bert Greevenbosch
 
On Wed, Apr 13, 2011 at 10:44 PM, Bert Greevenbosch <[email-masked]> wrote:
I agree, the 30Hz (per view) case would give a flickering effect. Since normal TVs have a frequency of 60Hz or more, for a similar quality with shutter glasses the frame rate should be 120Hz (=60 Hz/view) or more.

       L1,R1,L2,R2,L3,R3,...,

 

a=3dFormat:FP SbS

a=3dFormat:FP SbS 30Hz

regards,

Ted Hardie

---

From: Stephen Botzko <[email-masked]>
Date: Sat, 23 Apr 2011 07:34:46 -0400

in-line

On Fri, Apr 22, 2011 at 10:55 PM, Bert Greevenbosch <
[email-masked]> wrote:

> Hi Ted, Stephen, all,
>
> I have thought a bit more about the frame rate issue, especially why it is
> originally defined as a maximum frame rate rather than an exact frame rate.
>
> The main problem with the negotiation of the frame rate is, that it is
> likely that the server can only provide a limited number of fixed frame
> rates. It could have the coded video ready for these frame rates, but it
> would be hard to provide any other frame rate. With the negotiation of a max
> frame rate, the source can choose a lower frame rate, as it already has the
> video stream with that frame rate available.
>
I suspect you are thinking mostly about video streaming of stored media.

There are video sources where you get substantially lower bit rates if you
use a dynamic frame rate when encoding - particularly PC screens which often
are relatively unchanging.  Digital signage might have similar
characteristics.

In video conferencing (or other real-time encoding applications) it is
common for encoders to drop frames during heavy motion or scene changes in
order to maintain a bit rate constraint (trading off image sharpness for
motion handling).

If the video encoding is layered (for instance H.264 Annex G), it can be
routed through a MANE.  In this case, the frame rate (and the spatial
resolution) are dynamically adjusted by the MANE in response to network
conditions, with different receivers getting different resolutions and frame
rates.

>
> So a fixed frame rate negotiation would only work if there is only a
> limited number of common shutter glass frequencies and associated video
> frame rates. In that case the server can have video representations with
> matching frame rates available.
>
> I think a common shutter glass frequency is 120 Hz (60 Hz/view). Common
> frame rates are 24 fps, 30 fps, 50 fps, 60 fps. If this is the case in
> general, negotiation with an exact frame rate can work. Notice that the 24
> fps and 50 fps are not divisors of 60 Hz, but by displaying some of the
> frames one time more than the other frames, it is still relatively easy to
> convert the frame rates to shutter glass frequencies.
>

@Stephen: do you want to keep the meaning of the maximum frame rate because
> you want to be able to choose a lower frame rate, for example because the
> video is to be multiplexed with other video in the same RTP stream?
>

Part of my thinking is simply that it is a bad design practice to overload a
parameter by giving it different meanings (even if the differences appear
slight).

Design principles aside, I believe that real-time applications (particularly
video conferencing) will need variable frame rate for 3D (as they already do
for 2D), for the reasons I pointed out above.  As you point out, there is
one easy (though sub-optimal) way to upsample / downsample frame rate to
match the needed refresh rate - simple duplicating / dropping of frames.
Since frames can be lost in transmission (and because sender and receiver
clocks are not precisely matched), receivers will need to employ such
techniques anyway.

And shuttered glasses are only one way of rendering 3D;  it is clear that
shuttering, polarization and lenticular techniques will all co-exist. For
many receivers, shutter rate doesn't apply at all.

So I am still thinking that negotiation of shuttering rate (or fixed frame
rate) is not required for 3D interoperability.  An encoder is already free
to send fixed frame rate video if it wishes; I think that is enough.

Best Regards,
Stephen

>
> @Ted: given the common frame rates in practise, do you think they still
> need to be negotiated or will the shutter glasses be able to accept all of
> these? Then we could also add some informative text about this in the draft.
>
> Best regards,
> Bert
>
>
>
> frames between Lk and L(k+1).
>
> >I know only know one manufacturer's system at all well, and my