Subject: [AVT] H.264 frame rate from bitstream

From: "Yedidia Amit" <[email-masked]>
Date: Thu, 10 Jul 2008 08:13:27 +0300

Dear experts,

>From what I know frame-rate of bitstream should be define outside the
bitstream (RTP Timestamp or containers like AVI and 3gp)
But I wonder if there is a way to extract the frame rate from the h.264
bitstream itself ?
If there is such way, is it best practice to do so?

Regards,

Amit Yedidia
Elbit System Ltd.

Email: [email-masked]
Tel: 972-4-8318905
----------------------------------------------------------

This e-mail message has been sent by Elbit Systems Ltd.
and is for the use of the intended recipients only.
The message may contain privileged or commercially sensitive information .
If you are not the intended recipient you are hereby notified that any use,
distribution or copying of this communication is strictly prohibited,
and you are requested to delete the e-mail and any attachments
and notify the sender immediately.
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Randell Jesup <[email-masked]>
Date: Thu, 10 Jul 2008 12:10:36 -0400

>But I wonder if there is a way to extract the frame rate from the h.264

You could watch the NAL unit types and Mark bit and try to infer the
(current) framerate, but this value is not static, and you have to be
careful to deal with some of the trickier aspects like slices.  You could
also delve into the NAL units themselves (if you can parse H.264).  You
need to be careful in that not every NAL is a frame (for example parameter
set NALs), and framerate can be highly variable, depending on the
application.

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Gary Sullivan <[email-masked]>
Date: Thu, 10 Jul 2008 09:24:51 -0700

There is a flag called the fixed_frame_rate_flag.  If it is equal to 1, the frame rate can be expressed as time_scale / (2 * num_units_in_tick) frames per second.

If the fixed_frame_rate_flag is equal to 0, the concept of a frame rate may not exactly apply.

Best Regards,

Gary Sullivan

+> -----Original Message-----
+> From: [email-masked] [mailto:[email-masked]] On
+> Behalf Of Randell Jesup
+> Sent: Thursday, July 10, 2008 9:11 AM
+> To: Yedidia Amit
+> Cc: [email-masked]
+> Subject: Re: [AVT] H.264 frame rate from bitstream
+>
+> "Yedidia Amit" <[email-masked]> writes:
+> >From what I know frame-rate of bitstream should be define
+> outside the
+> >bitstream (RTP Timestamp or containers like AVI and 3gp)
+> >But I wonder if there is a way to extract the frame rate
+> from the h.264
+> >bitstream itself ?
+>
+> You could watch the NAL unit types and Mark bit and try to infer the
+> (current) framerate, but this value is not static, and you have to be
+> careful to deal with some of the trickier aspects like
+> slices.  You could
+> also delve into the NAL units themselves (if you can parse
+> H.264).  You
+> need to be careful in that not every NAL is a frame (for
+> example parameter
+> set NALs), and framerate can be highly variable, depending on the
+> application.
+>
+> --
+> Randell Jesup, Worldgate (developers of the Ojo videophone),
+> ex-Amiga OS team
+> [email-masked]
+> "The fetters imposed on liberty at home have ever been
+> forged out of the weapons
+> provided for defence against real, pretended, or imaginary
+> dangers from abroad."
+>                 - James Madison, 4th US president (1751-1836)
+>
+> _______________________________________________
+> Audio/Video Transport Working Group
+> [email-masked]
+> https://www.ietf.org/mailman/listinfo/avt
+>
+>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: "Yedidia Amit" <[email-masked]>
Date: Sun, 13 Jul 2008 08:32:20 +0300

Thank you for your answer.

If fixed_frame_rate_flag is set
1. Does skipping frames (for rate control) can be detected?
2. Does the use in this flag is common, or other methods (e.g.
Timestamp) are generally preffered?

Regards,

Amit Yedidia
Elbit System Ltd.

Tel: 04-8318905

-----Original Message-----
From: Gary Sullivan [mailto:[email-masked]] 
Sent: Thursday, July 10, 2008 7:25 PM
To: Randell Jesup; Yedidia Amit
Cc: [email-masked]
Subject: RE: [AVT] H.264 frame rate from bitstream

There is a flag called the fixed_frame_rate_flag.  If it is equal to 1,
the frame rate can be expressed as time_scale / (2 * num_units_in_tick)
frames per second.

If the fixed_frame_rate_flag is equal to 0, the concept of a frame rate
may not exactly apply.

Best Regards,

Gary Sullivan

---

From: Gary Sullivan <[email-masked]>
Date: Sat, 12 Jul 2008 22:39:21 -0700

It is a little hard to understand your questions.

Perhaps what you're looking for is the maximum possible frame rate rather than the actual exhibited frame rate.  The maximum possible frame rate is always equal to time_scale / (2 * num_units_in_tick), regardless of the value of fixed_frame_rate_flag.

Frame skipping is not allowed when fixed_frame_rate_flag is equal to 1.  Frame skipping violates the concept of a frame rate in the strict sense.  If frame skipping is happening, the actual frame rate is varying.

Best Regards,

Gary Sullivan

+> -----Original Message-----
+> From: Yedidia Amit [mailto:[email-masked]]
+> Sent: Saturday, July 12, 2008 10:32 PM
+> To: Gary Sullivan; Randell Jesup
+> Cc: [email-masked]
+> Subject: RE: [AVT] H.264 frame rate from bitstream
+>
+> Thank you for your answer.
+>
+> If fixed_frame_rate_flag is set
+> 1. Does skipping frames (for rate control) can be detected?
+> 2. Does the use in this flag is common, or other methods (e.g.
+> Timestamp) are generally preffered?
+>
+>
+> Regards,
+>
+>
+> Amit Yedidia
+> Elbit System Ltd.
+>
+> Email: [email-masked]
+> Tel: 04-8318905
+> ----------------------------------------------------------
+>
+>
+> -----Original Message-----
+> From: Gary Sullivan [mailto:[email-masked]]
+> Sent: Thursday, July 10, 2008 7:25 PM
+> To: Randell Jesup; Yedidia Amit
+> Cc: [email-masked]
+> Subject: RE: [AVT] H.264 frame rate from bitstream
+>
+>
+> There is a flag called the fixed_frame_rate_flag.  If it is
+> equal to 1,
+> the frame rate can be expressed as time_scale / (2 *
+> num_units_in_tick)
+> frames per second.
+>
+> If the fixed_frame_rate_flag is equal to 0, the concept of a
+> frame rate
+> may not exactly apply.
+>
+> Best Regards,
+>
+> Gary Sullivan
+>
+> +> -----Original Message-----
+> +> From: [email-masked] [mailto:[email-masked]] On
+> +> Behalf Of Randell Jesup
+> +> Sent: Thursday, July 10, 2008 9:11 AM
+> +> To: Yedidia Amit
+> +> Cc: [email-masked]
+> +> Subject: Re: [AVT] H.264 frame rate from bitstream
+> +>
+> +> "Yedidia Amit" <[email-masked]> writes:
+> +> >From what I know frame-rate of bitstream should be define
+> +> outside the
+> +> >bitstream (RTP Timestamp or containers like AVI and 3gp)
+> +> >But I wonder if there is a way to extract the frame rate
+> +> from the h.264
+> +> >bitstream itself ?
+> +>
+> +> You could watch the NAL unit types and Mark bit and try
+> to infer the
+> +> (current) framerate, but this value is not static, and
+> you have to be
+> +> careful to deal with some of the trickier aspects like
+> +> slices.  You could
+> +> also delve into the NAL units themselves (if you can parse
+> +> H.264).  You
+> +> need to be careful in that not every NAL is a frame (for
+> +> example parameter
+> +> set NALs), and framerate can be highly variable, depending on the
+> +> application.
+> +>
+> +> --
+> +> Randell Jesup, Worldgate (developers of the Ojo videophone),
+> +> ex-Amiga OS team
+> +> [email-masked]
+> +> "The fetters imposed on liberty at home have ever been
+> +> forged out of the weapons
+> +> provided for defence against real, pretended, or imaginary
+> +> dangers from abroad."
+> +>                 - James Madison, 4th US president (1751-1836)
+> +>
+> +> _______________________________________________
+> +> Audio/Video Transport Working Group
+> +> [email-masked]
+> +> https://www.ietf.org/mailman/listinfo/avt
+> +>
+> +>
+>
+> This e-mail message has been sent by Elbit Systems Ltd.
+> and is for the use of the intended recipients only.
+> The message may contain privileged or commercially sensitive
+> information .
+> If you are not the intended recipient you are hereby
+> notified that any use,
+> distribution or copying of this communication is strictly prohibited,
+> and you are requested to delete the e-mail and any attachments
+> and notify the sender immediately.
+>
+>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Gary Sullivan <[email-masked]>
Date: Sat, 12 Jul 2008 22:54:13 -0700

Refinement of response: Actually there are also some other upper bounds on maximum possible frame rate.  However, that is one of them.

Another is that the frame rate can't exceed MaxMBPS divided by PicWidthInMbs * FrameHeightInMbs, where MaxMBPS is a variables that depends on the "level" of the bitstream and PicWidthInMbs * FrameHeightInMbs is the frame size in units of macroblocks.

Another is that the frame rate can't exceed 172 frames per second.

Best Regards,

Gary Sullivan

+> -----Original Message-----
+> From: [email-masked] [mailto:[email-masked]] On
+> Behalf Of Gary Sullivan
+> Sent: Saturday, July 12, 2008 10:39 PM
+> To: Yedidia Amit; Randell Jesup
+> Cc: [email-masked]
+> Subject: Re: [AVT] H.264 frame rate from bitstream
+>
+>
+> It is a little hard to understand your questions.
+>
+> Perhaps what you're looking for is the maximum possible
+> frame rate rather than the actual exhibited frame rate.  The
+> maximum possible frame rate is always equal to time_scale /
+> (2 * num_units_in_tick), regardless of the value of
+> fixed_frame_rate_flag.
+>
+> Frame skipping is not allowed when fixed_frame_rate_flag is
+> equal to 1.  Frame skipping violates the concept of a frame
+> rate in the strict sense.  If frame skipping is happening,
+> the actual frame rate is varying.
+>
+> Best Regards,
+>
+> Gary Sullivan
+>
+> +> -----Original Message-----
+> +> From: Yedidia Amit [mailto:[email-masked]]
+> +> Sent: Saturday, July 12, 2008 10:32 PM
+> +> To: Gary Sullivan; Randell Jesup
+> +> Cc: [email-masked]
+> +> Subject: RE: [AVT] H.264 frame rate from bitstream
+> +>
+> +> Thank you for your answer.
+> +>
+> +> If fixed_frame_rate_flag is set
+> +> 1. Does skipping frames (for rate control) can be detected?
+> +> 2. Does the use in this flag is common, or other methods (e.g.
+> +> Timestamp) are generally preffered?
+> +>
+> +>
+> +> Regards,
+> +>
+> +>
+> +> Amit Yedidia
+> +> Elbit System Ltd.
+> +>
+> +> Email: [email-masked]
+> +> Tel: 04-8318905
+> +> ----------------------------------------------------------
+> +>
+> +>
+> +> -----Original Message-----
+> +> From: Gary Sullivan [mailto:[email-masked]]
+> +> Sent: Thursday, July 10, 2008 7:25 PM
+> +> To: Randell Jesup; Yedidia Amit
+> +> Cc: [email-masked]
+> +> Subject: RE: [AVT] H.264 frame rate from bitstream
+> +>
+> +>
+> +> There is a flag called the fixed_frame_rate_flag.  If it is
+> +> equal to 1,
+> +> the frame rate can be expressed as time_scale / (2 *
+> +> num_units_in_tick)
+> +> frames per second.
+> +>
+> +> If the fixed_frame_rate_flag is equal to 0, the concept of a
+> +> frame rate
+> +> may not exactly apply.
+> +>
+> +> Best Regards,
+> +>
+> +> Gary Sullivan
+> +>
+> +> +> -----Original Message-----
+> +> +> From: [email-masked] [mailto:[email-masked]] On
+> +> +> Behalf Of Randell Jesup
+> +> +> Sent: Thursday, July 10, 2008 9:11 AM
+> +> +> To: Yedidia Amit
+> +> +> Cc: [email-masked]
+> +> +> Subject: Re: [AVT] H.264 frame rate from bitstream
+> +> +>
+> +> +> "Yedidia Amit" <[email-masked]> writes:
+> +> +> >From what I know frame-rate of bitstream should be define
+> +> +> outside the
+> +> +> >bitstream (RTP Timestamp or containers like AVI and 3gp)
+> +> +> >But I wonder if there is a way to extract the frame rate
+> +> +> from the h.264
+> +> +> >bitstream itself ?
+> +> +>
+> +> +> You could watch the NAL unit types and Mark bit and try
+> +> to infer the
+> +> +> (current) framerate, but this value is not static, and
+> +> you have to be
+> +> +> careful to deal with some of the trickier aspects like
+> +> +> slices.  You could
+> +> +> also delve into the NAL units themselves (if you can parse
+> +> +> H.264).  You
+> +> +> need to be careful in that not every NAL is a frame (for
+> +> +> example parameter
+> +> +> set NALs), and framerate can be highly variable,
+> depending on the
+> +> +> application.
+> +> +>
+> +> +> --
+> +> +> Randell Jesup, Worldgate (developers of the Ojo videophone),
+> +> +> ex-Amiga OS team
+> +> +> [email-masked]
+> +> +> "The fetters imposed on liberty at home have ever been
+> +> +> forged out of the weapons
+> +> +> provided for defence against real, pretended, or imaginary
+> +> +> dangers from abroad."
+> +> +>                 - James Madison, 4th US president (1751-1836)
+> +> +>
+> +> +> _______________________________________________
+> +> +> Audio/Video Transport Working Group
+> +> +> [email-masked]
+> +> +> https://www.ietf.org/mailman/listinfo/avt
+> +> +>
+> +> +>
+> +>
+> +> This e-mail message has been sent by Elbit Systems Ltd.
+> +> and is for the use of the intended recipients only.
+> +> The message may contain privileged or commercially sensitive
+> +> information .
+> +> If you are not the intended recipient you are hereby
+> +> notified that any use,
+> +> distribution or copying of this communication is strictly
+> prohibited,
+> +> and you are requested to delete the e-mail and any attachments
+> +> and notify the sender immediately.
+> +>
+> +>
+> _______________________________________________
+> Audio/Video Transport Working Group
+> [email-masked]
+> https://www.ietf.org/mailman/listinfo/avt
+>
+>
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt