Subject: [AVT] RE: Carrying SMPTE time-codes in RTP streams, discussion email

From: "Link, Brian" <[email-masked]>
Date: Fri, 11 Feb 2005 15:11:46 -0800

Hi Dave,

This is good. You've given this a lot of thought. Here are a few
comments to add to the mix:

1.  One (or more) actual SMPTE time stamp values are needed in each
packet for the commercial application I described. I don't see how the
mapping between SMPTE time code and RTP time stamps over long runs in
your proposal could be sufficient. The application is a quality control
(QC) test of the final master audio and video for a DVD, before they are
multiplexed into a program stream. Part of what needs to be checked, is
that the SMPTE time stamps that are "striped" on a hard disk with the
audio (one SMPTE time stamp value per AC-3 audio frame) are all
correctly lined up with the video so that the multiplex operation will
work correctly. The mapping between RTP time stamps and SMPTE time
stamps isn't sufficient because it requires the receiver to synthesize
SMPTE time stamps between mapping points, which may mask errors in the
source.

2.  Since synthesized time code is acceptable in many applications, it
presumably is undesirable to require all streams that use SMPTE to carry
a time stamp for every RTP packet. Would it be possible to send a
mapping RTCP APP packet for every RTP packet? Or even more than one per
RTP packet, since there is a SMPTE time stamp associated with each audio
frame? Then no packet would get stamped with a synthesized time stamp.
This might be a way to enable the QC application.

3.  For that matter, what's the advantage of putting these mappings in
separate RTCP packets? Instead, encapsulating them in the same packet as
the media data instead would give a greater assurance that the receiver
is associating the right SMPTE time value with the media data. 

4.  Note that regardless of where the mapping exists, there needs to be
a mechanism so that the receiver can determine if and when it has missed
a mapping packet (and therefore a run boundary) so it knows that any
further synthesized values are invalid. 

5.  Finally, I've specified a different representation of SMPTE time
code values for the AC-3 payload format than you have. There's an
extension defined in SMPTE 339M for applying time code values to audio
frames, which may not be synchronous with their associated video frames.
The key difference is that the 339M format adds a 'sample' field after
the 'frames' field. This indicates which PCM sample in the coded audio
frame the (video) frame number in the time stamp is associated with.

* * *

Because of these points, I still think the QC application is best served
by including actual SMPTE time stamps associated with each audio frame
inside each RTP packet. It seems like the QC application is a special
case compared to the applications that would generally use your mapping.
Maybe there is a good way to handle the special case within the mapping.
I've already proposed a straightforward way to include it in the AC-3
payload format specification. Since it is optional, it can be omitted
from most streams and doesn't interfere with the general case.

Regards,
Brian

---
Brian Link
Dolby Laboratories, 100 Potrero Ave., San Francisco, CA 94103
phone: 415-558-0324  fax: 415-863-1373
email: [email-masked]   alt email: [email-masked]

-----Original Message-----
From: Dave Singer [mailto:[email-masked]] 
Sent: Thursday, February 10, 2005 4:25 PM
To: [email-masked]
Cc: Colin Perkins; [email-masked]; Link, Brian
Subject: Carrying SMPTE time-codes in RTP streams, discussion email

Here are some thoughts on carrying SMPTE time-code information in RTP.

First a brief background on SMPTE time-codes.

SMPTE time-codes count frames.  There are two common forms of
display:  either a simple counter, or what looks like a normal clock
value (hh:mm:ss.frame).  When the frame rate is truly integer, then this
can be a normal clock value, in that seconds tick by at the same rate as
the seconds we know and love.

However, NTSC video infamously runs slightly slower than 30
frames/second.  Some people call it 29.97 (which isn't quite right) and
some say that a frame takes 1001 ticks of a 30000 tick/second clock
(which is closer).  Be that as it may, SMPTE time codes count 30 of
these frames and deem that to make a second.

This causes a SMPTE time-code display to 'run slow' compared to
real-time.  To ameliorate this, a format called drop-frame is used. 
Some of the frame numbers are skipped, so that the counter periodically
'catches up' (so some time-code-seconds actually only have 28 or 29
frames in them).

So, what we desire is a system that allows us to associate a SMPTE
time-code with some media.  Since in RTP all media has a clock already,
we can leverage that fact.  If we treat the media as having 'runs' of
time in which the time-code is simply counting up, then the time-code
anywhere within a run can be calculated if you know 1. the RTP timestamp
of the start of the run 2. the RTP timestamp where you want to know the
time-code 3. the time-code of the start of the run 4. the counting rate
and other parameters of the time-code.

My proposal is that we put periodic mappings between (1) and (3) into
RTCP packets, and provide (4) the 'setup' information out-of-band, for
example in SDP.

The setup information includes:
   (the timescale of the RTP stream, already provided)
   the duration, in that timescale, of a single frame-count in the
'frames' portion of the time-code
   the number of those frames that make a time-code-second
   the following booleans:
     is-NTSC-drop-frame:  should the usual 'left out numbers' of
drop-frame be applied or not?
     wrap-at-24-hours:  should the hours portion wrap from 23 to 0, or
keep counting up?
     allow-negative-time-codes:  are negative time-codes used in this
stream?
     display-time-code-as-counter:  should the display be an integer
frame-count, or hh:mm:ss.fr format?
     time-code-displayed:  is it intended that this time-code be
displayed somehow?

For example, if associated with a video track with the common time-scale
of 90000, then frame-duration of 3003 and frames-per-tc-second of 30
would yield a 'normal' SMPTE time-code for NTSC video.  Similarly values
of 3750 and 24 yield a time-code for 24 fps film content, and so on.

Now, we put into an RTCP APP packet (or a new RTCP packet), a mapping
between an RTP time-stamp value and the time-code.  The RTP timestamp
and the that time-code are 32 bits;  the time-code is either a signed
counter value (if we're in counter format), or it is the format
   hours(8) -- 0 to 255
   sign(1) -- 1 for negative, 0 for positive
   minutes(7) -- 0 to 59
   seconds(8) -- 0 to 59
   frames(8) -- 0 to (frames-per-tc-second - 1)

This establishes the time-code for all RTP times greater than or equal
to the one given, until a subsequent APP packet reestablishes the
mapping.  It's unfortunate that the sign is in the middle, but that
allows the hours to use the full range, and the minutes don't need to.

* * *

It might be argued that we could set the initial mapping also in the
SDP, since RTCP packets might get lost.  But this means that the SDP now
has to have knowledge of the RTP random offset (nasty);  and if one puts
this APP packet into all sender reports, it's probably good enough.
Then if you don't have time-codes, you don't have audio-video-sync
either.

This associates the time-code with a particular media stream.  An
alternative would be to make it an RTP stream in its own right;  but the
data rate is so low, this seems egregious.  And by packing it inline, we
can do this backwards-compatible for gateways etc. that already handle
dual-stream.

Comments?

--
David Singer
Apple Computer/QuickTime

-----------------------------------------
This message (including any attachments) may contain confidential
information intended for a specific individual and purpose.  If you are not
the intended recipient, delete this message.  If you are not the intended
recipient, disclosing, copying, distributing, or taking any action based on
this message is strictly prohibited.

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: lazzaro <[email-masked]>
Date: Fri, 11 Feb 2005 15:52:42 -0800

On Feb 11, 2005, at 3:11 PM, Link, Brian wrote:
> Part of what needs to be checked, is

Let's assume the sender of the RTP stream has access to the hard disk,
with its SMPTE time stamp stripe.  If the sender makes sure that the 
mapping
it sends in the RTCP packets is such that for every AC-3 audio frame, a
synthesized timestamp by the receiver will always exactly match the 
time code
on the hard disk, then it would seem to be impossible to mask errors -- 
the
synthesis algorithm would  always yield the timestamp that exists on 
the stripe.

Am I missing something here?

> is associating the right SMPTE time value with the media data.

It sounds like you're suggesting the requirements of your application
require a SMPTE timestamp on each RTP packet ... as my comment
above shows, I don't see why that would be true (yet) ... but lets 
assume it is.

In this case, the general-purpose options would be doing it as an RTP
profile (i.e. each RTP header has an extra N octets for a SMPTE 
timecode)
or as an RTP header extension.  Traditionally, I had thought extensions 
were
meant for experimental deployments, not standards-track items ... so 
lets
assume an RTP profile for the moment.

Would this sort of profile result in full functionality for your 
application?
Or did the timestamps in your AC-3 payload format (implicitly) do more
than simply associate the RTP timestamp of each packet with a SMPTE 
time stamp?

> further synthesized values are invalid.

Phrased in this way, RTP over a network that can lose packets might not
satisfy your requirement -- what if a packet sequence is lost that 
contains two
run boundaries?  Or are you assuming you terminate the session if an RTP
sequence number break is detected?

---
John Lazzaro
http://www.cs.berkeley.edu/~lazzaro
lazzaro [at] cs [dot] berkeley [dot] edu
---

---

From: Dave Singer <[email-masked]>
Date: Sun, 13 Feb 2005 15:45:11 -0800

Brian, your email raised interesting enough questions that I was 
trying to think about it before reacting.  However, Colin wants an 
I-D in tomorrow morning (it can be revised later), so here goes.

>SMPTE time stamps between mapping points, which may mask errors in the

If you lose RTCP packets, yes, you may lose a discontinuity.  I tried 
to discuss this a bit in the draft.

>a time stamp for every RTP packet.

Every RTP packet already has a timestamp.  We're at cross-purposes 
here;  I am leveraging the *existing* timestamp;  no change at all to 
RTP packets.  Completely neutral with respect to profile, encryption, 
and payload format.

>Would it be possible to send a

If they are in every packet, there is no need for a mapping.  If they 
are imputed from previous packets, they can be imputed from RTCP. 
The advantage of this approach is the low-bandwidth and isolation 
from the RTP formats.

>further synthesized values are invalid.

I can't see how to do this perfectly;  for stored media, one could 
have a range in the mapping (when a segment end-time is known in 
advance).  This isn't possible live, of course.

>frame the (video) frame number in the time stamp is associated with.

That's important.  Thanks.  It needs thought.  If the RTP clock is 
sample-accurate, then I think I'm OK, as I am associating the 
time-code with an RTP timestamp, which may not be the timestamp of 
any given frame.

-- 
David Singer
Apple Computer/QuickTime

---

From: lazzaro <[email-masked]>
Date: Sun, 13 Feb 2005 16:01:50 -0800

On Feb 13, 2005, at 3:45 PM, Dave Singer wrote:
> time-code with an RTP timestamp, which may not be the timestamp of any 
> given frame.

Yes, this issue becomes harder if we end up supporting an RTP profile
that includes a SMPTE time code in the RTP header.  We'll need some
way to SMPTE stamp headers whose RTP timestamp does not coincide
with an exact SMPTE timecode value, probably using something like
the 'sample' field Brian describes.

For example, as a rule, RTP MIDI RTP timestamps generated from a MIDI
controller keyboard will not coincide with SMPTE timestamp positions ...

---
John Lazzaro

---

From: Dave Singer <[email-masked]>
Date: Sun, 13 Feb 2005 16:06:01 -0800

>controller keyboard will not coincide with SMPTE timestamp positions ...

agreed.

here is the draft.  we can revise it later in the week, though I hope 
I have caught enough open issues to cover your questions and 
concerns.  I think I had better send this off to the IETF editors now!

Internet Engineering Task Force
INTERNET-DRAFT                                                 D. Singer
draft-singer-smpte-rtp-00                                 Apple Computer

                                                              Feb 13 2005
                                                     Expires: Aug 13 2005

              Associating SMPTE time-codes with RTP streams

IPR Notice

By submitting this Internet-Draft, I certify that any applicable patent
or other IPR claims of which I am aware have been disclosed, or will be
disclosed, and any of which I become aware will be disclosed, in
accordance with RFC 3668.

Status of This Memo

    This document is an Internet-Draft and is in full conformance with
    all provisions of Section 10 of RFC2026.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as Internet-
    Drafts.

    Internet-Drafts are draft documents valid for a maximum of six months
    and may be updated, replaced, or obsoleted by other documents at any
    time.  It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than a "work in progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/1id-abstracts.html

    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
    document are to be interpreted as described in RFC 2119.

Distribution of this document is unlimited.

Copyright Notice

    Copyright (C) The Internet Society (2005).  All Rights Reserved.

D. Singer                                                       [Page 1]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

Abstract

    This document describes a mechanism for associating SMPTE time-codes
    with media streams, in a way that is independent of the RTP payload
    format of the media stream itself.

1 Introduction

    First a brief background on SMPTE time-codes [SMPTE].

    SMPTE time-codes count frames.  There are two common forms of
    display:  either a simple counter, or what looks like a normal clock
    value (hh:mm:ss.frame).  When the frame rate is truly integer, then
    this can be a normal clock value, in that seconds tick by at the same
    rate as the seconds we know and love.

    However, NTSC video infamously runs slightly slower than 30
    frames/second.  Some people call it 29.97 (which isn't quite right)
    and some say that a frame takes 1001 ticks of a 30000 tick/second
    clock (which is closer).  Be that as it may, SMPTE time codes count
    30 of these frames and deem that to make a second.

    This causes a SMPTE time-code display to 'run slow' compared to real-
    time.  To ameliorate this, sometimes a format called drop-frame is
    used.  Some of the frame numbers are skipped, so that the counter
    periodically 'catches up' (so some time-code-seconds actually only
    have 28 or 29 frames in them).

    It is worth noting that in neither case is the SMPTE time-code an
    accurate clock;  in the first case, it runs slow, and in the second,
    the adjustments are abrupt and periodic - and still not quite
    accurate.  Hence in the rest of this document I try to be clear when
    referring to a second in a time-code as a 'time-code second'.

    However, SMPTE time-codes do run in real-time when used with systems
    with integral frames/second (e.g. film content at 24 frames/second,
    or PAL video).  The 'drift' issue is (I believe) unique to NTSC
    video.

2 Design Goals

    What we desire is a system that allows us to associate a SMPTE time-
    code with some media in an RTP [RTP] stream.  Since in RTP all media
    has a clock already, we can leverage that fact.  If we treat the
    media as having 'segments' of time in which the time-code is simply
    counting up, then the time-code anywhere within a segment can be
    calculated if you know:

D. Singer                                                       [Page 2]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

       1. the RTP timestamp of the start of the segment;
       2. the time-code of the start of the segment;
       3. the counting rate and other parameters of the time-code;
       4. the RTP timestamp where you want to know the time-code.

    My proposal is that we put periodic mappings between (1) and (2) into
    RTCP packets, and provide (3) the 'setup' information out-of-band,
    for example in SDP.  Then given a timestamp (4), we can calculate
    exactly what the time-code must be.

    The setup information includes:
        (the timescale of the RTP stream, already provided);
       the duration, in that timescale, of a single frame-count in the
       'frames' portion of the time-code
       the number of those frames that make a time-code-second
       the following booleans:
          is-NTSC-drop-frame:  should the usual 'left out numbers' of
          drop-frame be applied or not?
          wrap-at-24-hours:  should the hours portion wrap from 23 to 0,
          or keep counting up?
          allow-negative-time-codes:  are negative time-codes used in
          this stream?
          display-time-code-as-counter:  should the display be an integer
          frame-count, or hh:mm:ss.fr format?
          time-code-displayed:  is it intended that this time-code be
          displayed somehow?

    For example, if associated with a video track with the common time-
    scale of 90000, then frame-duration of 3003 and frames-per-tc-second
    of 30 would yield a 'normal' SMPTE time-code for NTSC video.
    Similarly values of 3750 and 24 yield a time-code for 24 fps film
    content, and so on.

    counter value (if we're in counter format), or it is the format:

    This establishes the time-code for all RTP times greater than or
    equal to the one given, until a subsequent APP packet reestablishes
    the mapping.  It's unfortunate that the sign is in the middle, but
    that allows the hours to use the full range, and the minutes don't
    need to.

D. Singer                                                       [Page 3]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

    Note that the RTP time-stamp in the mapping may not match the time-
    stamp of any frame in the media stream.  For video, it normally
    would;  but a time-stamp transition may happen part-way through a
    decoded audio frame.  Since they share the same clock, the timing of
    that transition and the timing of the audio stream itself have the
    same accuracy.

3 Discussion

    This design has the advantage of introducing no new IP packets into
    the sessions, using low-bandwidth (vanishingly low in the case of
    streams with no discontinuities), and is independent of the design of
    the RTP packets themselves:  the RTP profile (including possibly
    encryption) and the RTP payload format.  SMPTE time-codes can be
    associated with any RTP stream, including those with existing payload
    formats.

    It might be argued that we could set the initial mapping also in the
    SDP, since RTCP packets might get lost.  But this means that the SDP
    now has to have knowledge of the RTP random offset, which is nasty;
    and if one puts this APP packet into all sender reports, it's
    probably good enough.  Then if you don't have time-codes, you don't
    have audio-video-sync either.

    This associates the time-code with a particular media stream.  An
    alternative would be to make it an RTP stream in its own right;  but
    the data rate is so low, this seems egregious.  And by packing it
    inline, we can do this backwards-compatible for gateways etc. that
    already handle dual-stream.

    The APP packets need not use the same RTP timestamp as the sender
    report in the same RTCP packet.  They can be sent 'ahead of need' if
    possible (e.g. for stored content, when the server can look-ahead) or
    just-in-time - send an RTCP immediately a discontinuity in the time-
    code is detected, and allow media-buffering in the client the chance
    to 'catch' the RTCP before the matching RTP packet is processed and
    displayed,

    If time-codes change more rapidly than RTCP packets are normally
    sent, then this might force a more rapid transmission.  However, it
    should be possible to send multiple mappings in one RTCP packet.

    There is no way in this draft to detect that an RTCP packet has been
    lost, and that a mapping may be being used outside its intended
    range.  The likelihood of this happening can be reduced, however, by
    permitting a pair of RTP times in the mapping, and defining that the
    mapping is only valid between those times.  This only works for
    stored media, when look-ahead is possible, of course.

D. Singer                                                       [Page 4]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

    This design has the advantage of being independent of both the RTP
    profile and the media-specific RTP payload design.  However, it does
    this at the expense of certainty.  It is also possible to imagine
    some payloads with explicit provision for this mapping.  However,
    such provision need not go as far as a full SMPTE time-code in every
    packet.  If the RTCP APP packets have a small 'time-code epoch'
    indicator, then that same epoch indicator can be in each packet.

    The current design assumes that clients will hold mappings until they
    are superseded, and that a client may need to buffer some number of
    upcoming mappings.  It may be necessary to introduce explicit
    statements about the amount of buffering needed.

    For trick modes, it may be desirable to signal that a given section
    of media has the time-code running in reverse;  this would require a
    new sign bit in the mapping record.

4 Security Considerations

    SMPTE time-codes are only informative and it is hard to see security
    considerations from associating them with media streams.

5 IANA Considerations

    None.

6 RFC Editor Considerations

    None.

7 Full Copyright Statement

    Copyright (C) The Internet Society (2005).  This document is subject
    to the rights, licenses and restrictions contained in BCP 78, and
    except as set forth therein, the authors retain all their rights.

    This document and the information contained herein are provided on an
    "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
    OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
    ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
    INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
    INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

8 Intellectual Property Notice

    The IETF takes no position regarding the validity or scope of any
    intellectual property or other rights that might be claimed to

D. Singer                                                       [Page 5]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; neither does it represent that it
    has made any effort to identify any such rights.  Information on the
    IETF's procedures with respect to rights in standards-track and
    standards-related documentation can be found in BCP-11.  Copies of
    claims of rights made available for publication and any assurances of
    licenses to be made available, or the result of an attempt made to
    obtain a general license or permission for the use of such
    proprietary rights by implementers or users of this specification can
    be obtained from the IETF Secretariat.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights which may cover technology that may be required to practice
    this standard.  Please address the information to the IETF Executive
    Director.

Acknowledgments

    Both Brian Link and John Lazzaro provided helpful comments on an
    initial draft (not all of which are yet addressed).

D. Singer                                                       [Page 6]

Internet Draft        draft-singer-smpte-rtp-00.doc        Feb 13 2005

Authors' Contact Information
    David Singer
    Apple Computer, Inc.
    One Infinite Loop, MS:302-3MT
    Cupertino  CA 95014
    USA
    Email: [email-masked]
    Tel: +1 408 974 3162

6. References
    [RTP]
    RFC3550, STD0064, RTP: A Transport Protocol for Real-Time
    Applications, H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson,
    July 2003

    [SMPTE-12M]
    SMPTE 12M-1999, Television, Audio and Film - Time and Control  Code

Dates
                              Written: Feb 13 2005
                              Expires: Aug 13 2005

D. Singer                                                       [Page 7]

-- 
David Singer
Apple Computer/QuickTime

---

From: lazzaro <[email-masked]>
Date: Sun, 13 Feb 2005 16:35:06 -0800

On Feb 13, 2005, at 4:06 PM, Dave Singer wrote:
> here is the draft.

Looks good, I never realized:

>    accurate.

Abrupt and periodic are both OK, but if over long periods of time,
if there's going to be a systematic slip of the RTP time base relative
to the SMPTE time base in the case of NTSC ... I could imagine that
being a problem in some applications.

Is there a standardized way to do a second-order adjustment
on top of the drop-frame adjustment, to make the timing perfect over
long periods of time?  Or is the standard method to make "nominal"
"real-time" conform to SMPTE time, sort of how we drop a second
or add a second from "official time" on New Years Eve every few years?

---
John Lazzaro

---

From: Dave Singer <[email-masked]>
Date: Sun, 13 Feb 2005 16:40:45 -0800

>or add a second from "official time" on New Years Eve every few years?

I need to do some research on this one.  Glenn 
Adams sent me this since we're discussing these 
time-codes also in the W3C.  Quoted here without 
his permission:

* * *

I found an answer to this question in Timecode, A 
User's Guide, by John Ratcliff, 1999, on pg 58 (a 
book which I recalled having in my library but 
having never read before - looks like it is about 
time to read it):

"Š the NTSC frame rate is not exactly 29.97 Hz: 
it is 29.970 026 17 Hz. This difference results 
in a long-term residual discrepancy of 0.000 026 
17 * 60 * 60 * 24 = 2.261 frames over a 24 hour 
period, equivalent to 86.4 µs. Good operational 
practice requires that the timecode generator be 
reset at regular intervals if long-term errors 
are not to accumulate. Commercial timecode 
generators are available which allow the 86 µs 
discrepancy to be corrected at a time when least 
disruption to programme making will occur. The 
correction is made daily, automatically, and at a 
time programmable by the user."

However, this raises a new question since:

30000/1001 = 29.970029997002997Š is not equal to 29.97002617 cited above

Having just done a little bit more research, it 
turns out that the 29.97002617 is correct, and is 
derived as follows:

Horizontal Scanning Frequency (lines/sec) / 525 (lines/frame) where

Horizontal Scanning Frequency = Chrominance Sub-Carrier * 2/455

Now, since Chrominance Sub-Carrier for NTSC is 
exactly 3,579,545Hz, then doing the arithmetic 
one ends up with:

( 3,579,545Hz * 2/455 ) lines/sec / 525 
lines/frame = 29.970026164311878597592883307169 
frames/sec

Note that this is still not quite 29.97002617, 
which seems to be a result of rounding up at the 
10 parts per billion resolution.

So I guess the final conclusion is that the exact 
frame rate for NTSC turns out NOT to be 
30000/1001, but rather to be 7159090/238875, 
which can be reduced to 1431818/47775.

* * * * * *

Is anyone in enough pain yet?
-- 
David Singer
Apple Computer/QuickTime

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 23 Feb 2005 11:16:29 +0100

Hi,

Although it seems good with minimal overhead with synthesized time-codes 
they seem to have some problems. What I can understand you have the 
following problems:

1. Needs update when the time-code is changed from content side. For 
example when concatenating program and commercials.

2. Needs update when the time-code to RTP TS mapping changes due to 
transport. Example is if one uses RTSP, which also support requesting 
media using the SMPTE time code. Then every time one hits pause and then 
play a new mapping needs to be established, however here RTSP do provide 
an initial mapping if one uses that time scale in the PLAY request.

3. There is need for a mechanism that ensures that the regenerator uses 
the same drop mechanism with the same start point, otherwise up to two 
frames errors in the numbering can occur. If unpredictable dropping of 
frames occurs, each must result in an update of the time code.

Some questions to help construct requirements on the solution.

A. Is it acceptable to get the update a bit late? Using RTCP can easily 
result in that the time code update arrives up to a few seconds late. 
However if some latency is acceptable, the RTCP bandwidth and the usage 
of the AVPF profile can ensure that one maintain a limit on the latency 
to send the update.

B. What need exist to have the SMPTE to indicate a specific audio sample 
  within a audio frame or packet? This concerns the need to be able to 
indicate a RTP timestamp value that corresponds to a sample within an 
audio stream that isn't the first in a frame etc. Also how does one 
handle SMPTE time-codes for a interlaced video where the RTP timestamp 
isn't used to indicate the half-frames display time.

Comments on the proposal of having something in each media packet and 
the RTP mechanism available for it. This is if the requirements raised 
and deemed important points result in such a solution.

In general I think the SMPTE time code updates would be nice header 
extensions. However I am very hesitant to produce yet another RTP 
profile. Here I wished the RTP header mechanism was more flexible and 
would allow for multiple header extensions, like the DCCP options. If 
one defines a RTP header field as part of a RTP profile for this, then 
we would get into deep profile problems. Then we would have RTP 
extension in four dimensions: SMPTE-time codes, security, feedback, and 
congestion control. Thus resulting in a total of 16 combinations. the 
process of defining and combining the different profile is starting to 
get to big.

Thus if there is need to have something in each packet, I would propose 
to consider a wrapper payload, in the line of RFC 2198 audio redundancy. 
That would allow for efficient fixed size additions followed by the 
normal payload, or if really required even an variable sized payload, at 
the overhead cost of 7-24 bits.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Dave Singer <[email-masked]>
Date: Wed, 23 Feb 2005 16:16:49 +0000

At 2:58 PM +0100 2/23/05, Magnus Westerlund wrote:
>Hi Dave,
>
>One thing that hasn't been discussed that much, is the need to 
>actually define these things to work within RTP. Which I think puts 
>two important requirements. There will be an individual binding for 
>each SSRC present in the session. Secondly, how do you handle 
>payload type switches that results in changed RTP timestamp rate?

The SDP assocation has to be per-payload-type, not per media-type, yes.

>
>
>Dave Singer wrote:
>>
>>>2. Needs update when the time-code to RTP TS mapping changes due 
>>>to transport. Example is if one uses RTSP, which also support 
>>>requesting media using the SMPTE time code. Then every time one 
>>>hits pause and then play a new mapping needs to be established, 
>>>however here RTSP do provide an initial mapping if one uses that 
>>>time scale in the PLAY request.
>>
>>
>>Ah yes, some update of RTSP might be desirable to express the 
>>mapping at the beginning of a play-spurt without having to wait for 
>>RTCP.
>>
>Which is possible if one uses PLAY request with SMPTE Ranges. 
>However it might be the case that one like to use NPT and still get 
>the SMPTE codes.

Agreed.

>>
>>
>>>B. What need exist to have the SMPTE to indicate a specific audio 
>>>sample  within a audio frame or packet? This concerns the need to 
>>>be able to indicate a RTP timestamp value that corresponds to a 
>>>sample within an audio stream that isn't the first in a frame etc. 
>>>Also how does one handle SMPTE time-codes for a interlaced video 
>>>where the RTP timestamp isn't used to indicate the half-frames 
>>>display time.
>>
>>
>>There is no assumption that the RTP timestamp used in the SMPTE-RTP 
>>mapping is the timestamp of the beginning of an audio frame, i.e. 
>>it may be a time 'in the middle' of an RTP packet.
>>
>
>Okay, then it shouldn't be a problem at least not for codecs that 
>uses sampling rate = timestamp rate. Otherwise there will be a small 
>error. Which unfortunately will accumulate I think.

I don't see that it accumulates;  the math is not incremental 
per-packet, but is based from the origin point.  Essentially, your 
SMPTE time-code can be as accurate as your synchronization, since 
they use the same media clock.
-- 
David Singer
Apple Computer/QuickTime

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 23 Feb 2005 14:58:50 +0100

Hi Dave,

One thing that hasn't been discussed that much, is the need to actually 
define these things to work within RTP. Which I think puts two important 
requirements. There will be an individual binding for each SSRC present 
in the session. Secondly, how do you handle payload type switches that 
results in changed RTP timestamp rate?

Dave Singer wrote:
>>transport. Example is if one uses RTSP, which also support 
>>requesting media using the SMPTE time code. Then every time one hits 
>>pause and then play a new mapping needs to be established, however 
>>here RTSP do provide an initial mapping if one uses that time scale 
>>in the PLAY request.
> 
> 
> Ah yes, some update of RTSP might be desirable to express the mapping 
> at the beginning of a play-spurt without having to wait for RTCP.
> 
Which is possible if one uses PLAY request with SMPTE Ranges. However it 
might be the case that one like to use NPT and still get the SMPTE codes.

> 
>>3. There is need for a mechanism that ensures that the regenerator 
>>uses the same drop mechanism with the same start point, otherwise up 
>>to two frames errors in the numbering can occur. If unpredictable 
>>dropping of frames occurs, each must result in an update of the time 
>>code.
> 
> 
> The location of the dropped numbers is mandated by SMPTE 12M;  it's 
> something like "frame numbers 0 and 1 are not used in every nTH 
> minute" and so on, so I don't think this can occur.
> 

Okay, so it is based on the SMPTE timescale. So if I receive a frame 
that has RTP timestamp corresponding without compensation that show 
0:10:00.0, then if I every 10th minute needs to remove two frames, then 
I directly skip to 0:10:00.2? The every 10 minute being an example, 
rather than the correct number. Then I don't think there will be a problem.

>>A. Is it acceptable to get the update a bit late? Using RTCP can 
>>easily result in that the time code update arrives up to a few 
>>seconds late. However if some latency is acceptable, the RTCP 
>>bandwidth and the usage of the AVPF profile can ensure that one 
>>maintain a limit on the latency to send the update.
> 
> 
> I *think* so...

This is an important point, so further feedback on this is crucial.

>>sample  within a audio frame or packet? This concerns the need to be 
>>able to indicate a RTP timestamp value that corresponds to a sample 
>>within an audio stream that isn't the first in a frame etc. Also how 
>>does one handle SMPTE time-codes for a interlaced video where the 
>>RTP timestamp isn't used to indicate the half-frames display time.
> 
> 
> mapping is the timestamp of the beginning of an audio frame, i.e. it 
> may be a time 'in the middle' of an RTP packet.
> 

Okay, then it shouldn't be a problem at least not for codecs that uses 
sampling rate = timestamp rate. Otherwise there will be a small error. 
Which unfortunately will accumulate I think.

Cheers

Magnus Westerlund

---

From: Dave Singer <[email-masked]>
Date: Wed, 23 Feb 2005 12:26:10 +0000

Thanks, these are useful comments.  Some responses below.

>example when concatenating program and commercials.

yes, this is the reason for the use of RTCP for the mappings.

I *think* so...

>
>
>Comments on the proposal of having something in each media packet 
>and the RTP mechanism available for it. This is if the requirements 
>raised and deemed important points result in such a solution.
>
>profile. Here I wished the RTP header mechanism was more flexible 
>and would allow for multiple header extensions, like the DCCP 
>options. If one defines a RTP header field as part of a RTP profile 
>for this, then we would get into deep profile problems. Then we 
>would have RTP extension in four dimensions: SMPTE-time codes, 
>security, feedback, and congestion control. Thus resulting in a 
>total of 16 combinations. the process of defining and combining the 
>different profile is starting to get to big.
>
>Thus if there is need to have something in each packet, I would 
>propose to consider a wrapper payload, in the line of RFC 2198 audio 
>redundancy. That would allow for efficient fixed size additions 
>followed by the normal payload, or if really required even an 
>variable sized payload, at the overhead cost of 7-24 bits.

It does seem way overdue to have a simple mechanism for using the X 
bit in the RTP header which is extensible (e.g. that the header 
extension has a type and an X bit of its own, so it's chainable...)

If particular payloads want to include in-band mappings, they can be 
designed to allow that, of course.
-- 
David Singer
Apple Computer/QuickTime

---

From: Chuck Harrison <[email-masked]>
Date: Wed, 23 Feb 2005 20:47:12 -0800

Hiya,

> >or add a second from "official time" on New Years Eve every few years?

There is a draft practice under standardization in SMPTE called
"Date Compensated Count" which addresses this issue. However
I think there is no panacea; in many cases the right answer is
"cheat at the time of night when the fewest people are watching".

There is also a SMPTE standard "Absolute Time Reference (ATR)"
which establishes precise relationships among timebases in
different production standards (NTSC, PAL, AES audio, etc.).
ATR also has a mechanism for leap second insertion; this is
necessary because the rotation of the earth -- represented
by UTC -- is not as stable as international atomic time
(TAI).

Not enough pain yet? Read on...

> turns out that the 29.97002617 is correct, and is

I do not think this statement is true. Most official
standards documents define NTSC frame rate as 30/1.001.
This would correspond to a Fsc of  3,579,545.4545... Hz.

As I recall the story, the reason the frame rate was pulled
0.1% down was to keep the TV sound carrier from beating with
some of the video signal components during over-air transmission.
When color TV was introduced it was determined that the zillions
of black-and-white sets already in homes would easily handle a
small change in scan rate but their tuned-circuit FM sound
demodulators wouldn't have tracked an 0.1% shift in carrier
frequency as nicely. "It seemed like a good idea at the time."

It seems fairly likely that a reference citing 3,579,545Hz
as the exact Fsc was simply rounding to an integer frequency.

It is equally possible that different official standards have
been independently published that disagree on this.

The only situations where sub-part-per-million precision
matters involve long-term accumulation (such as discussed at
the beginning of this email). I would stick with 30/1.001
for this because that's what all the recent SMPTE documents
in this area have been using. I've never heard of anyone
using the 3,579,545.00Hz-based number in this kind of work.

Cheers,
  Chuck Harrison
  Far Field Associates, LLC
  Seattle, WA

---

From: Dave Singer <[email-masked]>
Date: Thu, 24 Feb 2005 09:01:23 +0000

Chuck

many thanks.  It strikes me that it may not be 
important whether we're off by a small factor in 
the NTSC clock rate, as in any case it counts 
actual frames, and it's possible that the source 
video clock is off by a small factor anyway. 
Clocks that drift with respect to each other are 
part of the joy of distributed multimedia.

-- 
David Singer
Apple Computer/QuickTime