Subject: [Cellar] Matroska documentation on Cues

From: Dave Rice <[email-masked]>
Date: Tue, 22 Nov 2016 21:41:02 -0500

Hi all,
Official documentation on the Matroska Cues Element has been sparse though the pull request at https://github.com/Matroska-Org/matroska-specification/pull/69 has provided some good information about recommended practices in use of the Cues Element beyond the mandates provided in Matroska's Schema. Mostly based on Moritz's comments at https://github.com/Matroska-Org/matroska-specification/pull/69#issuecomment-261938346, I started a draft of a documentation about Cues (reviewable at https://github.com/Matroska-Org/matroska-specification/blob/start-cues-documentation/cues.md) and also pasted below. There's a separate document in to repo about Element Ordering recommendations so there's a little more about Cues there. Is there more to say about Cues?

# Cues

## Introduction

The `Cues Element` provides an index of certain `Cluster Elements` to allow for optimized seeking to absolute timestamps within the `Segment`. The `Cues Element` contains one or many `CuePoint Elements` which each MUST reference an absolute timestamp (via the `CueTime Element`), a `Track` (via the `CueTrack Element`), and a `Segment Position` (via the `CueClusterPosition` Element). Additional non-mandated Elements are part of the `CuePoint Element` such as `CueDuration`, `CueRelativePosition`, `CueCodecState` and others which can improve seeking performance in some Matroska players.

## Recommendations

The following recommendations are provided to optimize Matroska performance.

- Unless Matroska is used as a live stream, it SHOULD contain a `Cues Element`. Note that some Matroska players may not be able to play a Matroska file without a `Cues Element`.
- For each video track, each keyframe SHOULD be referenced by a `CuePoint Element`.
- It is RECOMMENDED to not reference non-keyframes of video tracks in `Cues` unless it references a `Cluster Element` which contains a `CodecState Element` but no keyframes.
- For each subtitle track present, each subtitle frame SHOULD be referenced by a `CuePoint Element` with a `CueDuration Element`.
- Audio tracks SHOULD only be referenced in `CuePoint Elements` if no video track is present. In this case `CuePoint Elements` SHOULD reference audio keyframes at most once every 500 milliseconds
- If the referenced frame is not stored within the first `SimpleBlock` or first `BlockGroup` within its `Cluster Element` then the `CueRelativePosition Element` SHOULD be written to reference where in the `Cluster` the reference frame is stored.
- If a `CuePoint Element` references `Cluster Element` that includes a `CodecState Element` then that `CuePoint Element` MUST use a `CueCodecState Element`.
- `CuePoint Elements` SHOULD be numerically sorted in storage order by the value of the `CueTime Element`.

Best Regards,
Dave Rice