Subject: Re: [Cellar] Review of draft-lhomme-cellar-ebml-00 (Was: recent CELLAR documents)

From: Dave Rice <[email-masked]>
Date: Sun, 11 Sep 2016 22:46:49 -0400

Hi,
Last response to this, but I’ve worked through many of these suggestions. I think everything is responded to with links to the corresponding commit, else it is moved to the issue tracker in the ebml repo.

TL;DR A pull request responding to this thread can be reviewed at https://github.com/Matroska-Org/ebml-specification/pull/99. Non-resolved issues were moved to the tickets 88-98, see https://github.com/Matroska-Org/ebml-specification/issues. There’s a few questions to Tim, Moritz, and Steve mixed in below prefixed with their name. Thanks much  to Marion for taking the first pass at this.

> On Jul 16, 2016, at 12:10 AM, Timothy B. Terriberry <[email-masked]> wrote:
> 
> Dave Rice wrote:
>> Hi all,
>> 
>> I wanted to point out the posting of three documents in https://datatracker.ietf.org/wg/cellar/documents/.
>> 
>> EBML: https://datatracker.ietf.org/doc/draft-lhomme-cellar-ebml/
> 
> I thought it might be helpful to have some detailed feedback early in the process so I reviewed the EBML document. These are just my opinions as an individual contributor. This should not be taken as a formal chair review in any way, and is by no means a complete review, as I am sure I've missed things. At a minimum, while I noted some typos and grammatical faults, that wasn't my focus, and there are still plenty of places the language could be improved.
> 
> In general, this document uses a lot of terms and concepts before defining or explaining them, without providing cross-references, requiring a good deal of flipping back-and-forth to understand everything. While adding cross-references is one useful step, a more logical organization of the document to explain concepts before introducing constructs which rely on them would reduce the amount of back-and-forth required and make things much easier to understand. I've included some specific suggestions of what should move where below.

I did some drafting to expand the introduction to terminology. A draft is in this PR https://github.com/Matroska-Org/ebml-specification/pull/87 and discussion moved to a separate thread at https://mailarchive.ietf.org/arch/msg/cellar/J7JwDeHQQswIlIRaqAnIpcyQvHI.

> Detailed comments by section follow.
> 
> == Abstract ==
> 
>> "...this document defines an EBML Schema to convey the semantics..."
> It defines a single schema, or it defines HOW schemas convey semantics?

Updated to say: "this document defines how EBML Schemas are created to convey the semantics of an EBML Document” at https://github.com/Matroska-Org/ebml-specification/commit/ae2915c69e7eb161ade2b1d63da7505a3fdfc6eb.

>> "EBML is designed as a binary equivalent to XML and utilizes a storage-efficient approach..."
> I would replace every instance of the word "utilize" (or its conjugations) in this document with "use”.

Re: ‘utilize’ to ‘use’: Marion Jaks fixed most of this https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a663aa959006a4950b45c7e012.
I found one more in the front_matter and fixed it here: https://github.com/Matroska-Org/ebml-specification/commit/c15c275460b19bdeed530f25a31c47dab422959a.

> === EBML specifications ===
> 
> I would just remove the entire top-level section here, so that "Introduction" is Section 1 instead of 1.1, "Notations and Conventions" is Section 2 instead of 1.2, etc.

I dropped all section levels by 1 here: https://github.com/Matroska-Org/ebml-specification/commit/d246912bdd44cbcdbabb483b7006ea8a7a00acba

> == Introduction ==
> 
> Should expand/define XML.

Added a parenthetical "(a framework for structuring data)”. Deserve more? https://github.com/Matroska-Org/ebml-specification/commit/6d408daf4298b9f8338ad972afec393656bb476c

> == Notation and Conventions ==
> 
> The reference to RFC2119 is sufficient, you don't also need the URL.

Removed. https://github.com/Matroska-Org/ebml-specification/commit/6b3cbcedaa6373edb94f3864e4ff11ba1a9dfb8b

> == Security Considerations ==
> 
>> "...only marginally useful and effective data integrity options..."
> This sentence fragment should be turned into an independent sentence.

Paragraph re-ordered and rewritten to:

"EBML itself does not offer any kind of security and does not provide confidentiality. EBML does not provide any kind of authorization. EBML only offers marginally useful and effective data integrity options, such as CRC elements.” https://github.com/Matroska-Org/ebml-specification/commit/c43c113bee8d4f8ae0be0204d2b175d8256a8fe0

>> "Attacks on an EBML reader may include:"
> List formatting.

Formatting error. Fixed at https://github.com/Matroska-Org/ebml-specification/commit/ef1dd12599cb2ce5e75630976dbac9b51a484881.

> This section refers to a number of things defined elsewhere in the document. You should probably either a) provide forward references to the places where these things are defined or b) move this section to the end of the document, after those things are defined.

With the terminology PR at https://github.com/Matroska-Org/ebml-specification/pull/87, this would define Element ID, Element Data Size, Parent Element, EBML Header, and EBML Element before the Security section.

In markdown it’s been hard to make forward references to things that are defined in tables. So I’m thinking after the current pull requests are accepted to work on this issue https://github.com/Matroska-Org/ebml-specification/issues/89 to move some definitions from tables to hierarchical sections so that forward references can work a bit easier.

> A number of the things listed as "attacks" are simply things you said someone writing an EBML Document MUST NOT do (or not doing things you said they MUST do). But in none of those cases do you say what is expected of someone reading a document containing these attacks.

This is true. There’s a few places where the behavior of a ‘reader’ is mentioned, but an EBML Reader is not yet defined nor is its behavior or expectations described in detail. This is a larger issue so I moved it to an issue in the EBML repo. https://github.com/Matroska-Org/ebml-specification/issues/88

> == Variable Size Integer ==
> 
>> "...shall be referred to..."
> "SHALL" is an RFC 2119 keyword, but appears here in lower case. Nothing in 2119 explicitly says that the case matters, so the careful document author will refrain from using these words in non-2119 contexts (i.e., unless they intentionally wish to impose a normative requirement). This is the subject of some debate (<https://www.ietf.org/mail-archive/web/ietf/current/msg73338.html>), but if you do the careful thing, you will avoid getting your document drawn into the argument and getting held up until it is resolved (which may never happen).

I didn’t think these ‘shall’s were too close to the intent of the rfc2119:shall, so I reworded the cases where it was used. https://github.com/Matroska-Org/ebml-specification/commit/49ab2b6adcd2142b33ff15f974051b6c020b34f5

> = VINT_WIDTH =
> 
> How are these bits packed into octets?

In VINT_DATA we already have "If the number of bits required for VINT\_DATA are less than the bit size of VINT\_DATA, then VINT\_DATA may be zero-padded to the left to a size that fits.”

In the Variable Size Integer section I added: "Variable Size Integers SHALL left-pad the VINT\_DATA value with zero bits so that the whole Variable Size Integer is octet-aligned.” https://github.com/Matroska-Org/ebml-specification/commit/5b3afb36d11d96e885b44ca0362931b7651b1095

> (the "VINT Examples" section makes this clear, but you shouldn't just rely on examples) Should there be a limit on the number of zero-value bits?

The limit would be inferred from EBMLMaxIDLength or EBMLMaxSizeLength depending on if the VINT is used as an Element ID or an Element Data Size. Could use help making the language less awkward but I drafted these sentences to add to the end of the VINT_WIDTH section.

"Within the EBML Header the VINT\_WIDTH MUST NOT exceed three bits in length (meaning that the Variable Size Integer MUST NOT exceed four octets in length). Within the EBML Body, when VINTs are used to express an Element ID, the maximum length allowed for the VINT\_WIDTH is one less than the value set in EBMLMaxIDLength. Within the EBML Body, when VINTs are used to express an Element Data Size, the maximum length allowed for the VINT\_WIDTH is one less than the value set in EBMLMaxSizeLength.” https://github.com/Matroska-Org/ebml-specification/commit/e8a1c82ecaee8258e4882adf6ed85f22135a56c5

> If so, what happens if this limit is exceeded?

Not sure, but moved the question to the EBML Reader issue: https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-244650162.

> What are you supposed to do if attempting to read a length (e.g., of a String, which has no defined maximum length) and the entire rest of the file consists of zero bytes?

same.

> The document later defines some context-specific limits, but never answers the question about what you're supposed to do if these limits are violated for any context.
> 
> = VINT_DATA =
> 
>> "...one bit per octet..."
> It would be clearer to say "one out of eight bits", since the bit is not, in fact, taken out of each octet.

Thanks. https://github.com/Matroska-Org/ebml-specification/commit/866a3ddb00cf5d9a1e80bc7a0a4c0326446b9da0

>> "...then VINT_DATA may be zero-padded..."
> Another RFC 2119 keyword ("MAY"). But why is this just a MAY? What other things are you allowed to do if "the number of bits required is less than the bit size of VINT_DATA [sic]”?

Corrected to change the ‘may’ to ‘SHOULD’. https://github.com/Matroska-Org/ebml-specification/commit/698acf450fd87ea1865e3b9bc76a0af93f11b4e7

>> "...a big-endian unsigned integer."
> It might be useful to state up-front that these Variable Size Integers are only unsigned (so, e.g., the reader is not left wondering if instead of zero-padding above they are expected to sign-extend, or something else silly).

The last sentence of the VINT_DATA section is "The VINT\_DATA value MUST be expressed as a big-endian unsigned integer”. Since the VINT_MARKER is simply a single bit and VINT_WIDTH is a tally of sequential zero bits, the concept of signedness seems odd to me to apply to these as well, so I prefer the reference to unsignedness only in the VINT_DATA section rather than in the VINT introduction. Suggestions?

> = VINT Examples =
> 
> This switches from talking about "octet lengths" as is done in the "VINT_WIDTH" section to "octet widths". You should be consistent.

Normalized to ‘octet length’. https://github.com/Matroska-Org/ebml-specification/commit/6d7cb999122636925d0aec40877aba0c3575fe32

> The "Size" column should make it clear that this is just the VINT_DATA size (in bits).

Added sentence to clarify this, "The Size column refers to the size of the VINT\_DATA in bits.": https://github.com/Matroska-Org/ebml-specification/commit/4181a7cdaedce433e38b386dd597639ddb85c9d3.

> == Element ID ==
> 
>> "By default, EBML Element IDs may be encoded... although Element IDs of greater lengths MAY be used..."
> MAY?

Reworded “may/MAY be” to “are”. https://github.com/Matroska-Org/ebml-specification/commit/c1a894ef053e056a70301af8054e106d9c61af11

> You should include a forward reference to EBMLMaxIDLength.

Forward references will be easier to handle if some of the definitions which are currently in tables are moved into sections with their own section headers. I filed an issue about this and will work on it after the current PRs are resolved. https://github.com/Matroska-Org/ebml-specification/issues/89

>> "The VINT_DATA component of the Element ID MUST NOT be set to either all zero values or all one values."
> What is an implementation supposed to do if it encounters a file that violates this restriction? What is the purpose of reserving the various all-ones Element IDs? Or the zero Element ID, for that matter?

Not sure. Steve? Moritz? I filed an issue on this question at https://github.com/Matroska-Org/ebml-specification/issues/90.

>> "The VINT_DATA component of the Element ID MUST be encoded at the shortest valid length."
> What is the definition of "valid”?

I added comments to the issue of defining EBML Reader and valid elements at https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-244797740.

> Is the Element ID 0100 0000 0111 1111 allowed? Although the shortest length required to encode the integer 127 is one octet, that encoding has VINT_DATA set to all ones, which you just said I MUST NOT do. So is one the shortest valid length or two? I can guess what you meant by looking at the Number of Possible Element IDs in the table below, but it would be better to say so explicitly.

We have an example sentence following this which states: "For example, an Element ID with binary encoding of 1011 1111 is valid, whereas an Element ID with binary encoding of 0100 0000 0011 1111 stores a semantically equal VINT\_DATA but is invalid because a shorter VINT encoding is possible.”

I added one additional example sentence to cover the case you mention: "Additionally, an Element ID with binary encoding of 1111 1111 is invalid since the VINT\_DATA section is set to all one values, whereas an Element ID with binary encoding of 0100 0000 0111 1111 stores a semantically equal VINT\_DATA and is the shortest VINT encoding is possible.”

I expanded the example table to cover the all-0 and all-1 VINT_DATA scenarios like:

VINT\_WIDTH | VINT\_MARKER | VINT\_DATA     | Element ID Status
-----------:|-------------:|---------------:|:-----------------
            | 1            |        0000000 | Invalid: VINT\_DATA MUST NOT be set to all 0
0           | 1            | 00000000000000 | Invalid: VINT\_DATA MUST NOT be set to all 0
            | 1            |        0000001 | Valid
0           | 1            | 00000000000001 | Invalid: A shorter VINT\_DATA encoding is available.
            | 1            |        0111111 | Valid
0           | 1            | 00000000111111 | Invalid: A shorter VINT\_DATA encoding is available.
            | 1            |        1111111 | Invalid: VINT\_DATA MUST NOT be set to all 1
0           | 1            | 00000001111111 | Valid

commit at https://github.com/Matroska-Org/ebml-specification/commit/aa3606a2cae295254040567f2c08dd1c35664e1f

>> "The octet length of an Element ID determines its EBML Class."
> I'd add, "...as defined in Table XX" (and give the table a number and a caption).
> 
> Should you say something about the classes for Element IDs of longer lengths, since you explicitly allow them above with an appropriate EBMLMaxIDLength declaration?
> 
> == Element Data Size ==
> 
> You should include a forward reference to EBMLMaxSizeLength.

Waiting on https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "...the VINT_DATA component of the Element Data Size is not required to be encoded at the shortest valid length."
> "REQUIRED" is a 2119 keyword, but "NOT REQUIRED" is not, so you may wish to rephrase this. You may also wish to mention the redundant, equivalent encodings as a potential side channel in your Security Considerations section.

Reworded “not required” to “not mandated”.  https://github.com/Matroska-Org/ebml-specification/commit/c1a894ef053e056a70301af8054e106d9c61af11

Side channel section started here: https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de65e359ce4a3e4501e8e4e5217 (please review).

>> "...both store a semantically equal value."
> It might be helpful to the reader to explicitly say what that value is.

Added "(both 0b00000000111111 and 0b0111111, the VINT\_DATA sections of the examples, represent the integer 63)” to the end of the sentence. https://github.com/Matroska-Org/ebml-specification/commit/ae9371f622c5203f7bc3f83d775edd519edfa6c5

> You should define or forward reference what an "EBML Data Type" is. This is the only mention of it in the document. Did you mean EBML Element Type?
> 
>> "...definition of the EBML Element Types."
> Why is this plural?

Typo. Fixed in https://github.com/Matroska-Org/ebml-specification/commit/5e496766e446558f1387a5add206eb51b4fdacb7?short_path=01f3b0f#diff-01f3b0f2fbf3e44327e6072f64ddc8a0.

>> "If an Empty Element has a 'default' value declared then that default value MUST be interpreted as the value of the Empty Element. If an Empty Element has no 'default' value declared then the semantic meaning of Empty Element is defined as part of the definition of the EBML Element Types."
> This part is confusingly worded. The normative requirement combined with passive voice makes it unclear on whom this requirement is being imposed. I think what you are trying to say is that first, you check to see if the definition of the Element ID corresponding to this element has a declared default value, and, if not, you then take the default from the Element Type associated with that Element ID. But since neither sentence mentions Element ID, this is pretty unclear.

Reworded to "If an Empty Element has a `default` value declared then the EBML Reader MUST interpret the value of the Empty Element as the `default` value. If an Empty Element has no `default` value declared then the EBML Reader MUST interpret the value of the Empty Element as defined as part of the definition of the corresponding EBML Element Type associated with the Element ID.” I reference the requirement imposed onto the EBML Reader to make it less passive, though EBML Reader still needs a definition. https://github.com/Matroska-Org/ebml-specification/commit/197765732acfdbebf0432ef872f041594e2e4bc9?short_path=01f3b0f#diff-01f3b0f2fbf3e44327e6072f64ddc8a0

>> "...SHOULD NOT be used unnecessarily."
> Why not MUST? Can you describe reasons someone might reasonably violate this? What qualifies as "necessary”?

I re-ordered and rewrote the paragraph about Unknown Sized Elements.

"An Element Data Size with all VINT\_DATA bits set to one is reserved as an indicator that the size of the Element is unknown. The only reserved value for the VINT\_DATA of Element Data Size is all bits set to one. An Element with an unknown Element Data Size is referred to as an `Unknown-Sized Element`. Only Master Elements SHALL be Unknown-Sized Elements. Master Elements MUST NOT use an unknown size unless the `unknownsizeallowed` attribute of their EBML Schema is set to true. The use of Unknown-Sized Elements allows for an Element to be written and read before the size of the Element is known. Unknown-Sized Element MUST NOT be used or defined unnecessarily; however if the Element Data Size is not known before the Element Data is written, such as in some cases of data streaming, then Unknown-Sized Elements MAY be used. The end of an Unknown-Sized Element is determined by the beginning of the next element that is not a valid sub-element of the Unknown-Sized Element.”

https://github.com/Matroska-Org/ebml-specification/commit/95bedcc76d3586b1e9778196729651ec61951508

>> "...MUST be a Master-element..."
> I don't think hyphenation is what you want here (hyphenation is used in phrases that function as adjectives or adverbs, not phrases that function as nouns). Might it be clearer to say that unknown Element Data Size values MUST NOT be used unless the element is a Master Element?

Historically this EBML data type had been consistently referred to as “Master-element”. Marion updated it to “Master Element” in https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a663aa959006a4950b45c7e012.

>> "...MAY only..."
> This is not how 2119 MAY is meant to work. I'd rephrase with a different keyword, e.g., again, "...MUST NOT use an unknown size unless the 'unknownsizeallowed' attribute... is set to true." You should forward reference the definition of the "unknownsizeallowed" attribute.

Marion beat me to it: https://github.com/Matroska-Org/ebml-specification/commit/54952268951129a663aa959006a4950b45c7e012.

> How does someone know if an element is a valid sub-element of a Master Element? Can you forward reference some text that defines this?

Will add forward references when working on https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "The end of a Master-element with unknown size is determined by the beginning of the next element that is not a valid sub-element of that Master-element."
> Is an invalid element ID (the all-zeros or all-ones element IDs) considered the beginning of an element that is not a valid sub-elmeent of a Master Element?

Hmmmm. Later in the definition of Master Element it speaks to this: "When EBML is used in transmission or streaming, data that is not part of an EBML Element is permitted to be present within a Master Element if `unknownsizeallowed` is enabled within that Master Element's definition. In this case, the reader should skip data until a valid Element ID of the same level or the next greater level of the Master Element is found.” I think an all-1 or all-0 Element ID would be considered “not part of an EBML Element”.

In this section I changed the sentence you referenced to indicate that the ‘next element’ must be a ‘defined element’: "The end of an Unknown-Sized Element is determined by the beginning of the next element, defined by this document or the corresponding EBML Schema, that is not a valid sub-element of the Unknown-Sized Element.” https://github.com/Matroska-Org/ebml-specification/commit/edcf6659a7b3f378fd1cfc72ced21321e6a15c5a?short_path=01f3b0f#diff-01f3b0f2fbf3e44327e6072f64ddc8a0

> == EBML Element Types ==
> 
> The section header says "EBML Element Type", but earlier text used both "EBML Data Type" and "EBML Element Type", and the tables in this section use "Element Data Type". You should be consistent.

Normalized to "EBML Element Type". https://github.com/Matroska-Org/ebml-specification/commit/c76d630f5b7d16dda4ed4478f97b0d9d5378b363

>> "Each defined EBML Element MUST have a declared Element Type."
> What would it mean for an EBML Element to be "defined" but not have a declared Element Type?

The EBML Schema requires a type declared for each element. I changed the introductory sentence of this section to: "EBML Element are defined by an EBML Schema which MUST declare one of the follow EBML Element Types for each EBML Element.” https://github.com/Matroska-Org/ebml-specification/commit/309898127c64f2bcaea040b8f2fdfb0e521eb1ef

> What does it mean to "have" that declaration? I.e., what normative requirement are you actually attempting to impose here? How does someone know that they have met it?

Good question. Since the declaration are all performed with an XML Document (an EBML Schema), I’ve been thinking that it may make sense to write an XML Schema as an alternate machine-readible definition of that XML Document. I see a few other RFC docs such as https://tools.ietf.org/html/rfc5168 and https://tools.ietf.org/html/rfc5935 so perhaps this is worthwhile here as well. I filed an issue about this at https://github.com/Matroska-Org/ebml-specification/issues/91.

>> "...may be constrained..."
> MAY?

I rewrote that sentence as "An EBML Element Type defines a concept of storing data within an EBML Element that describes such characteristics as length, endianness, and definition.” https://github.com/Matroska-Org/ebml-specification/commit/d031d83aca0307654720b0d76674db5ecc55e6c8

>> "An Signed [sic] Integer Element with a zero-octet length represents an integer value of zero."
> This is only true for Element IDs that do not have a declared default value, right? It would be a good idea to explicitly say that, otherwise it appears to contradict the earlier text in the "Element Data Size" section. The same applies for the defaults declared for every other Element Type.

Added an opening clause about this: "If the EBML Element is not defined to have a `default` value, then a Signed Integer Element with a zero-octet length represents an integer value of zero.” Added the same to other EBML Element Types where needed. https://github.com/Matroska-Org/ebml-specification/commit/80b522b1a0e14fcfd005f1f5cfbfff5ead097279

> You may also wish to mention how the signed integers are stored. I would assume two's complement, but that's not explicitly stated.

Added a clarification: "Signed Integers MUST be stored with two's complement notation with the leftmost bit being the sign bit.” https://github.com/Matroska-Org/ebml-specification/commit/d0d30380d65e23a95090e75941c7a2aff534aae6

>> "A Float Element MUST declare of length of..."
> You meant "...a length of…"

Marion fixed this earlier.

>> "A Float Elements stores a floating-point number as defined in IEEE 754."
> You meant "A Float Element stores…."

Marion fixed this earlier.

> You should include a normative reference to IEEE 754.

Added in https://github.com/Matroska-Org/ebml-specification/commit/1d32d654e2af37824462daefb621ce435f496912

>> "A String Element may declare any length from zero to 'VINTMAX'."
> MAY? VINTMAX is not a single value. Its value depends on the octet width (length?) of the Element Data Size. Is there any restriction on this octet width? If not, I would just phrase this as, "A String Element can have any length, including zero." However, it is obvious that no implementation will be able to support strings of completely arbitrarily length. Is there some required minimum that decoders are required to handle? When is a 32-bit system with no more than 4 GB of RAM justified in rejecting a string as too large? How about an embedded media player in someone's television? How would someone writing an EBML document know when writing a string if all decoders will be able to handle it?

Good question. Currently and historically there hasn’t been a conveyed limit on string lengths. I moved a comment to https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-244822687 as I think this needs some discussion.

>> "A String Element may either be empty (zero-length)..."
> MAY? The previous Element Types all define what it means when the length is zero in the section on "Length". Should you do the same for the string types?

Added "If the EBML Element is not defined to have a `default` value, then a String Element with a zero-octet length represents an empty string.” and an equivalent for the UTF-8 definition. Is there a better way to say this?
https://github.com/Matroska-Org/ebml-specification/commit/87205a7f03082b5f8ac16ec089d8f910a058c7cd

> Reference for ASCII?

Added normative reference for RFC0020. https://github.com/Matroska-Org/ebml-specification/commit/75b2b73bcbcb20730a08903e08e686e4749d7d68

> What is a decoder supposed to do when it encounters a string with octets outside of the Printable ASCII range (excluding zero octets at the end)?

Moved question to https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-244824292.

>> "Octets with all bits set to zero may follow the string value when needed."
> MAY? When is this "needed”?

For example, when the string value of a file is edited to reduce the size of the string by one octet. There is not enough room for a Void Element (needs two octets) so the string could be null padded to prevent having to rewrite the file. Or honestly the length of the Element Data Size could be extended by using an extra byte to write the length.

I extended the sentence to "Octets with all bits set to zero may follow the string value when needed, such as reducing the length of a stored string while maintaining the same Element Data Size.” https://github.com/Matroska-Org/ebml-specification/commit/1c37cba202442ed990a6fa2aeab08cd03ebcd029

> Are these zero bytes meant to be part of the string, or is a string with one or more zero bytes appended to it semantically equivalent to a string without any zero bytes appended to it?

If think so (Steve, Moritz, confirm?). I added a sentence to say "A string with one or more octets with all bits set to zero and a string without one or more octets with all bits set to zero are semantically equal.” https://github.com/Matroska-Org/ebml-specification/commit/1c37cba202442ed990a6fa2aeab08cd03ebcd029

> If so, in your Security Considerations section, you may wish to mention these equivalent values as a potential side channel.

I have a very basic understanding of side channel attacks and don’t fully understand how this equivalence could be used in one, but started a draft section on security considerations here: https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de65e359ce4a3e4501e8e4e5217. Please review, suggest.

> All of the above questions and comments for the String Element Type also apply to the UTF-8 Element Type.

Applied in https://github.com/Matroska-Org/ebml-specification/commit/6ffb2e87e1c45b8f896d2a149bbdf525a1a0b6cd.

> Again, the reference to RFC2279 is sufficient without the extra URL (which does not even point to the same place as the URL for 2119).

Cleaned up in https://github.com/Matroska-Org/ebml-specification/commit/95cf760bbed368674a3083ebdb5192ab896141a.

> Should the reference to RF2279 be normative? Can the element data be interpreted or validated without understanding the referenced document?

Changed RFC2779 to normative in https://github.com/Matroska-Org/ebml-specification/commit/477e1765603ae9ba980ab0d60a12bda0c897a824.

> The "Date" Element Type says "Endianness: None", but Signed Integers were specified to be big endian.

Oops, changed to big endian in https://github.com/Matroska-Org/ebml-specification/commit/7d4004b5548c6942c09fc8f64de728f7c56fe2ea.

>> "The Data Element MUST contain a Signed Integer..."
> What does it meant to "contain a Signed Integer" here, given that this is the definition of an Element Type, and a Signed Integer is also an Element Type. Maybe it would make more sense to say it stores an integer in the same format as a Signed Integer?

Updated to "The Date Element stores an integer in the same format as the Signed Integer Element that expresses a point in time referenced in nanoseconds from the precise beginning of the third millennium of the Gregorian Calendar in Coordinated Universal Time (also known as 2001-01-01T00:00:00.000000000 UTC).” https://github.com/Matroska-Org/ebml-specification/commit/2b4d4a0692b1d97c5d06a935df03483af46facee

> You should provide a normative reference for UTC.

Added in https://github.com/Matroska-Org/ebml-specification/commit/92b0a07f89a896971490a6adfb19266a6cde7584.

>> "A Master-element may declare any length from zero to 'VINTMAX'."
> Same comments as for strings.

Updated comment at https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-244822761.

>> "The Master-element may also use an unknown length. See the section on Element Data Size for rules that apply to elements of unknown length."
> MAY?

Many ‘may’s reworded or capitalized in https://github.com/Matroska-Org/ebml-specification/commit/d9a04b98ee0d8cf17ece67c4a0843d9f4a4064b6.

> This should include an explicit section cross-reference.

I’ve having trouble getting cross-references working properly through mmark and xml2rfc. I opened a ticket here; https://github.com/miekg/mmark/issues/94.

>> "Elements contained within a Master-element must be defined for use at levels greater than the level of the Master-element."
> MUST? What is a level? What does it mean to be defined for use at a level? What is the level of the Master Element? What does it mean to be valid at a level? You should forward reference text that explains all of this.

Will do cross-references to level after https://github.com/Matroska-Org/ebml-specification/issues/89 and https://github.com/miekg/mmark/issues/94.

>> "...all contained Elements must be valid at level 3."
> MUST?

Fixed in https://github.com/Matroska-Org/ebml-specification/commit/4a6d0939f9a8f6c6bed6cf13715ac93bb57a6820.

>> "For instance is a Master-element..."
> You meant, "For instance, if a Master Element…"

Marion fixed this one.

>> "Element Data stored within Master-elements SHOULD only consist of EBML Elements and SHOULD NOT contain any data that is not part of an EBML Element."
> Why not MUST?

Fixed in https://github.com/Matroska-Org/ebml-specification/commit/4a6d0939f9a8f6c6bed6cf13715ac93bb57a6820

> Can you describe reasons someone might reasonably violate these SHOULDs? You've specified _when_ they are allowed to, but not _why_. Is the "is permitted" case the only case in which it is permitted? If so, you should structure the normative language to make that clear. What does it mean for EBML to be "used in transmission or streaming"? If someone sends you an EBML document over HTTP, and you save it to disk, are you required to strip out any non-element data from Master Elements?

Moved to an issue for now: https://github.com/Matroska-Org/ebml-specification/issues/92

>> "Any data contained with a Master-element that is not part of an Element SHOULD be ignored."
> Why not MUST? Can you describe reasons someone might reasonably violate this SHOULD?

I suppose there’s some forensic scenarios where someone might choose not to ignore such data. At any rate, changed to MUST in https://github.com/Matroska-Org/ebml-specification/commit/b2edf60e6c634e9ad3c01c82c36c5ca4a22b6267.

> Also, you meant "within", not "with”.

Also fixed in https://github.com/Matroska-Org/ebml-specification/commit/b2edf60e6c634e9ad3c01c82c36c5ca4a22b6267.

> Also, to be pedantic, any data contained within a Master Element _is_ part of an element: the Master Element that contains it. Otherwise you are violating the "...MUST NOT contain any data that is not part of an EBML Element" in the "EBML Document" section below.

Changed "is not part of an Element” to "is not part of a Child Element” in https://github.com/Matroska-Org/ebml-specification/commit/7ea610207ec5ed8f470ab0d43b935be7a90d1892. Child Element’s definition in drafted in the terminology branch at https://github.com/Matroska-Org/ebml-specification/commit/347e54bb18c7f6c9b7b65d0ec3cf8889856d9bfb.

> In your Security Considerations section, you may wish to mention this data as a potential side channel.

Included within https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de65e359ce4a3e4501e8e4e5217.

>> "The contents of a Binary element should not be interpreted by the EBML parser."
> What is an EBML parser? This is the first mention of it in the document.

Changed to EBML Reader which already has an issue to be defined. https://github.com/Matroska-Org/ebml-specification/commit/e2ab139149a77605518f23c1ff06013340972535

> == EBML Document ==
> 
>> "An EBML Document MUST start with an EBML Header which declares significant characteristics of the entire EBML Body."
> Is the "which" clause meant to be restrictive or unrestrictive?

“which” -> “that” https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

>> "...MAY only..."
> This is not how 2119 MAY is meant to work.

also reworded in https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

> Since the concept of Master Element and level is used in multiple places in the text prior to this section, it may be helpful to move this short paragraph earlier in the document (possibly expanding it with additional material required to understand the subsequent text without forward references).
> 
> = EBML Header =
> 
>> "The EBML Header may be considered as analogous to an XML Declaration."
> MAY?

Reworded to "The EBML Header of an EBML Document is analogous to the XML Declaration of an XML Document.” https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

>> "All EBML Documents MUST begin with a valid EBML Header."
> You already said this in the preceding section.

Marion already removed this. 

>> "The EBML Header documents the EBML Schema (also known as the EBML DocType) that may be used to semantically interpret the structure and meaning of the EBML Document."
> MAY? What else would you use? What would you do if you didn't use it?

Changed ‘may’ to ‘is’, also in https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

>> "The EBML Header consists of a single Master-element with an Element ID of 'EBML'."
> Previously, you said, "The Element ID MUST be encoded as a Variable Size Integer." How is 'EBML' encoded as a Variable Size Integer? I assume what you actually meant is that one is supposed to take the numeric Element ID frame the table of the element _named_ EBML in Section 1.11.2, but since the reader hasn't seen that table yet, there's no forward reference, a name is not an ID, and nowhere do you explain how those IDs were derived, this is all pretty unclear.

Thanks for pointing that out. Reworded to "The EBML Header consists of a single Master Element with an Element Name of 'EBML' and Element ID of `0x1A45DFA3`.” https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

>> "The EBML Header MUST ONLY contain EBML Elements that are defined as part of the EBML Specification."
> ONLY is not a 2119 keyword (though at least in this case, "only" is consistent with the use of 2119 MUST).

Lowercased to ‘only’. https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

> What is "the EBML Specification"? I would just say, "...this document.”

Reworded to "This document defines precisely what EBML Elements are to be used within the EBML Header, but does not name or define what EBML Elements are to be used within the EBML Body.” https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

> Can an EBML Header contain an EBML Header element?

No. In the EBML Header section we already have "The EBML Header MUST only contain EBML Elements that are defined as part of this document.” and subsequently the ‘parent’ attribute definition that Steve is working on clarifies what is a valid Parent Element of each defined Element.

> Can it contain an EBML Body element?

No, for same reason as above.

> Both are defined in this document.
> 
>> "All EBML Elements within the EBML Header MUST NOT utilize any Element ID with a length greater than 4 octets.  All EBML Elements within the EBML Header MUST NOT utilize any Element Data Size with a length greater than 4 octets."
> What is a decoder supposed to do if these restrictions are violated?

Added this question to the issue on the definition of EBML Reader. https://github.com/Matroska-Org/ebml-specification/issues/88

> = EBML Body =
> 
>> "All data of an EBML Document following the EBML Header may be considered the EBML Body."
> MAY? How else would you consider it?

‘may be considered’ reworded to ‘is’. https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

>> "The end of the EBML Body, as well as the end of the EBML Document that contains the EBML Body, is considered as whichever comes first: the beginning of a new level 0 EBML Header or the end of the file."
> Is it possible to have a non-level-0 EBML Header?

Well a Matroska file could be stored as an attachment within another Matroska file, though in that case the attached Matroska file considered the content of a binary Element. I wrote it this way to imply that the next occurrence of 0x1A45DFA3 is not necessarily the end of an EBML Document.

> Can an EBML Body contain an EBML Header? Can it contain another EBML Body?

Only in binary storage, such as the attachment scenario. Otherwise the Elements that comprise the EBML Body and EBML Header are only defined to occur at certain levels which don’t permit storage at other levels.

>> "...MAY only..."
> This is not how 2119 MAY is meant to work.

“MAY only consist” reworded to “MUST consist only” https://github.com/Matroska-Org/ebml-specification/commit/02d6bab08a1d284461520d9e178a7d91ed83173e

> = EBML Schema =
> 
>> "...XML Schema [1]..."
> This URL should probably instead be at least an informative reference.

Added informative references for XML Schema and XML Document in https://github.com/Matroska-Org/ebml-specification/commit/470d4355143aab5e795be8785faa721fab371b5f.

>> "An EBML Schema must be expressed as well-formed XML."
> MUST? Although you give a URL for XML above, I think this statement means it needs to be a normative reference.

Upgraded in https://github.com/Matroska-Org/ebml-specification/commit/f6882cc793d3f202dc4d3ff2c25d0181d946d705.

> Should there be an XML Schema for valid EBML Schemas?

I was thinking the same. Issue filed in https://github.com/Matroska-Org/ebml-specification/issues/91.

>> "An EBML Document Type is identified by a unique string stored within the EBML Header element called DocType; for example 'matroska' or 'webm'."
> What does it mean to be unique? How is this string stored within the EBML Header? Is DocType an EBML Element? If so, you should call it that, instead of just a string, and provide a forward reference.

If someone reads the EBML specification and has knowledge of Matroska and decides to invent a new EBML Document Type called ‘webm’ it would cause confusion since there is already a ‘webm’ Document Type. Although there are not many EBML Document Types, without a registry system it would require omniscience to know if anyone had already used a given string to identify a document type.

I adjusted the section to say "An EBML Document Type is identified by a string stored within the EBML Header in the DocType Element; for example `matroska` or `webm`. The DocType value for an EBML Document Type SHOULD be unique and persistent.” https://github.com/Matroska-Org/ebml-specification/commit/220fc0297a6fa0e179c8c3829b7996cff659d431

>> "...which are defined in the section on Section 1.11.1.1."
> Delete "the section on”.

Waiting on the resolution of an issue with cross-referencing to fix this.

> Did you mean "that" instead of "which”?

Changed to ‘that’. 

>> "...but MUST NOT conflict is the definitions..."
> You meant "...with the definitions…."

Marion already fixed this.

>> "Within the EBML Schema each EBML Element is defined to occur at a specific level."
> This and the following description of Global EBML Elements, Root Element, and Top-Level Elements would have answered a number of the questions I raised above if presented earlier in the document.

Hoping that the new terminology section address this: https://github.com/Matroska-Org/ebml-specification/pull/87.

>> "...specificied..."
> Typo.

Marion already fixed this.

>> "...specifiied..."
> Typo.

Marion already fixed this.

>> "...the Child EBML Element refers to the EBML Elements..."
> Plural/singular mis-match.

Marion already fixed this.

>> "...that may be immediately contained within that Master-element."
> MAY?

Changed ‘may be’ to ‘is’.

>> "...the Parent EBML Element may be identified..."
> MAY? How else would you identify it?

Changed ‘may be’ to ‘is’, though this section MAY soon change based on a proposal Steve is working on. 

>> "The only exception for this rule are Global EBML Elements which may occur within any Parent EBML Element within the restriction of the Global EBML Element's range declaration."
> MAY? Is the which clause meant to be restrictive or unrestrictive?

Capitalized MAY. Will return here later as Steve’s refactor of the EBML Schema will impact this significantly.

>> "An EBML Schema MUST declare exactly one Element at Level 0 (referred to as the Root Element) that MUST occur exactly once within an EBML Document. The Root Element MUST be mandatory (with minOccurs set to 1) and MUST be defined to occur exactly once (maxOccurs set to 1)."
> Is the second sentence saying the same thing as the first sentence, or are these different requirements? If so, how do they differ? You should provide forward references to minOccurs and maxOccurs.

Dropped the second second in https://github.com/Matroska-Org/ebml-specification/commit/6505b07a8b342e54d2564019553fd119b53ae91d. Will add forward references when working on https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "Note that the EBML and Void Elements may also occur at Level 0 but are not considered to be Root Elements."
> Delete "Note that". You should provide a forward reference for Void Elements.

All “Note that”s at the beginning of sentences are removed in https://github.com/Matroska-Org/ebml-specification/commit/85ac2cd42bba44b9354c4ec3bc2adaa61aa42a48. Will add forward references when working on https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "...in the Section 1.11.2 section."
> Delete the second "section”.

Removed in https://github.com/Matroska-Org/ebml-specification/commit/9006d4d27ceb08688ede2470e9548fabc08a9678. More cleanup on cross-references to be done within https://github.com/Matroska-Org/ebml-specification/issues/89.

> = EBML Schema Element Attributes =
> 
>> "...'<EBMLSchema>' uses the following attributes to define the EBML Schema:"
> These attributes are not thMarion already fixed this.e whole definition of an EBML Schema, surely. I'd just end the sentence after "attributes”.

Marion already fixed this.

>> "Elements may be introduced and deprecated by using the "minver" and "maxver" attributes of ."
> Broken cross-reference.

Will fix when working on https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "The value of the name MUST be in the form of an NCName as defined by the XML Schema specification [3]."
> [3] should be a normative reference, not just a URL.

Corrected in https://github.com/Matroska-Org/ebml-specification/commit/37baf1d18a7871c9df087124581f19587854e25a.

>> "...or may occur..."
> MAY?

Changed "The level notes at what hierarchical depth the EBML Element may occur...” to "The level notes at what hierarchical depth or depths the EBML Element MUST occur if used….”. https://github.com/Matroska-Org/ebml-specification/commit/2dd46718bccfc07b7b6ad57a46dc87c7a15f7e71

>> "If the "global" attribute is not expressed..."
> What does it mean for an attribute to be "expressed"? Did you just mean "present”?

Updated phrasing of the meaning of non-mandatory attributes if not ‘present’ in https://github.com/Matroska-Org/ebml-specification/commit/43bd6f0c527d58c03918e24903e12d190e8bb2a8.
Mostly updating ‘expressed’ or ’used' to ‘present’ and dropped unneeded ‘it is assumed’ language.

>> "The Element ID expressed in hexadecimal notation prefixed by a '0x'."
> Is this meant to be the Variable Size Integer encoding of the element ID, or the the integer that that encoding encodes? From the examples, I believe you mean the former, but you should say so explicitly rather than requiring someone to work it out from the examples. If it is the former, you should also specify the byte order, since the definition of the Variable Size Integer merely specifies how it is encoded, bit-by-bit but here you are giving a single large integer value. Given that everything else is big-endian (except the CRC!), I can guess, but I shouldn't have to guess.

I clarified this in https://github.com/Matroska-Org/ebml-specification/commit/b9306b9a967c0561c4b3ceb770489c1f185e684e by changing "The Element ID expressed in hexadecimal notation” to "The Element ID encoded as a Variable Size Integer expressed in hexadecimal notation”. The sentence ends with "that is read and stored in big-endian order” so I think the endianness is clear from this.

>> "An integer to express the minimal number of occurrences that the EBML Element MUST occur within its Parent Element if its Parent Element is used."
> That's pretty tortured. How about, "An integer expressing the minimum number of occurrences of this EBML Element within its Parent Element. Each instance of the Parent Element MUST contain at least this many instances of this Element." What is one expected to do if it does not contain this many?

Changed as recommended in https://github.com/Matroska-Org/ebml-specification/commit/40fe4cb33d7e5d06e9f58d389f95d425437071ba. Moved the questions about expectations if minoccurs is not met to https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-246198633.

>> "If the Element has no Parent Level (as is the case with Elements at Level 0), then minOccurs refers to constaints on the Element's occurrence within the EBML Document."
> Earlier you said there had to be exactly one element at level 0, and that it could only occur once, and also that its minOccurs had to be set to 1. So why is this sentence necessary? What is the point it's trying to make?

Earlier it said that "An EBML Schema MUST declare exactly one Element at Level 0 (referred to as the Root Element) that MUST occur exactly once within an EBML Document. The EBML and Void Elements may also occur at Level 0 but are not considered to be Root Elements.” So Void is an exception here but still gets a maxOccurs value. For instance:

<EBML/>
<Void/>
<Void/>
<Void/>
<Segment/>
<Void/>
<Void/>
<Void/>

is valid, and only the <Segment> is considered a Root Element. Potentially if Void is changed from 0+ to 1+ then a lot of this language could be cleaned up. I started an issue at https://github.com/Matroska-Org/ebml-specification/issues/93.

>> "If the minOccurs attribute is not expressed for that Element then that Element shall be considered to have a minOccurs value of 0. This value of minOccurs MUST be a positive integer."
> These two sentences seem to literally say that a minOccurs value of 0 MUST be a positive integer. Did you mean "the" instead of "this"? Does that mean that the only way to express a minOccurs of value of 0 is to omit the attribute?

Oops, changing positive to non-negative.https://github.com/Matroska-Org/ebml-specification/commit/4e692a8972cb74175aace57fe2de3a1a285e8b7c

>> "...see the Section 1.11.1.6."
> Delete "the”.

Marion fixed this.

>> "A value to express the maximum number of occurrences that the EBML Element MAY occur within its Parent Element if its Parent Element is used."
> Just as tortured as above, but can be fixed in a similar way. What is one expected to do if there are more than this many occurrences? If maxOccurs is a positive integer, and maxOccurs instances of the element have already been encountered within a Master Element with an unknown size, is the another instance of this element considered "...the beginning of the next element that is not a valid sub-element of that Master-element”?

Good question about the end of unknown-sized elements. I moved the question to https://github.com/Matroska-Org/ebml-specification/issues/94.
Moved question about maxOccurs being exceeded to https://github.com/Matroska-Org/ebml-specification/issues/88#issuecomment-246200237.
And cleaned up the tortured language in https://github.com/Matroska-Org/ebml-specification/commit/ef04a4acc6cd621f6fdb354f463ffa611b35b2bb.

>> "...the Element is an Section 1.11.1.3."
> Broken cross reference.

Will fix with https://github.com/Matroska-Org/ebml-specification/issues/89.

>> "If specified that the value of the EBML Element MUST be within the defined range inclusively."
> Either you meant to remove "that" or this is an incomplete sentence fragment.

Marion fixed this.

>> "See the Section 1.11.1.4..."
> Delete "the”.

Marion fixed this.

>> "A default value may be provided."
> MAY? Is the existence of this attribute sufficient to provide it, or even if the attribute exists, is providing a default value something it merely MAY do?

I removed the sentence and rewrote the following sentence to be "If an Element is mandatory but not written within its Parent EBML Element or stored as an Empty Element, then the EBML Reader of the EBML Document MUST semantically insert this specified default value for the Element.” https://github.com/Matroska-Org/ebml-specification/commit/31a54bbab0ba48d66128543c92254092b95135ba 

>> "If an Element is mandatory but not written within its Parent EBML Element, then the parser of the EBML Document MUST insert the defined default value of the Element."
> What does it mean for an element to be mandatory? I think you mean minOccurs > 0,

Yes, I added that as a parenthetical to the above sentence so "is mandatory (has a minOccurs value greater than zero) but not written”. 

> though that then raises the question of what it means if minOccurs > 1. Are parsers then meant to insert enough copies of the element set to the default value to make up for any and all missing occurrences?

Moved this question to https://github.com/Matroska-Org/ebml-specification/issues/95.

> Also, again, what is a parser? What does it mean for it to insert something? Where does it insert it?

Changed ‘insert’ to ‘semantically interpret’. New version: "If an Element is mandatory (has a minOccurs value greater than zero) but not written within its Parent EBML Element or stored as an Empty Element, then the EBML Reader of the EBML Document MUST semantically interpret the Element as present with this specified default value for the Element.” https://github.com/Matroska-Org/ebml-specification/commit/ac34f1fd3eb366707597ce6985c5109ca0eb6729

>> "As defined within the Section 1.8, the type MUST be set..."
> Delete the first "the”.

Marion got this.

> Section 1.8 does not define that the type MUST be set to anything. I assume you mean that the section defines the content of the element for each type, but that's not what this sentence says.

Sentences re-ordered and clarified to: "The type MUST be set to one of the following values: 'integer' (signed integer), 'uinteger' (unsigned integer), 'float', 'string', 'date', 'utf-8', 'master', or 'binary'. The content of each type is defined within [section on EBML Element Types](#ebml-element-types).” https://github.com/Matroska-Org/ebml-specification/commit/275a8ee33bfbc678b60bbf8c03027d162b77521b

>> "The 'unknownsizeallowed' attribute only applies to Master-elements."
> Does that mean it's legal for a non-Master Element to include an "unknownsizeallowed" in its schema, but someone validating a document with that schema is required to ignore it? Same question for the "recursive" attribute.

Reworded to: "Elements that are not Master Elements MUST NOT set `unknownsizeallowed` to true.” Same approach with recursive. https://github.com/Matroska-Org/ebml-specification/commit/a28c86e5a6803a54f6dc6161cd9de1499c0e495c

>> "If the "maxver" attribute is not used it is assumed that the element has a maximum version equal to the value stored in the "version" attribute of ."
> Broken cross-reference. Also, what does it mean for an attribute to be "used"? Is that like "expressed"? Or does it really just meant "present" again?

Already fixed in https://github.com/Matroska-Org/ebml-specification/commit/43bd6f0c527d58c03918e24903e12d190e8bb2a8. Changed ‘used’ to ‘present’.

> What is one expected to do if "maxver" is smaller than "minver”?

Added: `maxver` MUST be greater than or equal to `minver`. https://github.com/Matroska-Org/ebml-specification/commit/7b6e2653046985d45168dd63b035b3c68c72a597

>> "The '<element>' nodes shall contain..."
> SHALL?

Removed ‘shall’.

>> "The '<documentation>' sub-element may use a 'lang' attribute which may be set to the RFC 5646 value of the language of the element's documentation."
> MAY (two times)? I'm also allowed to set it to some other completely unrelated language? RFC 5646 should be a normative reference.

Reworded to drop one may and capitalize the other: "The `<documentation>` sub-element MAY use a `lang` attribute which is set to the [@!RFC5646] value of the language of the element's documentation.” And RFC5646 is now normative. https://github.com/Matroska-Org/ebml-specification/commit/effffba7f90a8368ad85f47ad0b87e231b0a7d89

>> "The '<documentation>' sub-element may use a 'type' attribute to distinguish the meaning of the documentation."
> MAY?

Capitalized in above commit.

>> "Recommended values for the '<documentation>' sub-element's 'type' attribute include..."
> RECOMMENDED? What do these values mean? Why not MUST? Can you describe reasons someone might reasonably choose to use other "type" attribute values?

Updated to MUST. https://github.com/Matroska-Org/ebml-specification/commit/fbbe8b38e924399b9bffb27e74be48d1d9bc67fd

>> "An '<element>' node that defines an EBML Element which is a Child Element of another Parent Element MUST be stored as an immediate sub-element of the '<element>' node that defines the Parent Element."
> Is it allowed to have two <element> nodes with the same Element ID as children of different Parent Elements? Is one simply expected to duplicate the whole contents of that <element> node under both parents? Are they allowed to have different attribute values for things like "level", "minOccurs", "maxOccurs", etc.? If so, what is the complete list of attributes that are allowed to differ?

I think the answer to the first one is no. Suggest that this one is resolved along with Steve’s refactor of the EBML Schema and use of the parent attribute.

> = Identically Recurring Elements =
> 
>> "An Identically Recurring Element is an Element that may occur..."
> MAY?

Capitalized. https://github.com/Matroska-Org/ebml-specification/commit/a332fe9bd873ff084c1c9be1eb0fd6171cec6ef6

>> "...and optimize the use of EBML in transmission."
> What objective function is being optimized? How do Identically Recurrent Elements optimize it?

Trying to have a non-Matroska example, but added "For instance a pertinent Top-Level Element could be periodically resent within a data stream so that an EBML Reader which starts reading the stream from the middle could better interpret the contents.” https://github.com/Matroska-Org/ebml-specification/commit/fcbb1bbde9465aa4e2fbbbeab11823fc94f3cc7b

>> "Identically Recurring Elements SHOULD include a CRC-32 Element as a Child Element..."
> Why not MUST? Can you describe reasons someone might reasonably violate this?

Ping to Moritz. ;)

> This section is good for describing what someone is supposed to do if the Identically Recurring Elements are not, in fact, identical. Though, you may wish to point out in the Security Considerations that the extraneous copies or copies with an invalid CRC-32 Element are a potential side channel.

Included within https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de65e359ce4a3e4501e8e4e5217.

> = Expression of range [sic] =
> 
>> "The 'range' attribute does not support date EBML Elements."
> The definition of the range attribute said it could be used for Dates.

Oops, dates added. https://github.com/Matroska-Org/ebml-specification/commit/ba0b0d99861933158b14ed2630da50d57cd0dfe4

>> "The 'range' expression may contain whitespace for readability but whitespace within a 'range' expression MUST NOT convey meaning."
> How would whitespace within a 'range' expression convey meaning? I think this is a vague attempt to say _where_ whitespace is allowed, but as vague attempts go, it's pretty vague.

For instance there’s a range value which says "&gt; 0x0p+0”. Slightly easier to read than "&gt;0x0p+0”, but intended to mean the same thing. Also later the prefix `not ` is used which includes a space as `not 0` is more readable than `not0`.

>> "'x-y' where x and y are integers or floats and 'y' must be greater than "x", meaning that the value must be greater than or equal to 'x' and less than or equal to 'y'."
> MUST (two times)?

Capitalized in https://github.com/Matroska-Org/ebml-specification/commit/fa54ea4d365f78b007970b1e908f9e6e521a0ff7. 

> What is one supposed to do if 'y' is less than or equal to 'x'? Is there even any reason to forbid 'x' and 'y' from being the same?

Added a mandate that "`x` MUST be less than `y`.” in https://github.com/Matroska-Org/ebml-specification/commit/444973f62008fdf977c2cad2c9eb014d2a55f672.

>> "'>x' where 'x' is an integer or float, meaning that the value MUST be greater than 'x'."
> Earlier in the definition of the range attribute, you said something about "within the defined range inclusively", but this appears to define a range that excludes its end-point. So which is it, inclusive or exclusive? Is this meant to be treated as >=?

Good catch. I prefer to let the range expression define whether it is inclusive or exclusive, so I removed the work ‘inclusively’ from the sentence you quote. https://github.com/Matroska-Org/ebml-specification/commit/9c24ab7dba5a2f54ace4cf0e50839cd4286e3be6 I also added expressions for greater than or equal, less than, and less than or equal. https://github.com/Matroska-Org/ebml-specification/commit/0d9aef8904a5c4ed9878fc9f779d28ffe5e815f4

We have `x-y` as an inclusive range. Do we need expressions for an exclusive range?

>> "The 'range' may use the prefix 'not' to indicate that the expressed range is negated."
> MAY? What does this do to the inclusivity rules?

I think this is resolved by moving inclusive/exclusive to range expression rather than within the range definition.

>> "Please also see the section on Section 1.11.1.5."
> Delete "the section on”.

Marion fixed this.

>> "...as defined in the C11 standard ISO/IEC 9899:2011 [6]..."
> This should be a normative reference.

Done. https://github.com/Matroska-Org/ebml-specification/commit/96f9893d1f1ed5b822514aa284828ea30a97c7d6

> The discussion on hyphens also leaves out how the hyphen that separates the range is to be disambiguated from a signed (negative) float. Perhaps it's worth defining an EBNF grammar for this field?

Started issue here https://github.com/Matroska-Org/ebml-specification/issues/96.

> = Note on the Use of default attributes to define [sic] Mandatory EBML Elements =
> 
>> "If a Mandatory EBML Element..."
> This is the first time this term has appeared, at least in capitalized form. It appears here with no definition.

Added this sentence to the minOccurs definition: "An EBML Element defined with a `minOccurs` value greater than zero is called a Mandatory EBML Element.” https://github.com/Matroska-Org/ebml-specification/commit/77ef392a86698c1f3b88681eb4bdd127758ddd97

>> "In this case, the default value of the Mandatory EBML Element may be assumed although the EBML Element is not present within its Parent EBML Element."
> MAY? I'm allowed to not do any of that, also?

Correct to a MUST and rewritten to: "In this case, the default value of the Mandatory EBML Element MUST be interpretted by the EBML Reader although the EBML Element is not present within its Parent EBML Element.” https://github.com/Matroska-Org/ebml-specification/commit/b1e7cf9ebd8de3e10a5cff48a574cfbaf16da213

>> "Also in this case the parser of the EBML Document MUST insert the defined default value of the Element."
> Again, what is a parser? What does it mean to insert it? Where does it get inserted? Also, why say this again when you already said it? (with 2119 language both times!) That's just a recipe for having it interpreted two different ways.

Redundant final sentence removed, also in https://github.com/Matroska-Org/ebml-specification/commit/b1e7cf9ebd8de3e10a5cff48a574cfbaf16da213.

>> "...than the EBML Element must be present as well."
> You meant "then”.

Marion fixed this.

> MUST?

Capitalized in https://github.com/Matroska-Org/ebml-specification/commit/8aece89a65fdc2870a6c580fefce71d227beae74.

>> "...then the EBML Element MUST be used."
> What does it mean to be used?

Changed to present, rather than ‘used’. https://github.com/Matroska-Org/ebml-specification/commit/f4d03f6bedf520c763ea517e74b85da18aa03600

> If the Element is an Identically Recurring Element, how do I satisfy this rule (which says I must use every such value) and also the rule that I only use the first one with a valid CRC (or failing that, just the first one)?

Tim, could you restate this question. I’m confused about the “every such value” part.

>> "Is the Parent Element used?"
> What does it mean to be used?

Same. https://github.com/Matroska-Org/ebml-specification/commit/f4d03f6bedf520c763ea517e74b85da18aa03600

>> "Then is storing the EBML Element required?"
> REQUIRED?

Capitalized in https://github.com/Matroska-Org/ebml-specification/commit/8aece89a65fdc2870a6c580fefce71d227beae74.

> The last two lines of this table are also identical to the preceding two.

Good catch, removed in https://github.com/Matroska-Org/ebml-specification/commit/52f908dd57554b32a7bf8b863ee89f10a1e25839.

> = Note on the Use of default attributes to define [sic] non-Mandatory EBML Elements =
> 
> This section appears to simply be repeating something already said elsewhere.

This section seems redundant as you say. I removed it in https://github.com/Matroska-Org/ebml-specification/commit/6b3172965684aaba49a72534bc443e3a1a863f4c.

> = EBML Header Elements =
> 
> You've gone to a lot of trouble to define an XML schema for defining EBML elements, and then this section doesn't use it.

I’ve been uncertain as to how to handle this. For instance there is no means in an XML Schema to define an XML Declaration (the xml equivalent of an EBML Header). To define EBML Header in an EBML Schema would be incomplete since the EBML Schema needs a non-EBMLHeader Root Element.

> Do you think it would be a good idea to at least have the table row names match the schema attribute names? (capitalization, "EBML ID" vs. "id", "Mandatory"/"Multiple" vs. "minOccurs"/"maxOccurs", etc.) Does the lack of a Mandatory attribute in the XML schema and the lack of a definition for "mandatory" in this document mean that only elements defined in this section and the next can ever be Mandatory? Should the specification of invalid values like "-" for Range and Default be taken to mean that those attributes should be treated as "not expressed" or is it also legal to include values like that in an EBML Schema?

The language should be adjusted as noted. I filed a ticket. https://github.com/Matroska-Org/ebml-specification/issues/97

>> "Range       | 1"
> Do I take this to mean that the _only_ valid values for EBMLVersion and EBMLReadVersion are 1? What is one expected to do if one encounters a different version number? E.g., what if EBMLReadVersion is 1 but EBMLVersion is 2?

EBMLVersion as 2 is undefined, there’s no such definition. If CELLAR and/or IETF defines EBML version 2 then this range would be changed from ‘1’ to ‘1-2’.

I added a mandate that version>=readversion in both EBMLVersion and DocTypeVersion in https://github.com/Matroska-Org/ebml-specification/commit/f486236285f0af3b5ad959821eb3f1d989140f46.

>> "The version of EBML parser used to create the EBML Document."
> Why is a parser creating documents?

Changed to EBML Writer and updated the issue: https://github.com/Matroska-Org/ebml-specification/issues/88.

> This is why I ask what you might think is the obvious question, "What is a parser?”

Also in issue #88.

> Should there be a maximum values allowed for EBMLMaxIDLength and EBMLMaxSizeLength? How large of a value can an implementation put for these with a reasonable expectation that it will be supported?

Filed issue: https://github.com/Matroska-Org/ebml-specification/issues/98.

> = Global elements (used everywhere in the format) [sic] =
> 
>> "All Top-Level Elements of an EBML Document SHOULD include a CRC-32 Element as a Child Element."
> Why not MUST? Can you describe reasons someone might reasonably violate this?

Moritz, ping.

>> "The CRC in use is the IEEE-CRC-32 algorithm as used in the ISO 3309 standard and in section 8.1.1.6.2 of ITU-T recommendation V.42..."
> This should be cited as a normative reference.

Both ISO and ITU references now formatted as normative. https://github.com/Matroska-Org/ebml-specification/commit/8b4eaf9eb6d262ac07e1fbafe30c35026bd9de2d

> Void elements should probably also be called out in the Security Considerations section as a possible side channel.

Included within https://github.com/Matroska-Org/ebml-specification/commit/a3850f30c51c1de65e359ce4a3e4501e8e4e5217.

Thanks much for this comprehensive review!

Dave Rice