Subject: [Cellar] Proposal to work on Github Pages site

From: Ashley Blewer <[email-masked]>
Date: Sun, 6 Mar 2016 16:39:26 -0500

Hey all!

There's been some discussion about a collaborative framework with which to
continue work on the Matroska specification in a code-development context
after we've had conversations on this list.

I'd like to propose we migrate the Matroska specification to Github. As I
mentioned before, I'm happy to carry this transition to completion on a
MatroskaOrg-hosted or CELLAR-hosted Github organizational account, but
would like to hear from everyone on which site is the best for a
collaborative environment.

Option 1 <http://ablwr.github.io/mkv_jekyll_poc/>: Jekyll-based website
that automatically translates Markdown to HTML.
Option 1 repo: https://github.com/ablwr/mkv_jekyll_poc
*Please note this currently follows a Jekyll theme standard but can be made
to look like the Matroska site.*

The pros and cons are both in terms of ease of use. The benefits are
working in the more human-readable Markdown instead of HTML. The negative
is that running a local Jekyll server to preview work before submitting a
request is harder than opening up a static site.

Also, tables can be difficult to read and create in Markdown. However, the
primary table located on the specification page
<https://matroska.org/technical/specs/index.html> is rendered via XML and
would not have to be rewritten in Markdown (although this example does have
it written in Markdown).

Option 2 <http://ablwr.github.io/mkv_pages_poc/>: Standard HTML website.
Option 2 repo: https://github.com/ablwr/mkv_pages_poc
*This has been translated to replicate the Matroska site.*

Like I said above, previewing work doesn't require running a local server.
But work is harder to read because it's in HTML and most changes will be
very text-based anyway.

After we come to a decision, I can move relevant specification details to
this site and it can be used to make changes to the standard after
decisions are made here on CELLAR.

My best,

Ashley Blewer

---

From: Steve Lhomme <[email-masked]>
Date: Sun, 13 Mar 2016 11:24:12 +0100

2016-03-06 22:39 GMT+01:00 Ashley Blewer <[email-masked]>:
> MatroskaOrg-hosted or CELLAR-hosted Github organizational account, but would
> like to hear from everyone on which site is the best for a collaborative
> environment.
>
> Option 1: Jekyll-based website that automatically translates Markdown to
> HTML.
> Please note this currently follows a Jekyll theme standard but can be made
> to look like the Matroska site.
>
> The pros and cons are both in terms of ease of use. The benefits are working
> in the more human-readable Markdown instead of HTML. The negative is that
> running a local Jekyll server to preview work before submitting a request is
> harder than opening up a static site.
>
> primary table located on the specification page is rendered via XML and
> Option 2: Standard HTML website.
> This has been translated to replicate the Matroska site.
>
> this site and it can be used to make changes to the standard after decisions
> are made here on CELLAR.

I never heard of Jekyll before but it seems interesting. That would be
for the RFC based specs, so maybe the table of elements will go in the
end or be presented differently. Until the RFC work is done we should
keep the current specs as they are, just fixing things when we find
issues.
So as a starting project I think the Markdown approach is good. We can
always go back to pure HTML and continue from there later. While the
opposite is more work and not trivial.

> My best,
>
> Ashley Blewer
>
> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar
>

-- 
Steve Lhomme
Matroska association Chairman

---

From: Ashley Blewer <[email-masked]>
Date: Sun, 13 Mar 2016 10:24:03 -0400

We can always dedicate the table of elements to its own page -- the current
page is pretty lengthy. Unless there are others who strongly favor the
other approach, I can continue working on the Jekyll-based site and it will
be ready for future collaboration with version-control built in.

Steve, do you think the Jekyll site should mimic/replicate exactly the
current site design or is some change in aesthetics fine? Don't know if
we're tied to having the Github Page site look like Matroska.org or if,
since it's for spec viewing and work in particular, it's fine to appear
differently.

Thanks much!

Ashley

On Sun, Mar 13, 2016 at 6:24 AM, Steve Lhomme <[email-masked]> wrote:

> would
> working

---

From: Steve Lhomme <[email-masked]>
Date: Mon, 14 Mar 2016 09:29:39 +0100

2016-03-13 15:24 GMT+01:00 Ashley Blewer <[email-masked]>:
> page is pretty lengthy. Unless there are others who strongly favor the other
> approach, I can continue working on the Jekyll-based site and it will be
> ready for future collaboration with version-control built in.
>

The design doesn't matter too much. As long as it's easy to read and navigate.

---

From: Dave Rice <[email-masked]>
Date: Thu, 17 Mar 2016 22:45:40 -0400

Hi all,

To test this out, I submitted a Pull Request to Ashley’s mkv_jekyll_poc repository. I am open to either matroska-spec-repo approach as Ashley demonstrated in GitHub; however I have a preference for the jekyll version as we could work on the specifications in Markdown rather than directly in HTML. I find Markdown patches a lot easier to read than HTML patches.

The pull request is here: https://github.com/ablwr/mkv_jekyll_poc/pull/1.

The PR mainly has two changes:

- The introduction is rephrased to say that the specification is WIP and associate it to CELLAR WG.

> This document is a work-in-progress specification defining the Matroska file format as part of the [IETF Cellar working group](https://datatracker.ietf.org/wg/cellar/charter/). But since it's quite complete it is used as a reference for the development of libmatroska. Legacy versions of the specification can be found [here](/files/matroska.pdf) (PDF doc by Alexander Noé -- outdated).
> 
>  For a simplified diagram of the layout of a Matroska file, see the [Diagram page](../diagram/index.html).

I could also move the reference to Alexander Noé’s document into a new `Legacy Matroska` section to reference historical Matroska documents while contextualizing them as superseded by CELLAR’s work.

- I also removed the section of the Matroska specification that is redundant to the EBML specification. Instead I added a section that states that Matroska is dependent on EBML like this:

> ### Basis in EBML    
>                     
> Matroska is a Document Type of EBML (Extensible Binary Meta Language). This specification is dependent on the [EBML Specification](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown). For an understanding of Matroska's EBML Schema, see in particular the sections of the EBML Specification covering [EBML Element Types](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-element-types), [EBML Schema](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-schema), and [EBML Structure](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#structure).

One disadvantage of the mkv_jekyll_poc repo over the mkv_pages_poc repo is the presentation of the Matroska Element table, http://ablwr.github.io/mkv_jekyll_poc/#elements-semantic, but I am interested in the idea of re-designing the presentation of the table.

Thanks,
Dave Rice

> On Mar 14, 2016, at 4:29 AM, Steve Lhomme <[email-masked]> wrote:
>

---

From: Ashley Blewer <[email-masked]>
Date: Thu, 24 Mar 2016 21:38:29 -0400

Hi all,

I intentionally haven't merged this PR since it's being used as an example.
I am happy to merge it in, but I myself am not representative of
MatroskaOrg so it doesn't feel right for me to approve of Dave's work on
this.

Would it be better if this demo was brushed up (that's on me) and moved to
MatroskaOrg Github account and work can be commented on via the listserv
and approved there?

Let me know your thoughts!

Ashley

On Thu, Mar 17, 2016 at 10:45 PM, Dave Rice <[email-masked]> wrote:

> Hi all,
>
> To test this out, I submitted a Pull Request to Ashley’s mkv_jekyll_poc
> repository. I am open to either matroska-spec-repo approach as Ashley
> demonstrated in GitHub; however I have a preference for the jekyll version
> as we could work on the specifications in Markdown rather than directly in
> HTML. I find Markdown patches a lot easier to read than HTML patches.
>
> > This document is a work-in-progress specification defining the Matroska
> file format as part of the [IETF Cellar working group](
> https://datatracker.ietf.org/wg/cellar/charter/). But since it's quite
> complete it is used as a reference for the development of libmatroska.
> Legacy versions of the specification can be found
> [here](/files/matroska.pdf) (PDF doc by Alexander Noé -- outdated).
> >
> I could also move the reference to Alexander Noé’s document into a new
> `Legacy Matroska` section to reference historical Matroska documents while
> contextualizing them as superseded by CELLAR’s work.
>
> - I also removed the section of the Matroska specification that is
> redundant to the EBML specification. Instead I added a section that states
> that Matroska is dependent on EBML like this:
>
> > ### Basis in EBML
> >
> > Matroska is a Document Type of EBML (Extensible Binary Meta Language).
> This specification is dependent on the [EBML Specification](
> https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown).
> For an understanding of Matroska's EBML Schema, see in particular the
> sections of the EBML Specification covering [EBML Element Types](
> https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-element-types),
> [EBML Schema](
> https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-schema),
> and [EBML Structure](
> https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#structure
> ).
>
> One disadvantage of the mkv_jekyll_poc repo over the mkv_pages_poc repo is
> the presentation of the Matroska Element table,
> http://ablwr.github.io/mkv_jekyll_poc/#elements-semantic, but I am
> interested in the idea of re-designing the presentation of the table.
>
> Thanks,
> Dave Rice
>
>
> other
> >>>> There's been some discussion about a collaborative framework with
> which
> >>>> to
> >>>> I'd like to propose we migrate the Matroska specification to Github.
> As
> >>>> I

---

From: Dave Rice <[email-masked]>
Date: Sat, 26 Mar 2016 18:42:27 -0400

Hi all,

> On Mar 24, 2016, at 9:38 PM, Ashley Blewer <[email-masked]> wrote:
> 
> Hi all,
> 
> I intentionally haven't merged this PR since it's being used as an example. I am happy to merge it in, but I myself am not representative of MatroskaOrg so it doesn't feel right for me to approve of Dave's work on this.
> 
> Would it be better if this demo was brushed up (that's on me) and moved to MatroskaOrg Github account and work can be commented on via the listserv and approved there?

I’ll leave this to WG Chairs, Steve, and Moritz and whomever contributes to clarifying a contribution and approval process for the Matroska spec, now that the one for EBML is getting very close to completed (I hope).

The current Matroska spec is on a website, in Drupal posts, which is not so RFC-like and doesn’t offer the same list of features for collaboration as github. I suggest we work Matroska specification website in a GitHub repo (wherever that may be located).

Ashley’s draft repo at https://github.com/ablwr/mkv_jekyll_poc/blob/gh-pages/index.md <https://github.com/ablwr/mkv_jekyll_poc/pull/1> represents only https://matroska.org/technical/specs/index.html. My initial PR at https://github.com/ablwr/mkv_jekyll_poc/pull/1/files <https://github.com/ablwr/mkv_jekyll_poc/pull/1> mostly focuses on removing the redundancy with the EBML specification and instead deferring to that as a dependency.

For next steps I propose moving the Element Table from https://matroska.org/technical/specs/chapters/index.html. I think eventually we could have a build process that combining information from specdata.xml with the rest of the specification to make nice human-readable versions and interactive tables of the Matroska spec, but for specification work I think it may help to separate the EBML Schema data of the table and the rules and specification of Matroska into separate documents (one in xml and one in markdown).

Also I propose moving copies of other Matroska specification pages to the GitHub repo for refinement. I suggest starting with these:
https://matroska.org/technical/diagram/index.html
https://matroska.org/technical/specs/notes.html
https://matroska.org/technical/order/index.html

And then moving on the the pages to focus on the Top Level Elements, such as:
https://matroska.org/technical/specs/chapters/index.html
https://matroska.org/technical/cover_art/index.html (expanding this to cover other types of Attachments)

I think through this work we could separate the current content of matroska.org in information that should become part of a Matroska RFC spec and other data that is intended to support that (such as the info on user guides, faq, downloads, logos, etc).

I’m a bit inspired by the IETF work on Opus where there is now the Opus RFC https://tools.ietf.org/html/rfc6716 and a supplementary https://www.opus-codec.org website that contextualizes the RFC and integrations non-spec documentation, samples, and community information. I think a similar approach may work with Matroska.

Comments welcome.

Best Regards,
Dave

> On Thu, Mar 17, 2016 at 10:45 PM, Dave Rice <[email-masked] <mailto:[email-masked]>> wrote:
> Hi all,
> 
> The pull request is here: https://github.com/ablwr/mkv_jekyll_poc/pull/1 <https://github.com/ablwr/mkv_jekyll_poc/pull/1>.
> 
> > This document is a work-in-progress specification defining the Matroska file format as part of the [IETF Cellar working group](https://datatracker.ietf.org/wg/cellar/charter/ <https://datatracker.ietf.org/wg/cellar/charter/>). But since it's quite complete it is used as a reference for the development of libmatroska. Legacy versions of the specification can be found [here](/files/matroska.pdf) (PDF doc by Alexander Noé -- outdated).
> >
> > Matroska is a Document Type of EBML (Extensible Binary Meta Language). This specification is dependent on the [EBML Specification](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown <https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown>). For an understanding of Matroska's EBML Schema, see in particular the sections of the EBML Specification covering [EBML Element Types](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-element-types <https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-element-types>), [EBML Schema](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-schema <https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-schema>), and [EBML Structure](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#structure <https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#structure>).
> 
> One disadvantage of the mkv_jekyll_poc repo over the mkv_pages_poc repo is the presentation of the Matroska Element table, http://ablwr.github.io/mkv_jekyll_poc/#elements-semantic <http://ablwr.github.io/mkv_jekyll_poc/#elements-semantic>, but I am interested in the idea of re-designing the presentation of the table.
> 
> Thanks,
> Dave Rice
> 
> 
> > On Mar 14, 2016, at 4:29 AM, Steve Lhomme <[email-masked] <mailto:[email-masked]>> wrote:
> >
> > 2016-03-13 15:24 GMT+01:00 Ashley Blewer <[email-masked] <mailto:[email-masked]>>:
> >> On Sun, Mar 13, 2016 at 6:24 AM, Steve Lhomme <[email-masked] <mailto:[email-masked]>> wrote:
> >>>
> >>> 2016-03-06 22:39 GMT+01:00 Ashley Blewer <[email-masked] <mailto:[email-masked]>>:
> >>>> Hey all!
> >>>>
> >>>> Option 1 repo: https://github.com/ablwr/mkv_jekyll_poc <https://github.com/ablwr/mkv_jekyll_poc>
> >>>> Option 2 repo: https://github.com/ablwr/mkv_pages_poc <https://github.com/ablwr/mkv_pages_poc>
> >>>> [email-masked] <mailto:[email-masked]>
> >>>> https://www.ietf.org/mailman/listinfo/cellar <https://www.ietf.org/mailman/listinfo/cellar>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Steve Lhomme
> > [email-masked] <mailto:[email-masked]>
> > https://www.ietf.org/mailman/listinfo/cellar <https://www.ietf.org/mailman/listinfo/cellar>
> 
>

---

From: Steve Lhomme <[email-masked]>
Date: Mon, 28 Mar 2016 14:19:38 +0200

2016-03-26 23:42 GMT+01:00 Dave Rice <[email-masked]>:
> I am happy to merge it in, but I myself am not representative of MatroskaOrg
> so it doesn't feel right for me to approve of Dave's work on this.

LGTM

> MatroskaOrg Github account and work can be commented on via the listserv and
> approved there?

Not sure, how can we preview the content of this repository as a
webpage ? Can pull requests be visible as web pages too ? That would
make it easier to validate changes.

> I’ll leave this to WG Chairs, Steve, and Moritz and whomever contributes to
> clarifying a contribution and approval process for the Matroska spec, now
> that the one for EBML is getting very close to completed (I hope).

IMO yes, EBML is in a good shape now.

> The current Matroska spec is on a website, in Drupal posts, which is not so
> RFC-like and doesn’t offer the same list of features for collaboration as
> github. I suggest we work Matroska specification website in a GitHub repo
> (wherever that may be located).

Yes. Also the old specs will likely remain as they are, at least a
legacy reference. We'll probably have a different URL for the official
RFC for EBML and Matroska (in addition to being published wherever
RFCs are).

> Ashley’s draft repo at
> https://github.com/ablwr/mkv_jekyll_poc/blob/gh-pages/index.md represents
> only https://matroska.org/technical/specs/index.html. My initial PR at
> https://github.com/ablwr/mkv_jekyll_poc/pull/1/files mostly focuses on
> removing the redundancy with the EBML specification and instead deferring to
> that as a dependency.
>
> For next steps I propose moving the Element Table from
> https://matroska.org/technical/specs/chapters/index.html. I think eventually
> we could have a build process that combining information from specdata.xml
> with the rest of the specification to make nice human-readable versions and
> interactive tables of the Matroska spec, but for specification work I think
> it may help to separate the EBML Schema data of the table and the rules and
> specification of Matroska into separate documents (one in xml and one in
> markdown).

Sounds good.

> Also I propose moving copies of other Matroska specification pages to the
> GitHub repo for refinement. I suggest starting with these:
> I think through this work we could separate the current content of
> matroska.org in information that should become part of a Matroska RFC spec
> and other data that is intended to support that (such as the info on user
> guides, faq, downloads, logos, etc).

We could use /technical/drafts/

> I’m a bit inspired by the IETF work on Opus where there is now the Opus RFC
> https://tools.ietf.org/html/rfc6716 and a supplementary
> https://www.opus-codec.org website that contextualizes the RFC and
> integrations non-spec documentation, samples, and community information. I
> think a similar approach may work with Matroska.

http://specs.matroska.org/ ?

>> > file format as part of the [IETF Cellar working
>> > group](https://datatracker.ietf.org/wg/cellar/charter/). But since it's
>> > quite complete it is used as a reference for the development of libmatroska.
>> > This specification is dependent on the [EBML
>> > Specification](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown).
>> > sections of the EBML Specification covering [EBML Element
>> > Types](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-element-types),
>> > [EBML
>> > Schema](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#ebml-schema),
>> > and [EBML
>> > Structure](https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#structure).
>>
>> >>>> Also, tables can be difficult to read and create in Markdown.
>> >>>> However,
>> >>>> the
>> >>>> would not have to be rewritten in Markdown (although this example
>> >>>> does
>> >>>> have
>> >>>> it written in Markdown).
>> >>>>
>> >>>> After we come to a decision, I can move relevant specification
>> >>>> details
>> >>>> to

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Mon, 28 Mar 2016 11:13:11 -0700

> IMO yes, EBML is in a good shape now.

Are you planning to submit this as an internet-draft?

---

From: Dave Rice <[email-masked]>
Date: Tue, 29 Mar 2016 11:58:51 -0400

> On Mar 28, 2016, at 2:13 PM, Timothy B. Terriberry <[email-masked]> wrote:
> 
> Steve Lhomme wrote:

I think the EBML specification, as seen at https://github.com/MediaArea/ebml-specification/blob/master/specification.markdown, is about ready for a more thorough review. There are a few remaining issues that I can see:

- finishing defining how to express ranges in an EBML Schema for float elements. I suppose we need the same rules for expressing a default.

- Define how to express a conditional or equation as a default value or range in an EBML Schema; for instance I think that saying BlockDuration should default to DefaultDuration is not specific enough, but should explain the relationship between these two elements. It would have to be something like ../../Tracks/TrackEntry/DefaultDuration, though perhaps even this isn't clear enough because the correct TrackEntry must be selected.

- We could probably use a better style for presenting an EBML Element definition than the style currently used at https://github.com/MediaArea/ebml-specification/blob/master/specification.markdown#ebml-header-elements.

There may be a few other open issues, but probably not very many.

Tim, how 'done' should we be before submitting as a draft?

Dave Rice

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 29 Mar 2016 10:26:05 -0700

> Tim, how 'done' should we be before submitting as a draft?

The normal bar for an individual draft is, "There are some words on a page."

---

From: Dave Rice <[email-masked]>
Date: Tue, 29 Mar 2016 13:52:08 -0400

> On Mar 29, 2016, at 1:26 PM, Timothy B. Terriberry <[email-masked]> wrote:
> 
> Dave Rice wrote:

In that case, here's https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown.
dave

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 29 Mar 2016 10:59:31 -0700

> In that case, here's https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown.

It still has to be submitted through the normal datatracker:
https://datatracker.ietf.org/submit/

Submissions are currently closed in preparation for next week's meeting 
in Buenos Aires (as they are two weeks before every meeting, to give 
people time to read the drafts before the meeting). They will re-open on 
Monday.