Subject: [Cellar] Risk assessment for FFV1 and Matroska

From: "Casey, Michael T" <[email-masked]>
Date: Wed, 21 Sep 2016 18:49:14 +0000

All,

I have a couple of questions from a large digitization project in Europe about the risk associated with choosing FFV1 and Matroska. This is from their email:
 
You guys have chosen for FFv1 last year, but the standardisation process is still going on (IETF, CELLAR) as you know. 
 
What did you think about the risk of going for a not-yet-standardized codec? Did you somehow take into account the chance that the standardized FFv1 will look a little different than the one you're using now?
 
I am wondering if you have any thoughts on this that I could pass along?
 
Thanks,
 
Mike

------------
Mike Casey
Director of Technical Operations
Media Digitization and Preservation Initiative
Indiana University
812-855-8090

https://mdpi.iu.edu/

---

From: Jerome Martinez <[email-masked]>
Date: Wed, 21 Sep 2016 21:48:57 +0200

Le 21/09/2016 à 20:49, Casey, Michael T a écrit :
> [...]

The goal of CELLAR is to describe what is currently the most used FFV1 
(definition of FFV1 version 0/1/3), before creating a new (and breaking 
compatibility) version 4.
So the standard will not look a little different: it will be the 
description of the most used FFV1.

As a proof that we prioritize what you are using instead of the draft of 
specification, an example:
https://www.ietf.org/mail-archive/web/cellar/current/msg00823.html
You can see that we discovered a little (16-bit YUV is not widely used) 
difference between the main implementation and the draft of spec, we 
decided to update the draft of spec instead than creating an 
incompatibility between the standard and the files in the wild (and 
we'll simplify the bitstream format in FFV1 version 4, not yet in the wild).

The most used FFV1 encoder/decoder is FFmpeg.
There may be a risk in case you don't use FFmpeg for encoding. Do you?

Another argument is that as it is lossless compression, it would be 
relatively easy to move from the "little different" bitstream to the 
"standard" without losing quality, in the case we decide (we won't) to 
have a little difference somewhere.

That said, which FFV1 encoder do you use? I can say that I deeply 
reviewed FFmpeg 3.1 (the last one) for FFV1 8-, 10- and 16-bit, for 
Gray, YUV and RGB, with Golomb Rice (8-bit) and Range Coder 
(8/10/16-bit) coders, with version 3 of FFV1, I can say that I created 
another decoder which confirms that the current draft of spec is 
correct, so I can say that I have big doubts that the spec and FFmpeg 
3.1 have or will have a little difference. if you use one of these 
configurations, go ahead, else provide the configuration you use and 
I'll prioritize it in my review.

The same applies to Matroska but I am less involved in it, I can only 
say that the idea is the same and the risk is only in case you use tools 
we don't check (e.g. if you use a super rare muxer) and in case you use 
super rare Matroska features (e.g. encryption, deprecated signature...), 
else we review files from several muxers e.g. tools using libmatroska or 
FFmpeg. Our priority is to have no "little difference" with the most 
used muxers.

Jérôme

---

From: Dave Rice <[email-masked]>
Date: Wed, 21 Sep 2016 16:02:23 -0400

Hi all,

> On Sep 21, 2016, at 3:48 PM, Jerome Martinez <[email-masked]> wrote:
> 
> The goal of CELLAR is to describe what is currently the most used FFV1 (definition of FFV1 version 0/1/3), before creating a new (and breaking compatibility) version 4.
> So the standard will not look a little different: it will be the description of the most used FFV1.

This is clarified in the charter <https://datatracker.ietf.org/wg/cellar/charter/> where we are standardizing the existing formats and working on the next version of the formats.

> As a proof that we prioritize what you are using instead of the draft of specification, an example:
> You can see that we discovered a little (16-bit YUV is not widely used) difference between the main implementation and the draft of spec, we decided to update the draft of spec instead than creating an incompatibility between the standard and the files in the wild (and we'll simplify the bitstream format in FFV1 version 4, not yet in the wild).
> 

Isn't this just as true with libav as well. Is there another FFV1 encoder beyond FFmpeg and libav?

> Another argument is that as it is lossless compression, it would be relatively easy to move from the "little different" bitstream to the "standard" without losing quality, in the case we decide (we won't) to have a little difference somewhere.
> 
> That said, which FFV1 encoder do you use? I can say that I deeply reviewed FFmpeg 3.1 (the last one) for FFV1 8-, 10- and 16-bit, for Gray, YUV and RGB, with Golomb Rice (8-bit) and Range Coder (8/10/16-bit) coders, with version 3 of FFV1, I can say that I created another decoder which confirms that the current draft of spec is correct, so I can say that I have big doubts that the spec and FFmpeg 3.1 have or will have a little difference. if you use one of these configurations, go ahead, else provide the configuration you use and I'll prioritize it in my review.

Mike, the beginning of the output of 
ffmpeg -debug 1 -i MikeCaseysConfiguration.mkv -f null -
would answer these questions.

> The same applies to Matroska but I am less involved in it, I can only say that the idea is the same and the risk is only in case you use tools we don't check (e.g. if you use a super rare muxer) and in case you use super rare Matroska features (e.g. encryption, deprecated signature...), else we review files from several muxers e.g. tools using libmatroska or FFmpeg. Our priority is to have no "little difference" with the most used muxers.

I think it's possible that some Matroska muxer could have exploited the vagueness of earlier versions of the Matroska spec so that what could be earlier argued as valid is now clarified as not valid by the CELLAR work. This is sort of the entire point of standardizing the documentation though. For instance, someone could have a file with CRC-32 in the middle of a Master Element of have <EBML><Segment><Segment><CRC-32><Segment> are argue for that as not restricted from pre-CELLAR documents whereas it is clearly invalid from current CELLAR drafts. But for most sane muxers I think a reverse compatibility issue is not a big concern. As part of the PREFORMA project I've been reviewing the validity of ~100,000 Matroska files from the Internet Archive and such files as the scenario I mentioned almost never happen.

Also on the topic, when we worked on the proposal for the working group, there was some concern that with the movement of change control and custody of the formats from Matroska and FFmpeg to the IETF that the IETF could change the specs to create a spec that do not match the existing use of the formats. I think this is unlikely as a proposal to the working group to break the formats would likely not be well-received unless it is not to fix something or close a design loophole. Many of the working group members also maintain software that is using these formats and having a specification for the format that is different from the use of the format is not likely to get wide support from the group.

Best Regards,
Dave Rice

---

From: "Peter B." <[email-masked]>
Date: Thu, 22 Sep 2016 09:17:53 +0200

Dear Michael,

Since we started using FFV1 at the Austrian Mediathek in 2010, without
assuming it'd become any standard soon, we worked on risk-assessment
considerations.

If you're interested, I'll write in more detail later :)

Kind regards,
Peter

---

From: "Peter B." <[email-masked]>
Date: Thu, 22 Sep 2016 09:22:10 +0200

> There may be a risk in case you don't use FFmpeg for encoding. Do you?

To my knowledge, there are only 2 implementations existing for FFV1
encoder/decoder:
1) FFmpeg
2) LibAV

Yet, both originate from the very same code, and FATE tests should make
sure that they don't deviate beyond compatibility level.

Therefore, I think the danger of having completely different FFV1 files
by a badly implemented, proprietary 3rd party encoder is very low :)

Does anyone of you know of another FFV1 encoder/decoder?
(MediaConch would only count as decoder, I guess :))

Kind regards,
Peter

---

From: "Casey, Michael T" <[email-masked]>
Date: Thu, 22 Sep 2016 16:50:07 +0000

Peter,

Yes, I'm interested and would like to know more details. I suspect that the European project that I am dialoguing with would also be interested.

Thanks,

Mike

-----Original Message-----
From: Cellar [mailto:[email-masked]] On Behalf Of Peter B.
Sent: Thursday, September 22, 2016 3:18 AM
To: [email-masked]
Subject: Re: [Cellar] Risk assessment for FFV1 and Matroska

Dear Michael,

Since we started using FFV1 at the Austrian Mediathek in 2010, without assuming it'd become any standard soon, we worked on risk-assessment considerations.

Kind regards,
Peter

> I am wondering if you have any thoughts on this that I could pass along?

_______________________________________________
Cellar mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/cellar

---

From: "Peter B." <[email-masked]>
Date: Wed, 28 Sep 2016 13:20:28 +0200

Dear Mike,
Dear list :)

At the Mediathek, we have never written a formal paper - but we've put
together certain aspects to consider - and how we are planning to deal
with them.

I've now finally taken the time to write them down in a way that's more
suitable for public viewing:

http://download.das-werkstatt.com/pb/mthk/info/video/risk_assessment.html

I wrote it in a way which is short enough, so people read it - and not
too technical, while being detailed enough to reproduce our decisions
and considerations.

Please let me know if you find it useful or have comments :)

Kind regards,
Pb

---

From: Reto Kromer <[email-masked]>
Date: Wed, 28 Sep 2016 13:54:33 +0200

Peter B. wrote:

>I've now finally taken the time to write them down in a way
>that's more suitable for public viewing:
>

In my opinion, this is a very useful document for advocating
purposes. Thank you very much indeed, Peter!

Best regards, Reto

AV Preservation by reto.ch
chemin du Suchet 5 | 1024 Ecublens | Switzerland
Web: https://reto.ch | Twitter: @retoch

---

From: "Peter B." <[email-masked]>
Date: Wed, 28 Sep 2016 14:35:56 +0200

> purposes. Thank you very much indeed, Peter!

Happy to hear you find it useful :D
Thank you for the compliments!

I hope it doesn't sound like an Open Source evangelism paper.
It's just so that from my technical point of view, it *is* the open
nature which makes things better for us archivists.

Feedback/criticism greatly welcome.

Regards,
Pb