Subject: [Cellar] Standardizing FLAC

From: Nick Krabbenhoeft <[email-masked]>
Date: Tue, 13 Sep 2016 09:20:42 -0400

What's the status of standardizing FLAC?

I work at NYPL where we are using WAVE32 for our audio preservation
masters. However, WAVE32's have a 4GB size limit, which means that long
media like 2+ hour open-reel recordings are split into part files.

I'm looking at FLAC as a potential solution to my container problem, and I
would be happy to help with standardization efforts.

Best,
Nick
-- 

*Nick Krabbenhoeft | The New York Public Library*
*Head of Digital Preservation*

Special Collections Administration
476 Fifth Avenue, New York, NY 10018-2788
T: (917) 229-9596 | [email-masked]

---

From: Tessa Fallon <[email-masked]>
Date: Tue, 13 Sep 2016 09:52:28 -0400

HI Nick,

To date there has not been much discussion of FLAC as efforts have been
focused on FFV1 and Matroska.  If you're interested in moving this forward,
you're welcome to initiate the discussion,  We did get initial buy-in from
the FLAC developer community when we chartered the working group. I'm not
aware of any crossover between that group and CELLAR at the current time,
and if that is really the case I'd very much like it to change.  Happy to
answer any questions.

Best,

Tessa

On Tue, Sep 13, 2016 at 9:20 AM, Nick Krabbenhoeft <
[email-masked]> wrote:

> _______________________________________________
> Cellar mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cellar
>
>

---

From: Dave Rice <[email-masked]>
Date: Tue, 13 Sep 2016 09:57:14 -0400

Hi Nick,

> On Sep 13, 2016, at 9:20 AM, Nick Krabbenhoeft <[email-masked]> wrote:
> 

AFAIK there's no progress (yet) towards the CELLAR goals on FLAC.

> I work at NYPL where we are using WAVE32 for our audio preservation masters. However, WAVE32's have a 4GB size limit, which means that long media like 2+ hour open-reel recordings are split into part files.
> 
> I'm looking at FLAC as a potential solution to my container problem, and I would be happy to help with standardization efforts.

FLAC doesn't have a limit in that area and includes a lot of other preservation features: checksum of decoded audio in header, audio frames with crcs, ability to store any WAV metadata of the source file via -keep-foreign-metadata. I wrote up some preservation considerations of flac here <http://dericed.com/2013/flac-in-the-archives/>.

If you'd like to contribute I suggest this as first steps:

Should check with FLAC community whether it's preferable to work in https://github.com/xiph/flac or possibly start a new repo such as https://github.com/xiph/flac-specification so that the specification work can be separated into a more specific repository.

Re-format existing document: The existing FLAC spec is in html at https://github.com/xiph/flac/blob/master/doc/html/format.html which is rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, and Matroska we first re-formatted the existing specifications from their original state to markdown (FFV1 was lyx to markdown, EBML and Matroska were HTML to markdown). We have a markdown to rfc process working in the other repos of the working group and the approach could be copied to FLAC.

Produce a Makefile to render the flac spec markdown into rfc form as well as whatever other formats that the community needs. See the version for FFV1 <https://github.com/FFmpeg/FFV1/blob/master/Makefile> and EBML <https://github.com/Matroska-Org/ebml-specification/blob/master/Makefile>.

Once the FLAC spec is in markdown, it could likely use a review and update of RFC2119 terminology and new needed sections for an RFC such as an Abstract and Security Considerations.

At the IETF96 CELLAR working group meeting I volunteered to start the steps listed above, but have been lately focused on EBML, so help or volunteering on FLAC would be greatly appreciated.

[...]

Best Regards,
Dave Rice

---

From: Nithin Mathew Kurien <[email-masked]>
Date: Tue, 13 Sep 2016 19:51:11 +0530

The RF64 file format [1] adopted by EBU is a backward-compatible extension
of WAV supporting files of size more than 4 GB. It is currently supported
by many programs including ffmpeg and libav.

Recently, the ITU has standardised the BW64 file format along with Audio
Definition Model [2,3,4,5]. In addition to large file support, it also
specifies metadata for immersive and object-based audio. It is being used
for encoding next generation immersive audio codecs like MPEG-H 3D Audio,
Dolby AC-4 and DTS:X for UHDTV broadcasts. If FLAC is being standardised by
CELLAR, IMHO it would be good if it would be able to preserve the metadata
in BW64. It may also be necessary to increase the supported channel count
of FLAC, which is currently limited to 8.

[1] http://tech.ebu.ch/docs/tech/tech3306-2009.pdf
[2] https://www.itu.int/rec/R-REC-BS.2088-0-201510-I/en
[3] http://www.itu.int/rec/R-REC-BS.2076-0-201506-I
[4] https://www.itu.int/rec/R-REC-BS.2094-0-201604-I/en
[5] http://www.itu.int/rec/R-REC-BS.2051-0-201402-I

---

From: Ashley Blewer <[email-masked]>
Date: Tue, 13 Sep 2016 10:34:58 -0400

All,

Is anyone within CELLAR currently also involved in the FLAC development
community, or know of a particular person to reach out to? I am also
interested and can facilitate bridging this gap by posting to their
developer's list, et al, but maybe someone else already has an established
relationship, or this may be something suitable for the CELLAR chairs.

Ashley

On Tue, Sep 13, 2016 at 9:52 AM, Tessa Fallon <[email-masked]>
wrote:

>> I'm looking at FLAC as a potential solution to my container problem, and
>> I would be happy to help with standardization efforts.
>>
>> Best,
>> Nick
>> --
>>

---

From: Nick Krabbenhoeft <[email-masked]>
Date: Tue, 13 Sep 2016 10:57:34 -0400

Dave,

I'm happy to help with what you outlined. I think Ashley volunteered to
contact flac-dev, but I can get started on html->md.

Nick

On Tue, Sep 13, 2016 at 9:57 AM, Dave Rice <[email-masked]> wrote:

> FLAC doesn't have a limit in that area and includes a lot of other
> preservation features: checksum of decoded audio in header, audio frames
> with crcs, ability to store any WAV metadata of the source file via
> -keep-foreign-metadata. I wrote up some preservation considerations of flac
> here <http://dericed.com/2013/flac-in-the-archives/>.
>
> Should check with FLAC community whether it's preferable to work in
> https://github.com/xiph/flac or possibly start a new repo such as
> https://github.com/xiph/flac-specification so that the specification work
> can be separated into a more specific repository.
>
> Re-format existing document: The existing FLAC spec is in html at
> https://github.com/xiph/flac/blob/master/doc/html/format.html which is
> rendered at https://xiph.org/flac/format.html. With both FFV1, EBML, and
> Matroska we first re-formatted the existing specifications from their
> original state to markdown (FFV1 was lyx to markdown, EBML and Matroska
> were HTML to markdown). We have a markdown to rfc process working in the
> other repos of the working group and the approach could be copied to FLAC.
>
> Produce a Makefile to render the flac spec markdown into rfc form as well
> as whatever other formats that the community needs. See the version for
> FFV1 <https://github.com/FFmpeg/FFV1/blob/master/Makefile> and EBML
> <https://github.com/Matroska-Org/ebml-specification/blob/master/Makefile>.
>
> Once the FLAC spec is in markdown, it could likely use a review and update
> of RFC2119 terminology and new needed sections for an RFC such as an
> Abstract and Security Considerations.
>
> At the IETF96 CELLAR working group meeting I volunteered to start the
> steps listed above, but have been lately focused on EBML, so help or
> volunteering on FLAC would be greatly appreciated.
>
> [...]
>
> Best Regards,
> Dave Rice
>
>

-- 

*Inspiring Lifelong Learning* | *Advancing Knowledge* | *Strengthening Our
Communities*

---

From: Jerome Martinez <[email-masked]>
Date: Tue, 13 Sep 2016 18:50:45 +0200

Le 13/09/2016 à 15:20, Nick Krabbenhoeft a écrit :
> However, WAVE32's have a 4GB size limit, which means that long media 
> like 2+ hour open-reel recordings are split into part files.
>
> I'm looking at FLAC as a potential solution to my container problem (...)

Beside all other answers, especially the one from Dave citing some 
advantages of FLAC, I think that it is not good to say things done by 
FLAC not better than others.
In the case of the 4GB size limit, there are already updates of the 
original WAV formats for supporting 4GB+ files (RF64 and Wave64) with 
only small changes in the workflow that saying that we should move to 
FLAC due to 4GB size limit is not believable. Not citing QuickTime or 
any other modern container able to support PCM.

I would say that RF64 is the solution to your container problem with a 
very small cost but that you should move to FLAC for other reasons 
(detection of corrupted data, storage cost improvement...). note that 
this is the same reason I recommend FFV1 vs uncompressed video (v210...)

Jérôme