Free Lossless Audio Codec
draft-xiph-cellar-flac-00

Abstract

This document defines FLAC, which stands for Free Lossless Audio Codec, a free, open source codec for lossless audio compression and decompression.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on December 7, 2017.

Copyright Notice

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Introduction
• 2. Acknowledgments
• 3. Scope
• 4. Architecture
• 5. Definitions
• 6. Blocking
• 7. Interchannel Decorrelation
• 8. Prediction
• 9. Residual Coding
• 10. Format
• 10.1. Conventions
• 10.2. STREAM
• 10.3. METADATA_BLOCK
• 10.4. METADATA_BLOCK_HEADER
• 10.5. BLOCK_TYPE
• 10.6. METADATA_BLOCK_DATA
• 10.7. METADATA_BLOCK_STREAMINFO
• 10.8. METADATA_BLOCK_PADDING
• 10.9. METADATA_BLOCK_APPLICATION
• 10.10. METADATA_BLOCK_SEEKTABLE
• 10.11. SEEKPOINT
• 10.12. METADATA_BLOCK_VORBIS_COMMENT
• 10.13. METADATA_BLOCK_CUESHEET
• 10.14. CUESHEET_TRACK
• 10.15. CUESHEET_TRACK_INDEX
• 10.16. METADATA_BLOCK_PICTURE
• 10.17. PICTURE_TYPE
• 10.18. FRAME
• 10.19. FRAME_HEADER
• 10.19.1. FRAME HEADER RESERVED
• 10.19.2. BLOCKING STRATEGY
• 10.19.3. INTERCHANNEL SAMPLE BLOCK SIZE
• 10.19.4. SAMPLE RATE
• 10.19.5. CHANNEL ASSIGNMENT
• 10.19.6. SAMPLE SIZE
• 10.19.7. FRAME HEADER RESERVED2
• 10.19.8. CODED NUMBER
• 10.19.9. BLOCK SIZE INT
• 10.19.10. SAMPLE RATE INT
• 10.19.11. FRAME CRC
• 10.20. FRAME_FOOTER
• 10.21. SUBFRAME
• 10.22. SUBFRAME_HEADER
• 10.22.1. SUBFRAME TYPE
• 10.22.2. WASTED BITS PER SAMPLE FLAG
• 10.23. SUBFRAME_CONSTANT
• 10.24. SUBFRAME_FIXED
• 10.25. SUBFRAME_LPC
• 10.26. SUBFRAME_VERBATIM
• 10.27. RESIDUAL
• 10.27.1. RESIDUAL_CODING_METHOD
• 10.27.2. RESIDUAL_CODING_METHOD_PARTITIONED_EXP_GOLOMB
• 10.27.3. RESIDUAL_CODING_METHOD_PARTITIONED_EXP_GOLOMB2
• 10.27.4. ENCODED RESIDUAL
• Authors' Addresses

1. Introduction

This is a detailed description of the FLAC format. There is also a companion document that describes FLAC-to-Ogg mapping.

For a user-oriented overview, see About the FLAC Format.

2. Acknowledgments

FLAC owes much to the many people who have advanced the audio compression field so freely. For instance: - A. J. Robinson for his work on Shorten; his paper is a good starting point on some of the basic methods used by FLAC. FLAC trivially extends and improves the fixed predictors, LPC coefficient quantization, and Exponential-Golomb coding used in Shorten. - S. W. Golomb and Robert F. Rice; their universal codes are used by FLAC's entropy coder. - N. Levinson and J. Durbin; the reference encoder uses an algorithm developed and refined by them for determining the LPC coefficients from the autocorrelation coefficients. - And of course, Claude Shannon

3. Scope

It is a known fact that no algorithm can losslessly compress all possible input, so most compressors restrict themselves to a useful domain and try to work as well as possible within that domain. FLAC's domain is audio data. Though it can losslessly code any input, only certain kinds of input will get smaller. FLAC exploits the fact that audio data typically has a high degree of sample-to-sample correlation.

Within the audio domain, there are many possible subdomains. For example: low bitrate speech, high-bitrate multi-channel music, etc. FLAC itself does not target a specific subdomain but many of the default parameters of the reference encoder are tuned to CD-quality music data (i.e. 44.1 kHz, 2 channel, 16 bits per sample). The effect of the encoding parameters on different kinds of audio data will be examined later.

4. Architecture

Similar to many audio coders, a FLAC encoder has the following stages:

• Blocking (see Section 6). The input is broken up into many contiguous blocks. With FLAC, the blocks may vary in size. The optimal size of the block is usually affected by many factors, including the sample rate, spectral characteristics over time, etc. Though FLAC allows the block size to vary within a stream, the reference encoder uses a fixed block size.
• Interchannel Decorrelation (see Section 7). In the case of stereo streams, the encoder will create mid and side signals based on the average and difference (respectively) of the left and right channels. The encoder will then pass the best form of the signal to the next stage.
• Prediction (see Section 8). The block is passed through a prediction stage where the encoder tries to find a mathematical description (usually an approximate one) of the signal. This description is typically much smaller than the raw signal itself. Since the methods of prediction are known to both the encoder and decoder, only the parameters of the predictor need be included in the compressed stream. FLAC currently uses four different classes of predictors, but the format has reserved space for additional methods. FLAC allows the class of predictor to change from block to block, or even within the channels of a block.
• Residual Coding (See Section 9). If the predictor does not describe the signal exactly, the difference between the original signal and the predicted signal (called the error or residual signal) must be coded losslessly. If the predictor is effective, the residual signal will require fewer bits per sample than the original signal. FLAC currently uses only one method for encoding the residual, but the format has reserved space for additional methods. FLAC allows the residual coding method to change from block to block, or even within the channels of a block.

In addition, FLAC specifies a metadata system, which allows arbitrary information about the stream to be included at the beginning of the stream.

5. Definitions

Many terms like "block" and "frame" are used to mean different things in different encoding schemes. For example, a frame in MP3 corresponds to many samples across several channels, whereas an S/PDIF frame represents just one sample for each channel. The definitions we use for FLAC follow. Note that when we talk about blocks and subblocks we are referring to the raw unencoded audio data that is the input to the encoder, and when we talk about frames and subframes, we are referring to the FLAC-encoded data.

• Block: One or more audio samples that span several channels.
• Subblock: One or more audio samples within a channel. So a block contains one subblock for each channel, and all subblocks contain the same number of samples.
• Blocksize: The number of samples in any of a block's subblocks. For example, a one second block sampled at 44.1 kHz has a blocksize of 44100, regardless of the number of channels.
• Frame: A frame header plus one or more subframes.
• Subframe: A subframe header plus one or more encoded samples from a given channel. All subframes within a frame will contain the same number of samples.
• Exponential-Golomb coding: One of Robert Rice's universal coding schemes, FLAC's residual coder, compresses data by writing the number of bits to be read minus 1, before writing the actual value.
• LPC: Linear predictive coding.

6. Blocking

The size used for blocking the audio data has a direct effect on the compression ratio. If the block size is too small, the resulting large number of frames mean that excess bits will be wasted on frame headers. If the block size is too large, the characteristics of the signal may vary so much that the encoder will be unable to find a good predictor. In order to simplify encoder/decoder design, FLAC imposes a minimum block size of 16 samples, and a maximum block size of 65535 samples. This range covers the optimal size for all of the audio data FLAC supports.

Currently the reference encoder uses a fixed block size, optimized on the sample rate of the input. Future versions may vary the block size depending on the characteristics of the signal.

Blocked data is passed to the predictor stage one subblock (channel) at a time. Each subblock is independently coded into a subframe, and the subframes are concatenated into a frame. Because each channel is coded separately, it means that one channel of a stereo frame may be encoded as a constant subframe, and the other an LPC subframe.

7. Interchannel Decorrelation

In stereo streams, many times there is an exploitable amount of correlation between the left and right channels. FLAC allows the frames of stereo streams to have different channel assignments, and an encoder may choose to use the best representation on a frame-by-frame basis.

• Independent. The left and right channels are coded independently.
• Mid-side. The left and right channels are transformed into mid and side channels. The mid channel is the midpoint (average) of the left and right signals, and the side is the difference signal (left minus right).
• Left-side. The left channel and side channel are coded.
• Right-side. The right channel and side channel are coded

Surprisingly, the left-side and right-side forms can be the most efficient in many frames, even though the raw number of bits per sample needed for the original signal is slightly more than that needed for independent or mid-side coding.

8. Prediction

FLAC uses four methods for modeling the input signal:

• Verbatim. This is essentially a zero-order predictor of the signal. The predicted signal is zero, meaning the residual is the signal itself, and the compression is zero. This is the baseline against which the other predictors are measured. If you feed random data to the encoder, the verbatim predictor will probably be used for every subblock. Since the raw signal is not actually passed through the residual coding stage (it is added to the stream 'verbatim'), the encoding results will not be the same as a zero-order linear predictor.
• Constant. This predictor is used whenever the subblock is pure DC ("digital silence"), i.e. a constant value throughout. The signal is run-length encoded and added to the stream.
• Fixed linear predictor. FLAC uses a class of computationally-efficient fixed linear predictors (for a good description, see audiopak and shorten). FLAC adds a fourth-order predictor to the zero-to-third-order predictors used by Shorten. Since the predictors are fixed, the predictor order is the only parameter that needs to be stored in the compressed stream. The error signal is then passed to the residual coder.
• FIR Linear prediction. For more accurate modeling (at a cost of slower encoding), FLAC supports up to 32nd order FIR linear prediction (again, for information on linear prediction, see audiopak and shorten). The reference encoder uses the Levinson-Durbin method for calculating the LPC coefficients from the autocorrelation coefficients, and the coefficients are quantized before computing the residual. Whereas encoders such as Shorten used a fixed quantization for the entire input, FLAC allows the quantized coefficient precision to vary from subframe to subframe. The FLAC reference encoder estimates the optimal precision to use based on the block size and dynamic range of the original signal.

9. Residual Coding

FLAC uses Exponential-Golomb (a variant of Rice) coding as it's residual encoder. You can learn more about exp-golomb coding on Wikipedia

FLAC currently defines two similar methods for the coding of the error signal from the prediction stage. The error signal is coded using Exponential-Golomb codes in one of two ways:

1. the encoder estimates a single exp-golomb parameter based on the variance of the residual and exp-golomb codes the entire residual using this parameter;
2. the residual is partitioned into several equal-length regions of contiguous samples, and each region is coded with its own exp-golomb parameter based on the region's mean. (Note that the first method is a special case of the second method with one partition, except the exp-golomb parameter is based on the residual variance instead of the mean.)

The FLAC format has reserved space for other coding methods. Some possibilities for volunteers would be to explore better context-modeling of the exp-golomb parameter, or Huffman coding. See LOCO-I and pucrunch for descriptions of several universal codes.

10. Format

This section specifies the FLAC bitstream format. FLAC has no format version information, but it does contain reserved space in several places. Future versions of the format may use this reserved space safely without breaking the format of older streams. Older decoders may choose to abort decoding or skip data encoded with newer methods. Apart from reserved patterns, in places the format specifies invalid patterns, meaning that the patterns may never appear in any valid bitstream, in any prior, present, or future versions of the format. These invalid patterns are usually used to make the synchronization mechanism more robust.

All numbers used in a FLAC bitstream are integers; there are no floating-point representations. All numbers are big-endian coded. All numbers are unsigned unless otherwise specified.

Before the formal description of the stream, an overview might be helpful.

• A FLAC bitstream consists of the "fLaC" marker at the beginning of the stream, followed by a mandatory metadata block (called the STREAMINFO block), any number of other metadata blocks, then the audio frames.
• FLAC supports up to 128 kinds of metadata blocks; currently the following are defined:
  • STREAMINFO: This block has information about the whole stream, like sample rate, number of channels, total number of samples, etc. It must be present as the first metadata block in the stream. Other metadata blocks may follow, and ones that the decoder doesn't understand, it will skip.
  • APPLICATION: This block is for use by third-party applications. The only mandatory field is a 32-bit identifier. This ID is granted upon request to an application by the FLAC maintainers. The remainder is of the block is defined by the registered application. Visit the registration page if you would like to register an ID for your application with FLAC.
  • PADDING: This block allows for an arbitrary amount of padding. The contents of a PADDING block have no meaning. This block is useful when it is known that metadata will be edited after encoding; the user can instruct the encoder to reserve a PADDING block of sufficient size so that when metadata is added, it will simply overwrite the padding (which is relatively quick) instead of having to insert it into the right place in the existing file (which would normally require rewriting the entire file).
  • SEEKTABLE: This is an optional block for storing seek points. It is possible to seek to any given sample in a FLAC stream without a seek table, but the delay can be unpredictable since the bitrate may vary widely within a stream. By adding seek points to a stream, this delay can be significantly reduced. Each seek point takes 18 bytes, so 1% resolution within a stream adds less than 2K. There can be only one SEEKTABLE in a stream, but the table can have any number of seek points. There is also a special 'placeholder' seekpoint which will be ignored by decoders but which can be used to reserve space for future seek point insertion.
  • VORBIS_COMMENT: This block is for storing a list of human-readable name/value pairs. Values are encoded using UTF-8. It is an implementation of the Vorbis comment specification (without the framing bit). This is the only officially supported tagging mechanism in FLAC. There may be only one VORBIS_COMMENT block in a stream. In some external documentation, Vorbis comments are called FLAC tags to lessen confusion.
  • CUESHEET: This block is for storing various information that can be used in a cue sheet. It supports track and index points, compatible with Red Book CD digital audio discs, as well as other CD-DA metadata such as media catalog number and track ISRCs. The CUESHEET block is especially useful for backing up CD-DA discs, but it can be used as a general purpose cueing mechanism for playback.
  • PICTURE: This block is for storing pictures associated with the file, most commonly cover art from CDs. There may be more than one PICTURE block in a file. The picture format is similar to the APIC frame in ID3v2. The PICTURE block has a type, MIME type, and UTF-8 description like ID3v2, and supports external linking via URL (though this is discouraged). The differences are that there is no uniqueness constraint on the description field, and the MIME type is mandatory. The FLAC PICTURE block also includes the resolution, color depth, and palette size so that the client can search for a suitable picture without having to scan them all.
• The audio data is composed of one or more audio frames. Each frame consists of a frame header, which contains a sync code, information about the frame like the block size, sample rate, number of channels, et cetera, and an 8-bit CRC. The frame header also contains either the sample number of the first sample in the frame (for variable-blocksize streams), or the frame number (for fixed-blocksize streams). This allows for fast, sample-accurate seeking to be performed. Following the frame header are encoded subframes, one for each channel, and finally, the frame is zero-padded to a byte boundary. Each subframe has its own header that specifies how the subframe is encoded.
• Since a decoder may start decoding in the middle of a stream, there must be a method to determine the start of a frame. A 14-bit sync code begins each frame. The sync code will not appear anywhere else in the frame header. However, since it may appear in the subframes, the decoder has two other ways of ensuring a correct sync. The first is to check that the rest of the frame header contains no invalid data. Even this is not foolproof since valid header patterns can still occur within the subframes. The decoder's final check is to generate an 8-bit CRC of the frame header and compare this to the CRC stored at the end of the frame header.
• Again, since a decoder may start decoding at an arbitrary frame in the stream, each frame header must contain some basic information about the stream because the decoder may not have access to the STREAMINFO metadata block at the start of the stream. This information includes sample rate, bits per sample, number of channels, etc. Since the frame header is pure overhead, it has a direct effect on the compression ratio. To keep the frame header as small as possible, FLAC uses lookup tables for the most commonly used values for frame parameters. For instance, the sample rate part of the frame header is specified using 4 bits. Eight of the bit patterns correspond to the commonly used sample rates of 8/16/22.05/24/32/44.1/48/96 kHz. However, odd sample rates can be specified by using one of the 'hint' bit patterns, directing the decoder to find the exact sample rate at the end of the frame header. The same method is used for specifying the block size and bits per sample. In this way, the frame header size stays small for all of the most common forms of audio data.
• Individual subframes (one for each channel) are coded separately within a frame, and appear serially in the stream. In other words, the encoded audio data is NOT channel-interleaved. This reduces decoder complexity at the cost of requiring larger decode buffers. Each subframe has its own header specifying the attributes of the subframe, like prediction method and order, residual coding parameters, etc. The header is followed by the encoded audio data for that channel.
• FLAC specifies a subset of itself as the Subset format. The purpose of this is to ensure that any streams encoded according to the Subset are truly "streamable", meaning that a decoder that cannot seek within the stream can still pick up in the middle of the stream and start decoding. It also makes hardware decoder implementations more practical by limiting the encoding parameters such that decoder buffer sizes and other resource requirements can be easily determined. flac generates Subset streams by default unless the "--lax" command-line option is used. The Subset makes the following limitations on what may be used in the stream:
  • The blocksize bits in the FRAME_HEADER (see Section 10.19) must be 0001-1110. The blocksize must be <= 16384; if the sample rate is <= 48000 Hz, the blocksize must be <= 4608.
  • The sample rate bits in the FRAME_HEADER must be 0001-1110.
  • The bits-per-sample bits in the FRAME_HEADER must be 001-111.
  • If the sample rate is <= 48000 Hz, the filter order in LPC subframes (see Section 10.25) must be less than or equal to 12, i.e. the subframe type bits in the SUBFRAME_HEADER (see Section 10.22) may not be 101100-111111.
  • The Rice partition order in an exp-golomb coded residual section (see Section 10.27.2) must be less than or equal to 8.

10.1. Conventions

The following tables constitute a formal description of the FLAC format. Values expressed as u(n) represent unsigned big-endian integer using n bits. n may be expressed as an equation using * (multiplication), / (division), + (addition), or - (subtraction). An inclusive range of the number of bits expressed may be represented with an ellipsis, such as u(m...n). The name of a value followed by an asterisk * indicates zero or more occurrences of the value. The name of a value followed by a plus sign + indicates one or more occurrences of the value.

10.2. STREAM

10.3. METADATA_BLOCK

10.4. METADATA_BLOCK_HEADER

10.5. BLOCK_TYPE

10.6. METADATA_BLOCK_DATA

10.7. METADATA_BLOCK_STREAMINFO

NOTE

• FLAC specifies a minimum block size of 16 and a maximum block size of 65535, meaning the bit patterns corresponding to the numbers 0-15 in the minimum blocksize and maximum blocksize fields are invalid.

10.8. METADATA_BLOCK_PADDING

10.9. METADATA_BLOCK_APPLICATION

10.10. METADATA_BLOCK_SEEKTABLE

NOTE - The number of seek points is implied by the metadata header 'length' field, i.e. equal to length / 18.

10.11. SEEKPOINT

NOTES

• For placeholder points, the second and third field values are undefined.
• Seek points within a table must be sorted in ascending order by sample number.
• Seek points within a table must be unique by sample number, with the exception of placeholder points.
• The previous two notes imply that there may be any number of placeholder points, but they must all occur at the end of the table.

10.12. METADATA_BLOCK_VORBIS_COMMENT

10.13. METADATA_BLOCK_CUESHEET

10.14. CUESHEET_TRACK

10.15. CUESHEET_TRACK_INDEX

10.16. METADATA_BLOCK_PICTURE

10.17. PICTURE_TYPE

Other values are reserved and should not be used. There may only be one each of picture type 1 and 2 in a file.

10.18. FRAME

10.19. FRAME_HEADER

10.19.1. FRAME HEADER RESERVED

FRAME HEADER RESERVED must remain reserved for 0 in order for a FLAC frame's initial 15 bits to be distinguishable from the start of an MPEG audio frame (see also).

10.19.2. BLOCKING STRATEGY

The BLOCKING STRATEGY bit must be the same throughout the entire stream.

The BLOCKING STRATEGY bit determines how to calculate the sample number of the first sample in the frame. If the bit is 0 (fixed-blocksize), the frame header encodes the frame number as above, and the frame's starting sample number will be the frame number times the blocksize. If it is 1 (variable-blocksize), the frame header encodes the frame's starting sample number itself. (In the case of a fixed-blocksize stream, only the last block may be shorter than the stream blocksize; its starting sample number will be calculated as the frame number times the previous frame's blocksize, or zero if it is the first frame).

10.19.3. INTERCHANNEL SAMPLE BLOCK SIZE

10.19.4. SAMPLE RATE

10.19.5. CHANNEL ASSIGNMENT

For values 0000-0111, the value represents the (number of independent channels)-1. Where defined, the channel order follows SMPTE/ITU-R recommendations.

10.19.6. SAMPLE SIZE

10.19.7. FRAME HEADER RESERVED2

10.19.8. CODED NUMBER

The "UTF-8" coding used for the sample/frame number is the same variable length code used to store compressed UCS-2, extended to handle larger input.

if(variable blocksize)
  `u(8...56)`: "UTF-8" coded sample number (decoded number is 36 bits)
else
  `u(8...48)`:"UTF-8" coded frame number (decoded number is 31 bits)

10.19.9. BLOCK SIZE INT

if(`INTERCHANNEL SAMPLE BLOCK SIZE` == 0b0110)
  8 bit (blocksize-1)
else if(`INTERCHANNEL SAMPLE BLOCK SIZE` == 0b0111)
  16 bit (blocksize-1)

10.19.10. SAMPLE RATE INT

if(`SAMPLE RATE` == 0b1100)
  8 bit sample rate (in kHz)
else if(`SAMPLE RATE` == 0b1101)
  16 bit sample rate (in Hz)
else if(`SAMPLE RATE` == 0b1110)
  16 bit sample rate in tens of Hz)

10.19.11. FRAME CRC

CRC-8 (polynomial = x^8 + x^2 + x^1 + x^0, initialized with 0) of everything before the CRC, including the sync code

10.20. FRAME_FOOTER

10.21. SUBFRAME

10.22. SUBFRAME_HEADER

10.22.1. SUBFRAME TYPE

10.22.2. WASTED BITS PER SAMPLE FLAG

10.23. SUBFRAME_CONSTANT

10.24. SUBFRAME_FIXED

10.25. SUBFRAME_LPC

10.26. SUBFRAME_VERBATIM

10.27. RESIDUAL

10.27.1. RESIDUAL_CODING_METHOD

10.27.2. RESIDUAL_CODING_METHOD_PARTITIONED_EXP_GOLOMB

10.27.2.1. EXP_GOLOMB_PARTITION

10.27.2.2. EXP-GOLOMB PARTITION ENCODING PARAMETER

10.27.3. RESIDUAL_CODING_METHOD_PARTITIONED_EXP_GOLOMB2

10.27.3.1. EXP_GOLOMB2_PARTITION

10.27.3.2. EXP-GOLOMB2 PARTITION ENCODING PARAMETER

10.27.4. ENCODED RESIDUAL

The number of samples (n) in the partition is determined as follows:

• if the partition order is zero, n = frame's blocksize - predictor order
• else if this is not the first partition of the subframe, n = (frame's blocksize / (2^partition order))
• else n = (frame's blocksize / (2^partition order)) - predictor order

Copyright (c) 2000-2009 Josh Coalson, 2011-2014 Xiph.Org Foundation

Authors' Addresses