Matroska Stem Files

sam@samwhited.com https://blog.samwhited.com

General Internet Engineering Task Force audio matroska stems djing This document defines a multi-track profile of the Matroska container format for storing stems for use by DJ applications while remaining backwards compatible with existing media players.

Introduction Stem are recordings of individual instruments, or clusters of instruments, used by DJs and music producers for live mixing of music. Historically stem files have been stored as individual audio files, or using patent-encumbered or vendor specific proprietary container formats. The Matroska container format formally specified in is ideally situated as a container for stems. This specification documents a profile for the Matroska container format that allows it to store lossless or lossy stems as well as metadata about the stems and mastering information in a single file for use in DJ applications.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Requirements STEM files have a few basic requirements:

Backwards compatibility with existing media players
The ability to store multiple audio track
The ability to store file-level metadata and track-level metadata
Backwards compatibility when additional tracks have unknown formats that cannot be decoded

Track Layout

Audio Streams Each stem file may contain an arbitrary number of tracks containing audio and MUST include at least three audio tracks (the mixed audio and at least two stems). For stem files meant for live DJ use, it is RECOMMENDED that four or fewer stem tracks be used (as opposed to stem files meant for music production where a DAW may have a significantly larger number of tracks). For ease of decoding each track SHOULD be encoded using the same codec with the same parameters including bitrate, and sample rate. Stems are often recorded with a single channel and only the final mix is in stereo. Stems MAY have a different channel count or layout than the main audio track, however it is RECOMMENDED that all stem tracks maintain the same channel count and layout as the main track and have the same channel balance as their component parts in the final mix. For example, if the final mix is a stereo track that contains a fiddle that is 75% in the right channel and only 25% in the left channel, the stem track for the fiddle would also be in stereo with the stem mostly appearing from the right channel as in the final mix. The first track containing audio data MUST be the final post-mix audio in the default language. All tracks containing the final post-mix audio regardless of language MUST have the Matroska "Default" flag set to "1" (, ). This helps preserve backwards compatibility in media players which do not support this format which typically play the first audio stream found or may select based on the default flag. In addition, the "Enabled" flag for any main tracks MUST be set to "1" (). The remaining audio tracks will be individual stems and MUST have the same effective length as the first track such that playing each stem track from the beginning would result in the same audio (excluding mastering) as the final mix present in the first track. For example, if the original track is three minutes long and the stem file includes a percussion track but the percussion does not start until minute two the percussion stem would still be three minutes long but would contain a minute of silence at the start of the track, or would have a block timestamp () that sets the effective start time to one minute. Each stem track MUST have the Matroska "Default" flag set to "0" and MUST have the "Enabled" flag set to "0". The stem tracks SHOULD NOT have any gain normalization applied to bring the stems up to the same percieved volume. Instead they should retain the same levels as they would have in the final mix present in the default track so that if all stems were played at unity gain the levels would be equivalent to the final mix. Each stem track (ie. all tracks that are not the first track) MUST set the value of the \Segment\Tracks\TrackEntry\Name field () to a short, human-readable, track name for the stem that describes its contents, for example "Percussion" or "Vocals". These names are intended for display in playback applications and therefore should remain concise (generally no more than one word), but no specific format or length requirement is defined. For each stem track a \Segment\Tags\Tag () SHOULD also be set with its target set to the stem track. The tag, if present, MUST contain a SimpleTag element with the TagName field set to "STEM_COLOR" and the TagString field set to a color representing the track in RGB hex format (ie. "#145374").

Digital Signal Processor Because mastering happens post-mix and the stems are pre-mix audio the stem tracks SHOULD NOT have any mastering steps applied. Instead, metadata for configuring a compressor and limiter SHOULD be included in the file's global metadata as simple tags (see ). After mixing, playback applications MAY choose to feed the mix through a Digital Signal Processor (DSP) configured with the limiter and compressor settings read from the metadata. Each binary setting for the compressor or limiter is stored as a floating-point number in the 32-bit and 64-bit binary interchange format, as defined in with the additional restriction that they are limited to a minimum value of 0.0 and a maximum value of 1.0. Because different DSPs may use different ranges or scales for each value the playback software SHOULD interpret the 0-1 values as a linear scale and map them to the range and scale required by the DSP when configuring the DSP for playback. This may result in a loss of fidelity on some DSPs, but this is deemed an acceptable trade off for stem playback which would not normally be able to have a mastering step at all. During production of a stem track, vendor specific metadata MAY be embedded in the Matroska file for more accurately configuring a specific DSP, but if such metadata is included the scaled values SHOULD also be present for those without access to the specific DSP used for the track and such metadata MUST select tag names in such a way that they do not conflict with the tag names defined for the generic compressor or limiter.

Compressor Metadata Compressor metadata tags

Tag Name	Type	Values
COMPRESSOR_ENABLED	UTF-8	"TRUE" or "FALSE"
COMPRESSOR_RATIO	binary	0.0-1.0
COMPRESSOR_OUTPUT_GAIN	binary	0.0-1.0
COMPRESSOR_THRESHOLD	binary	0.0-1.0
COMPRESSOR_ATTACK	binary	0.0-1.0
COMPRESSOR_INPUT_GAIN	binary	0.0-1.0
COMPRESSOR_RELEASE	binary	0.0-1.0
COMPRESSOR_HP_CUTOFF	binary	0.0-1.0
COMPRESSOR_HP_DRY_WET	binary	0.0-1.0

Limiter Metadata Limiter metadata tags

Tag Name	Type	Values
LIMITER_ENABLED	UTF-8	"TRUE" or "FALSE"
LIMITER_RELEASE	binary	0.0-1.0
LIMITER_THRESHOLD	binary	0.0-1.0
LIMITER_CEILING	binary	0.0-1.0

IANA Considerations This memo modifies the "Matroska Tag Names" registry to add the following values: Additions to the "Matroska Tag Names" Registry

Tag Name	Tag Type	Reference
STEM_COLOR	UTF-8	This document,
COMPRESSOR_ENABLED	UTF-8	This document,
COMPRESSOR_RATIO	binary	This document,
COMPRESSOR_OUTPUT_GAIN	binary	This document,
COMPRESSOR_THRESHOLD	binary	This document,
COMPRESSOR_ATTACK	binary	This document,
COMPRESSOR_INPUT_GAIN	binary	This document,
COMPRESSOR_RELEASE	binary	This document,
COMPRESSOR_HP_CUTOFF	binary	This document,
COMPRESSOR_HP_DRY_WET	binary	This document,
LIMITER_ENABLED	UTF-8	This document,
LIMITER_RELEASE	binary	This document,
LIMITER_THRESHOLD	binary	This document,
LIMITER_CEILING	binary	This document,

Security Considerations This document should not affect the security of the Internet.

Normative References Informative References

Acknowledgements Thanks to the members of #matroska on the libera.chat IRC network, and to mosu and JanC in particular, for patiently explaining the basics of the format to me and for all their feedback. Thanks also to the members of the Ardour forums for their feedback on DAWs and mastering. Finally, thanks to the members of the IETF CELLAR working group, especially Steve Lhomme, for their feedback.