Internet DRAFT - draft-mzanaty-moq-loc

draft-mzanaty-moq-loc







Network Working Group                                          M. Zanaty
Internet-Draft                                             S. Nandakumar
Intended status: Informational                                     Cisco
Expires: 5 September 2024                                    P. Thatcher
                                                               Microsoft
                                                            4 March 2024


                      Low Overhead Media Container
                        draft-mzanaty-moq-loc-03

Abstract

   This specification describes a media container format for encoded and
   encrypted audio and video media data to be used primarily for
   interactive Media over QUIC transport (MOQ), with the goal of it
   being a low-overhead format.  It also defines the LOC Streaming
   Format for the MOQ Common Catalog format for publishers to annouce
   and describe their LOC tracks and for subscribers to consume them.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 5 September 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.










Zanaty, et al.          Expires 5 September 2024                [Page 1]

Internet-Draft               media container                  March 2024


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Notation and Conventions . . . . . . . . . .   3
     1.2.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  Payload Format  . . . . . . . . . . . . . . . . . . . . . . .   4
     2.1.  MOQ Object Mapping  . . . . . . . . . . . . . . . . . . .   4
     2.2.  LOC Header Metadata . . . . . . . . . . . . . . . . . . .   4
       2.2.1.  Common Header Data  . . . . . . . . . . . . . . . . .   5
       2.2.2.  Video Header Data . . . . . . . . . . . . . . . . . .   5
       2.2.3.  Audio Header Data . . . . . . . . . . . . . . . . . .   5
       2.2.4.  Header Data Registration  . . . . . . . . . . . . . .   5
   3.  Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . .   6
     3.1.  Catalog Fields  . . . . . . . . . . . . . . . . . . . . .   6
       3.1.1.  Optional Extensions for Video . . . . . . . . . . . .   6
       3.1.2.  Selection Parameters for Video  . . . . . . . . . . .   7
       3.1.3.  Optional Extensions for Audio . . . . . . . . . . . .   7
       3.1.4.  Selection Parameters for Audio  . . . . . . . . . . .   8
     3.2.  Catalog Examples  . . . . . . . . . . . . . . . . . . . .   8
   4.  Payload Encryption  . . . . . . . . . . . . . . . . . . . . .   8
   5.  Container Serialization . . . . . . . . . . . . . . . . . . .   8
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   8.  Normative References  . . . . . . . . . . . . . . . . . . . .   9
   Appendix A.  Acknowledgements . . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

1.  Introduction

   This specification describes a low-overhead media container format
   for encoded and encrypted audio and video media data to be used
   primarily for interactive Media over QUIC transport (MOQT)
   [MoQTransport], with the goal of it being a low-overhead format.  It
   also defines the LOC Streaming Format for the MOQ Common Catalog
   format [MoQCatalog] for publishers to annouce and describe their LOC
   tracks and for subscribers to consume them.






Zanaty, et al.          Expires 5 September 2024                [Page 2]

Internet-Draft               media container                  March 2024


   "Low-overhead" refers to minimal extra encapsulation as well as
   minimal application overhead when interfacing with WebCodecs
   [WebCodecs].

   The container format description is specified for all audio and video
   codecs defined in the WebCodecs Codec Registry
   [WEBCODECS-CODEC-REGISTRY].  The audio and video payload bitstream is
   identical to the "internal data" inside an EncodedAudioChunk and
   EncodedVideoChunk, respectively, specified in the registry.

   In addition to the media payloads, critical metadata is also
   specified for audio and video payloads.  (Note: Align with MOQT
   terminology of either "metadata" or "header".)

   A primary motivation is to align with media formats used in WebCodecs
   to minimize extra encapsulation and application overhead when
   interfacing with WebCodecs.  Other container formats like CMAF or RTP
   would require more extensive application overhead in format
   conversions, as well as larger encapsultion overhead which may burden
   some use cases like low bitrate audio scenarios.

   This specification can also be used by applications outside the
   context of WebCodecs or a web browser.  While the media payloads are
   defined by referring to the "internal data" of an EncodedAudioChunk
   or EncodedVideoChunk in the WebCodecs Codec Registry, this "internal
   data" is the elementary bitstream format of codecs without any
   encapsulation.  Referring to the WebCodecs Codec Registry avoids
   duplicating it in an identical IANA registry.

   *  Section 2 defines the core media payload formats.

   *  Section 2.2 defines the metadata associated with audio and video
      payloads.

   *  Section 3 describes the LOC Streaming Format bindings to the MoQ
      Common Catalog format including examples.

1.1.  Requirements Notation and Conventions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.







Zanaty, et al.          Expires 5 September 2024                [Page 3]

Internet-Draft               media container                  March 2024


1.2.  Terminology

   TODO

2.  Payload Format

   The WebCodecs Codec Registry defines the contents of an
   EncodedAudioChunk and EncodedVideoChunk for the audio and video codec
   formats in the registry.  The "internal data" in these chunks is used
   directly in this specification as the "LOC Payload" bitstream.  This
   "internal data" is the elementary bitstream format of each codec
   without any encapsulation.

   For video formats with multiple bitstream formats in the WebCodecs
   Registry, such as H.264/AVC or H.265/HEVC, the LOC Payload uses the
   "canonical" format ("avcc" or "hevc", not "annexB") with the
   following additions: * Parameter sets are sent in the bitstream
   before key frames. * 4 byte lengths are sent before each NAL Unit. *
   No start codes or emulation prevention are used in the bitstream. *
   No additional codec configuration information ("extradata") is
   needed.

2.1.  MOQ Object Mapping

   An application object when transported as a [MoQTransport] object is
   composed of a MOQ Object Header and its Payload.  Media objects
   encoded using the container format defined in this specification
   populate the MOQ Object Payload with a LOC Header and LOC Payload as
   shown below.

   The LOC Payload is the "internal data" of an EncodedAudioChunk or
   EncodedVideoChunk.

   +--------------+----------+-----------+
   |  MOQ Object  |  LOC     |  LOC      |
   |  Header      |  Header  |  Payload  |
   +--------------+----------------------+
                  <---------------------->
                     MOQ Object Payload

                     MOQ Object with LOC Container

2.2.  LOC Header Metadata

   The LOC Header carries metadata for the corresponding LOC Payload.
   This metadata provides necessary information for intermediaries such
   as media switches to perform their media switching decisions when the
   payload is inaccessible due to encryption.



Zanaty, et al.          Expires 5 September 2024                [Page 4]

Internet-Draft               media container                  March 2024


   Section Section 2.2.4 provides a framework for registering new LOC
   Header fields that aren't defined by this specification.

2.2.1.  Common Header Data

   The following metadata MUST be captured for each media frame.

   Sequence Number: Identifies a sequentially increasing variable length
   integer that is incremented per encoded media frame.  This may be
   replaced with the Object Sequence from the MOQ Object Header in cases
   where a MOQ Object is exactly one frame.

   Capture Timestamp in Microseconds: Captures the wall-clock time of
   the encoded media frame in a 64-bit unsigned integer.

2.2.2.  Video Header Data

   Flags for frames which are independent, discardable, or base layer
   sync points, as well as temporal and spatial layer identification.
   [Framemarking] .

2.2.3.  Audio Header Data

   Audio Level: Captures the magnitude of the audio level of the
   corresponding audio frame encoded in 7 bits as defined in section 3
   of [RFC6464].

2.2.4.  Header Data Registration

   This section details the procedures to register header data fields
   that might be useful for a particular class of media applications.

   Registering a given metadata field requires the following attributes
   to be specified.

   Shortname: Short name for the metadata.  (Not sent on the wire.)

   Description: Detailed description for the metadata.  (Not sent on the
   wire.)

   ID: Identifier assigned by the registry. (varint)

   Length: Length of metadata value in bytes. (varint)

   Value: Value of metadata. (length bytes)

   Registration of type "Specification Required" is followed for
   registering new metadata in the LOC Header.



Zanaty, et al.          Expires 5 September 2024                [Page 5]

Internet-Draft               media container                  March 2024


3.  Catalog

   A catalog is a MOQT Object that provides information about tracks
   from a given publisher.  A catalog is used by subscribers for
   consuming tracks and by publishers to advertise and describe the
   tracks.  The content of a catalog is opaque to the relays and may be
   end to end encrypted.  A catalog describes the details of tracks such
   as Track IDs and corresponding media configuration details, for
   example, audio/video codec details.

   The LOC Streaming Format uses the MoQ Common Catalog Format
   [MoQCatalog] to describe the content being produced by a publisher.

   Per Sect 5.1 of [MoQCatalog], this document registers an entry in the
   "MoQ Streaming Format Type" table, with the type value 2, the name
   "LOC Streaming Format", and the RFC XXX.

   Every LOC catalog track MUST declare a streaming format type (See
   Sect 3.2.1 of [MoQCatalog]) value of 2.

   Every LOC catalog track MUST declare a streaming format version (See
   Sect 3.2.1 of [MoQCatalog]) value of 1, which is the version
   described in this document.

   Every LOC catalog track MUST declare a packaging type (See Sect 3.2.9
   of [MoQCatalog]) of "loc".

   The catalog track MUST have a track name of "catalog".  A catalog
   object MAY be independent of other catalog objects or it MAY
   represent a delta update of a prior catalog object.  The first
   catalog object published within a new group MUST be independent.  A
   catalog object SHOULD only be published only when the availability of
   tracks changes.

   Each catalog update MUST be mapped to a discreet moq-transport
   object.

3.1.  Catalog Fields

   The MOQ Common Catalog defines the required base fields and optional
   extensions.

3.1.1.  Optional Extensions for Video

   The LOC Streaming Format allows the following optional extensions for
   video media.





Zanaty, et al.          Expires 5 September 2024                [Page 6]

Internet-Draft               media container                  March 2024


   *  temporalId: Identifies the temporal layer/sub-layer encoded,
      starting with 0 for the base layer, and increasing with higher
      temporal fidelity.

   *  spatialId: Identifies the spatial and quality layer encoded,
      starting with 0 for the base layer, and increasing with higher
      fidelity.

   *  depends: Identifies track dependencies for a given track, usually
      for video media with scalable layers in separate tracks.

   *  renderGroup: Identifies a group of time-aligned tracks which
      should be rendered simultaneously.

   *  selectionParams: Selection parameters for media quality, fidelity,
      etc.; see next section.

3.1.2.  Selection Parameters for Video

   Each video track can have the following associated Selection
   Parameters.

   *  codec: Codec information (including profile, level, tier, etc.),
      as defined by the codec registrations listed in
      [WEBCODECS-CODEC-REGISTRY].

   *  framerate: As defined in section 7.8 of
      [WEBCODECS-CODEC-REGISTRY].

   *  bitrate: As defined in section 7.7 and 7.8 of
      [WEBCODECS-CODEC-REGISTRY].

   *  width, height: As defined in section 7.8 of
      [WEBCODECS-CODEC-REGISTRY].

   *  displayWidth, displayheight: As defined in section 7.7 of
      [WEBCODECS-CODEC-REGISTRY].

3.1.3.  Optional Extensions for Audio

   The LOC Streaming Format allows the following optional extensions for
   audio media.

   *  renderGroup: Identifies a group of time-aligned tracks which
      should be rendered simultaneously.

   *  selectionParams: Selection parameters for media quality, fidelity,
      etc.; see next section.



Zanaty, et al.          Expires 5 September 2024                [Page 7]

Internet-Draft               media container                  March 2024


3.1.4.  Selection Parameters for Audio

   Each audio track can have the following associated Selection
   Parameters.

   *  codec: Codec information as defined by the codec registrations
      listed in [WEBCODECS-CODEC-REGISTRY].

   *  bitrate: As defined in section 7.7 and 7.8 of
      [WEBCODECS-CODEC-REGISTRY].

   *  samplerate: As defined in section 7.7 of
      [WEBCODECS-CODEC-REGISTRY].

   *  chanelConfig: As defined in section 7.7 of
      [WEBCODECS-CODEC-REGISTRY].

   *  lang: The primary language of the track, using standard tags from
      [RFC5646].

3.2.  Catalog Examples

   See section 3.4 of the MOQ Common Catalog [MoQCatalog].

4.  Payload Encryption

   When end to end encryption is supported, the encoded payload is
   encrypted with symmetric keys derived from key establishment
   mechanisms, such as [MOQ-MLS], and the payload itself is protected
   using mechanisms defined in [SecureObjects].

5.  Container Serialization

   The wire encoding of the payload conforming to this specification is
   a set of length delimited values as shown below.

   The Bytes is obtained as output of AEAD operation for encrypting the
   Payload with the header data as additional data input.

   +--------+------------+-------+------------+
   | Payload | Bytes | Payload  | Bytes |
   | Len     |  (0)  | Len (1)  |  (1)  | ...
   +--------+------------+-------+------------+

6.  Security Considerations

   TODO




Zanaty, et al.          Expires 5 September 2024                [Page 8]

Internet-Draft               media container                  March 2024


7.  IANA Considerations

   A new IANA registry for LOC Header Metadata is defined and populated
   with the information in section Section 2.2.4.  Specification
   required for new metadata registration.

   This document creates a new entry in the "MoQ Streaming Format"
   Registry (see [MoQTransport] Sect 8).  The type value is 0x002, the
   name is "LOC Streaming Format" and the RFC is XXX.

8.  Normative References

   [MoQTransport]
              Curley, L., Pugin, K., Nandakumar, S., Vasiliev, V., and
              I. Swett, "Media over QUIC Transport", Work in Progress,
              Internet-Draft, draft-ietf-moq-transport-02, 24 January
              2024, <https://datatracker.ietf.org/doc/html/draft-ietf-
              moq-transport-02>.

   [MoQCatalog]
              Nandakumar, S., Law, W., and M. Zanaty, "Common Catalog
              Format for moq-transport", Work in Progress, Internet-
              Draft, draft-wilaw-moq-catalogformat-02, 30 November 2023,
              <https://datatracker.ietf.org/doc/html/draft-wilaw-moq-
              catalogformat-02>.

   [Framemarking]
              Zanaty, M., Berger, E., and S. Nandakumar, "Video Frame
              Marking RTP Header Extension", Work in Progress, Internet-
              Draft, draft-ietf-avtext-framemarking-15, 26 July 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-avtext-
              framemarking-15>.

   [SecureObjects]
              "Secure Objects for Media over QUIC", n.d.,
              <https://suhashere.github.io/moq-secure-objects/#go.draft-
              jennings-moq-secure-objects.html>.

   [MOQ-MLS]  "Secure Group Key Agreement with MLS over MoQ", n.d.,
              <https://suhashere.github.io/moq-e2ee-mls/draft-jennings-
              moq-e2ee-mls.html>.

   [WebCodecs]
              "WebCodecs", July 2023,
              <https://www.w3.org/TR/webcodecs/>.






Zanaty, et al.          Expires 5 September 2024                [Page 9]

Internet-Draft               media container                  March 2024


   [WEBCODECS-CODEC-REGISTRY]
              "WebCodecs Codec Registry", July 2023,
              <https://www.w3.org/TR/webcodecs-codec-registry/>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/rfc/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

   [RFC6464]  Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time
              Transport Protocol (RTP) Header Extension for Client-to-
              Mixer Audio Level Indication", RFC 6464,
              DOI 10.17487/RFC6464, December 2011,
              <https://www.rfc-editor.org/rfc/rfc6464>.

   [RFC5646]  Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying
              Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646,
              September 2009, <https://www.rfc-editor.org/rfc/rfc5646>.

Appendix A.  Acknowledgements

   Thanks to Cullen Jennings for suggestions and review.

Authors' Addresses

   Mo Zanaty
   Cisco
   Email: mzanaty@cisco.com


   Suhas Nandakumar
   Cisco
   Email: snandaku@cisco.com


   Peter Thatcher
   Microsoft
   Email: pthatcher@microsoft.com









Zanaty, et al.          Expires 5 September 2024               [Page 10]