Internet DRAFT - draft-weaver-pef

draft-weaver-pef







Network Working Group                                          J. Weaver
Internet-Draft                                                   BBC R&D
Intended status: Standards Track                          April 27, 2021
Expires: October 29, 2021


            Packetised Essence Format for Uncompressed Video
                          draft-weaver-pef-00

Abstract

   This memo specifies a new proposed packing format for Uncompressed
   video data at a variety of bit-depths and with a variety of different
   component structures.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on October 29, 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.





Weaver                  Expires October 29, 2021                [Page 1]

Internet-Draft                 PEF-Format                     April 2021


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Video Data Formats, an Overview . . . . . . . . . . . . . . .   2
   3.  The Proposed Format in Detail . . . . . . . . . . . . . . . .   6
   4.  Security Considerations . . . . . . . . . . . . . . . . . . .  10
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
     5.1.  Proposed MIME Type Registration . . . . . . . . . . . . .  10
   6.  Unpacking of Video Data in Software and Hardware  . . . . . .  13
   7.  Normative References  . . . . . . . . . . . . . . . . . . . .  15
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  15

1.  Introduction

   This memo specifies a format for packing uncompressed video of a
   variety of formats into an easily processed format for software.  It
   is compatible with video of a variety of bit-depths, colour
   subsampling formats, picture resolutions, frame rates, scan-modes,
   and number of components.

1.1.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

2.  Video Data Formats, an Overview

   For purposes of this recommendation a video format is defined by a
   number of parameters:

   o  The components of the video.  See the fuller discussion below for
      what this means in general.

   o  The number of samples across a line of video, which may vary for
      different components (for example for video with a colour
      subsampling other than 4:4:4).

   o  The number of lines in a frame of video, which may vary for
      different components.

   o  The bit-depth of the samples.  Currently supported bit-depths are
      1, 2, 4, 8, 9, 10, 12, 14, and 16.  This can also vary for
      different components.  For example an Alpha channel or a depth map
      may use a lower bit depth than the main luma channel

   o  The frame-rate of the video.



Weaver                  Expires October 29, 2021                [Page 2]

Internet-Draft                 PEF-Format                     April 2021


   For the purposes of this format we support an arbitrary number of
   video components.  A video component is a plane of samples for the
   video which is in some way independent of other samples outside of
   that plane.  For progressively scanned monochrome video there is only
   one component, which is the video data's brightness (luma) values.
   For monochrome video with an interlaced scan mode, on the other hand,
   there are two components per frame: the luma of the first field and
   the luma of the second field.  For colour video there are generally
   three components per field, either Luma, and two colour-difference
   signals (Y,Cb,Cr) or three colour channels (R,G,B).  In this case for
   progressively scanned video there will be three components, eg.  Y,
   Cb, and Cr, and for video which is interlaced there would be six, for
   example Field 1 Y, Field 1 Cb, Field 1 Cr, Field 2 Y, Field 2 Cb, and
   Field 2 Cr.  Other video formats may have different numbers or
   arangements of components: for example a stereoscopic, interlaced,
   video stream making use of three colour channels and an alpha channel
   might have sixteen components: Left Field 1 Y, Left Field 1 Cb, Left
   Field 1 Cr, Left Field 1 A, Right Field 1 Y, Right Field 1 Cb, Right
   Field 1 Cr, Right Field 1 A, Left Field 2 Y, Left Field 2 Cb, Left
   Field 2 Cr, Left Field 2 A, Right Field 2 Y, Right Field 2 Cb, Right
   Field 2 Cr, and Right Field 2 A.  Future video formats may contain
   additional components which have yet to considered, but should still
   be compatible with this format.

   There are a plethora of existing video formats which arrange the
   video in a variety of ways.  Some formats are "interleaved" -- ie.
   they arrange samples from different components one after another in
   order.  Other formats are "planar" -- they arrange the data so that
   all the samples for one component for a frame or field are placed
   together and followed by all the samples for another component for
   the same frame or field.  In addition the arrangement of the data
   samples themselves within memory is also variable: some formats are
   called "packed" -- meaning that samples with a bit-depth which is not
   a power of two are placed within memory in a bit-contiguous fashion
   regardless of octet boundaries; others are "padded", meaning that
   samples are placed in the low-order bits of a larger octet-aligned
   container; Finally there are formats which both pad and pack the
   samples, the most well-known being V210, a format which places three
   10-bit samples into the least significant bits of a 32-bit word, but
   ignores the value of the top two bits.

   The advantage of "packed" formats is that they make use of all the
   bits available for storing video data, thus making them more space
   efficient in storage or more bandwidth efficient when used on a
   network.  The disadvantage is that they are complicated to deal with
   in software, since computer architectures are generally optimised to
   process multiples of 8, 16, 32, or 64 bits.




Weaver                  Expires October 29, 2021                [Page 3]

Internet-Draft                 PEF-Format                     April 2021


   The format recommended here uses a different approach.  Rather than
   storing data samples interleaved or in a planar structure this format
   arranges them into "blocks" of samples, which are then stored in a
   planar fashion.  The size of these "blocks" of samples can be
   selected so as to correspond to the block sizes used in various video
   compression and video processing techniques, allowing a more natural
   processing when using these techniques, or can be adjusted to provide
   a lower latency as required by the specific application in use.

   Rather than traditional packing or padding this format proposes the
   storage of the bits in the samples in a non-contiguous structure,
   allowing for 100% utilisation of the available bits, like a packed
   format, but with all data aligned conveniently on power-of-two
   boundaries as found in a padded format.





































Weaver                  Expires October 29, 2021                [Page 4]

Internet-Draft                 PEF-Format                     April 2021


   Packed 10-bit:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |     Sample 3      |     Sample 2      |     Sample 1      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    7      |     Sample 6      |     Sample 5      |     Sample 4
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   mple 10     |     Sample 9      |     Sample 8      |     Sample
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     Sample 13     |     Sample 12     |     Sample 11     |     Sam
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Sample 16     |     Sample 15     |     Sample 14     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


   Padded 10-bit in 16-bit:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X|     Sample 1      |X X X X X X|     Sample 2      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X|     Sample 3      |X X X X X X|     Sample 4      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X|     Sample 5      |X X X X X X|     Sample 6      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


   V210 Format (a hybrid of packing and padding):

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X|     Sample 3      |     Sample 2      |     Sample 1      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X|     Sample 6      |     Sample 5      |     Sample 4      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X|     Sample 9      |     Sample 8      |     Sample 7      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 1: Examples of packing formats








Weaver                  Expires October 29, 2021                [Page 5]

Internet-Draft                 PEF-Format                     April 2021


3.  The Proposed Format in Detail

   When making use of this format a number of parameters must be set.
   These will be introduced in the text below as they are explained.
   For the purposes of this format the components of the video are
   treated separately.  Some parameters must be the same for all
   components, but others may vary, and this will be noted below.
   Fundamentally this format involves breaking the video samples in a
   component for a specific frame into "blocks", which is to say two
   dimensional groupings of samples which form a rectangular shape in
   the active part of the frame.

   Bit Depth (D):  This is the number of bits used in each sample.  In
      principle this may vary by component, but in general it will
      usually be the same for all components.  Currently supported bit-
      depths are: 1, 2, 4, 8, 9, 10, 12, 14, and 16.

   Block Height (BH) and Block Width (BW):  These are integer values
      which give a number of samples of height and width for each block.
      There is no requirement for all components to use the same values
      for these, but there are restrictions on the valid choices of BH
      and BW.

      0                   1
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
   00 X X X X X X X X Y Y Y Y Y Y Y Y
    1 X X X X X X X X Y Y Y Y Y Y Y Y
    2 X X X X X X X X Y Y Y Y Y Y Y Y
    3 X X X X X X X X Y Y Y Y Y Y Y Y
    4 X X X X X X X X Y Y Y Y Y Y Y Y
    5 X X X X X X X X Y Y Y Y Y Y Y Y
    6 X X X X X X X X Y Y Y Y Y Y Y Y
    7 X X X X X X X X Y Y Y Y Y Y Y Y
    8 Z Z Z Z Z Z Z Z
    9 Z Z Z Z Z Z Z Z
   10 Z Z Z Z Z Z Z Z
    1 Z Z Z Z Z Z Z Z
    2 Z Z Z Z Z Z Z Z
    3 Z Z Z Z Z Z Z Z
    4 Z Z Z Z Z Z Z Z
    5 Z Z Z Z Z Z Z Z
    6
    7
    8
    9

    Figure 2: Example of Blocks (X marks one 8x8 block, Y another, Z a
                                  third)



Weaver                  Expires October 29, 2021                [Page 6]

Internet-Draft                 PEF-Format                     April 2021


                +----+-----------------------------------+
                | D  | Restriction on BH and BW          |
                +----+-----------------------------------+
                | 1  | BW x BH MUST be a multiple of 8   |
                |    |                                   |
                | 2  | BW x BH MUST be a multiple of 4   |
                |    |                                   |
                | 4  | BW x BH MUST be a multiple of 2   |
                |    |                                   |
                | 8  | No restriction                    |
                |    |                                   |
                | 9  | BW x BH MUST be a multiple of 128 |
                |    |                                   |
                | 10 | BW x BH MUST be a multiple of 64  |
                |    |                                   |
                | 12 | BW x BH MUST be a multiple of 32  |
                |    |                                   |
                | 14 | BW x BH MUST be a multiple of 64  |
                |    |                                   |
                | 16 | No restriction                    |
                +----+-----------------------------------+

   Each frame of video MUST be represented by one stream of octets per
   component.  The ordering of the components is a parameter of the
   video format itself.  So a frame of YCbCr video will consist of a
   stream of luma, followed by two streams of chroma differences, whilst
   a frame of RGBA video will consist of a stream of Red, one of Green,
   one of Blue, and one of Alpha.

   Each stream is assembled starting by taking the blocks for that
   component in that frame in raster-scan order.  If the width of a
   block is not a divisor of the number of samples across an active line
   of video for that component, or if the height is not a divisor of the
   height of the active video for that component then the data MUST be
   padded with additional samples to fill the block size.  The choice of
   values for padding is not specified, and MAY be any value desired.
   Mirroring, edge-extending, padding with black, and padding with mid-
   grey are all possible options.

   Each block of samples is converted into a stream of octets as
   follows:

   o  If D=1 simply pack the values of the samples in raster scan order
      as 1-bit values into octets.  Each block will thus be represented
      by BW x BH / 8 octets.






Weaver                  Expires October 29, 2021                [Page 7]

Internet-Draft                 PEF-Format                     April 2021


   o  If D=2 simply pack the values of the samples in raster scan order
      as 2-bit values into octets.  Each block will thus be represented
      by BW x BH / 4 octets.

   o  If D=4 simply pack the values of the samples in raster scan order
      as 4-bit values into octets.  Each block will thus be represented
      by BW x BH / 2 octets.

   o  If D=8 simply use the values of the samples in raster-scan order
      with each sample value occupying one octet.  Each block will thus
      be represented by BH x BW octets.

   o  If D=9 pack the least significant bit of each sample in raster
      scan order into the first BH x BW x 0.125 octets, then use the
      most significant 8 bits of each sample in the same order as the
      values of the next BH x BW octets.  Each block will thus be
      represented by BH x BW x 1.125 octets.  This arrangement, together
      with the restrictions on the choice of BH and BW ensures that the
      number of octets used to store the least significant bits will
      always be a multiple of 16.

   o  If D=10 pack the least significant 2 bits of each sample in raster
      scan order into the first BH x BW x 0.25 octets, then use the most
      significant 8 bits of each sample in the same order as the values
      of the next BH x BW octets.  Each block will thus be represented
      by BH x BW x 1.25 octets.  This arrangement, together with the
      restrictions on the choice of BH and BW ensures that the number of
      octets used to store the least significant bits will always be a
      multiple of 16.

   o  If D=12 pack the least significant 4 bits of each sample in raster
      scan order into the first BH x BW x 0.5 octets, then use the most
      significant 8 bits of each sample in the same order as the values
      of the next BH x BW octets.  Each block will thus be represented
      by BH x BW x 1.5 octets.  This arrangement, together with the
      restrictions on the choice of BH and BW ensures that the number of
      octets used to store the least significant bits will always be a
      multiple of 16.

   o  If D=14 pack the least significant 2 bits of each sample in raster
      scan order into the first BH x BW x 0.25 octets, then pack the
      next least significant 4 bits of each sample in the same order
      into the next BH x BW x 0.5 octets, then use the most significant
      8 bits of each sample in the same order as the values of the next
      BH x BW octets.  Each block will thus be represented by BH x BW x
      1.75 octets.  This arrangement, together with the restrictions on
      the choice of BH and BW ensures that the number of octets used to
      store the least significant bits will always be a multiple of 16,



Weaver                  Expires October 29, 2021                [Page 8]

Internet-Draft                 PEF-Format                     April 2021


      as will the number of octets used to store the next least
      significant bits of each sample.

   o  If D=16 use the values of the samples in raster-scan order with
      each sample value occupying two octets in network octet order.
      Each block will thus be represented by BH x BW x 2 octets.

   As an example of how the data is structured refer to Figure 3, which
   provides an example of a 64-sample block (such as an 8x8 block).  The
   initial section consist of 16 octets containing packed 2-bit values
   each containing bits 8 and 9 of each sample, whilst the remaining 64
   octets contain 8-bit values containing bits 0 to 7 of each sample.

   The more complex 14-bit packing is shown in Figure 4.  In this case
   the first 16 octets contain packed 2-bit values containing bits 12
   and 13 of each sample, the next 32 octets contain packed 4-bit values
   containing bits 8-11 of each sample, and the remaining 64- octets
   contain 8-bit values consisting of bits 0 to 7 of each sample.

   Following these rules each component of a frame of 1920 x 1080 10-
   bit progressive video with a block shape of 8x8 for all components
   will contain 32400 blocks, each represented by 80 octets broken into
   a 16-octet first part containing the least significant 2 bits of each
   sample and a 64-octet second part.  In this fashion this packing
   format is bit-rate efficient, making full use of all available bits.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   L+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   S| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |10 |11 |12 |13 |14 |15 |16 |
   B+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   s|17 |18 |19 |20 |21 |22 |23 |24 |25 |26 |27 |28 |29 |30 |31 |32 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |33 |34 |35 |36 |37 |38 |39 |40 |41 |42 |43 |44 |45 |46 |47 |48 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |49 |50 |51 |52 |53 |54 |55 |56 |57 |58 |59 |60 |61 |62 |63 |64 |
   M+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   S|  Sample 1     |  Sample 2     |  Sample 3     |  Sample 4     |
   B+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   s|  Sample 5     |  Sample 6     |  Sample 7     |  Sample 8     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   ...
    |  Sample 61    |  Sample 62    |  Sample 63    |  Sample 64    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 3: Arrangement of Data for an 8x8 block when D=10





Weaver                  Expires October 29, 2021                [Page 9]

Internet-Draft                 PEF-Format                     April 2021


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   L+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   S| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |10 |11 |12 |13 |14 |15 |16 |
   B+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   s|17 |18 |19 |20 |21 |22 |23 |24 |25 |26 |27 |28 |29 |30 |31 |32 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |33 |34 |35 |36 |37 |38 |39 |40 |41 |42 |43 |44 |45 |46 |47 |48 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |49 |50 |51 |52 |53 |54 |55 |56 |57 |58 |59 |60 |61 |62 |63 |64 |
   M+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   i|   1   |   2   |   3   |   4   |   5   |   6   |   7   |   8   |
   d+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   d|   9   |  10   |  11   |  12   |  13   |  14   |  15   |  16   |
   l+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   e                               ...
    |  57   |  58   |  59   |  60   |  61   |  62   |  63   |  64   |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   M|  Sample 1     |  Sample 2     |  Sample 3     |  Sample 4     |
   S+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   B|  Sample 5     |  Sample 6     |  Sample 7     |  Sample 8     |
   s+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   ...
    |  Sample 61    |  Sample 62    |  Sample 63    |  Sample 64    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 4: Arrangement of Data for an 8x8 block when D=14

4.  Security Considerations

   A blocking format for uncompressed video does not in of itself have
   any particular security implications, but also does not provide any
   additional security.  If the content is sensitive then some other
   mechanism needs to be used to keep it secure.

5.  IANA Considerations

   This format would require a mime type and associated RTP payload type
   to be registered.

5.1.  Proposed MIME Type Registration

   Type name:  video

   Subtype name:  pef

   Required parameters:




Weaver                  Expires October 29, 2021               [Page 10]

Internet-Draft                 PEF-Format                     April 2021


      sampling:  Determines the color (sub-)sampling mode of the video
         stream.  Currently defined values are:

         Monochrome:  (in which case for progressive scan the only
            component is component 0),

         RGB:  (in which case for progressive scan component 0 is R,
            component 1 is G, and component 2 is B),

         RGBA:  (in which case for progressive scan component 0 is R,
            component 1 is G, component 2 is B, and component 3 is A),

         BGR:  (in which case for progressive scan component 0 is
            B,component 1 is G, and component 2 is R),

         BGRA:  (in which case for progressive scan component 0 is B,
            component 1 is G, component 2 is R, and component 3 is A),

         YCbCr:  (in which case for progressive scan component 0 is Y,
            component 1 is Cb, and component 2 is Cr),

         New values may be registered as needed.

      width:  A comma separated list of values which determines the
         number of samples per line for each component.  Each value is
         an integer.

      height:  A comma separated list of values which determines the
         number of lines per frame for each component.  Each value is an
         integer.

      depth:  A comma separated list of values giving the number of bits
         per sample.  Each value is an integer with currently
         permissible values of 1, 2, 4, 8, 9, 10, 12, 14, and 16.

      colorimetry:  This parameter defines the set of colorimetric
         specifications and other transfer characteristics for the video
         source, by reference to an external specification.  Valid
         values and their specification are:

         BT601-5:  ITU Recommendation BT.601-5

         BT709-2:  ITU Recommendation BT.709-2

         SMPTE240M:  SMPTE standard 240M

      block-width:  A comma separated list of integers, each is the
         width of each block in samples for a specific component.



Weaver                  Expires October 29, 2021               [Page 11]

Internet-Draft                 PEF-Format                     April 2021


      block-height:  A comma separated list of integers, each is the
         width of each block in samples for a specific component.

   Optional parameters:

      interlace:  If this OPTIONAL parameter is present, it indicates
         that the video stream is presented as interlaced.  If absent,
         progressive scan is implied.  If present then the number of
         components in the stream is double that found in a
         progressively scanned stream with the same sampling parameter.

      stereo:  If this OPTIONAL parameter is present, it indicates that
         the video stream is presented as a stereo pair.  If present
         then the number of components in the stream is double that
         found in a stream with the same sampling parameter.

      chroma-position:  This OPTIONAL parameter defines the position of
         chrominance samples relative to luminance samples.  It is
         either a single integer or a comma separated pair of integers.
         Integer values range from 0 to 8, as specified in Figures 6-8
         of [RFC4175].  A single integer implies that Cb and Cr are co-
         sited.  A comma separated pair of integers designates the
         locations of Cb samples, respectively.  In its absence, a
         single value of assumed for color-subsampled video (chroma-
         position=0).

      gamma:  An OPTIONAL floating point gamma correction value.

   Encoding considerations:  framed

   Security considerations:  This format contains no active content, and
      has no particular security considerations.

   Interoperability considerations:  This format is an alternative form
      of raw uncompressed video distinct from and incompatible with
      video/raw.

   Published specification:  This Document

   Applications that use this media type:  Video communication.

   Fragment identifier considerations:  N/A

   Additional information:

      File Name Extensions:  .pef





Weaver                  Expires October 29, 2021               [Page 12]

Internet-Draft                 PEF-Format                     April 2021


   Person & email address to contact for further information:  James P.
      Weaver < james.barrett@bbc.co.uk >

   Intended usage:  COMMON

   Restrictions on usage:  N/A

   Author:  James P.  Weaver

   Change controller:  James P.  Weaver

   Provisional registration? (standards tree only):  YES

6.  Unpacking of Video Data in Software and Hardware

   The format here described has been designed with an eye to ease of
   unpacking in software and hardware.  In the case of 8-bit video this
   format is very similar to the conventional 8-bit formats and so
   requires no additional processing.  A traditional 10-bit format,
   however, either packs 10-bit samples in a way which overlaps octet
   boundaries or pads, and thereby doesn't make use of all bits.  This
   format makes full use of all bits like a packed format, but maintains
   octet alignment.

   For software processing purposes it is common to unpack 10-bit data
   so that each sample is stored in the lower 10-bits of a 16-bit word,
   essentially converting from a packed format to a padded one.  To do
   this for the traditional packed 10-bit format is a complex operation
   involving a significant amount of masking and shifting and is not
   easily amendable to processing using the efficient 128-bit Single
   Instruction Multiple Data instructions implemented on many modern
   processors.

   By comparison this new format is easily convertible to a padded
   format using SIMD instructions, with a general algorithm as described
   in the following.  This algorithm has been kept general and does not
   refer to specific SIMD instructions since the likelihood is that
   instruction sets will change in the future.

   The exact mechanism used for the decode will depend upon the size of
   the registers on which the SIMD instructions being used operate.  For
   these examples I will explain with 4 octet registers, simply because
   it makes the diagrams more easily viewed.

   Starting with an 80-octet stream containing the data representing a
   64 sample block of 10-bit video data we begin by loading the first
   four octets of the block into a register:




Weaver                  Expires October 29, 2021               [Page 13]

Internet-Draft                 PEF-Format                     April 2021


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |10 |11 |12 |13 |14 |15 |16 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   (The numbers in the centre of each two-bit space indicate the sample
   number from which that data originates)

   Then using a masked move or shuffle we are able to "spread" this data
   out into the octets of two registers:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X X X| 1 | 2 | 3 | 4 |X X X X X X X X| 1 | 2 | 3 | 4 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X X X| 1 | 2 | 3 | 4 |X X X X X X X X| 1 | 2 | 3 | 4 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Then using a SIMD multiplication instruction which treats each 32-bit
   register as two 16-bit values we multiply each block up in order to
   align the required data at the top of the low order octet of each
   16-bit sample:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X X X| 1 | 2 | 3 | 4 |X X X X X X| 1 | 2 | 3 | 4 |X X|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X| 1 | 2 | 3 | 4 |X X X X X X| 1 | 2 | 3 | 4 |X X X X X X|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Then the data from the first four octets of the second segment of the
   data stream (where the MSBs of the samples are stored) are loaded
   into a register

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sample 1 MSBs | Sample 2 MSBs | Sample 3 MSBs | Sample 4 MSBs |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   and a shuffle and move or blanking and bitwise or is used to
   distribute these octets into the high order octets of each 16-bit
   word in the working registers.





Weaver                  Expires October 29, 2021               [Page 14]

Internet-Draft                 PEF-Format                     April 2021


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sample 1 MSBs | 1 | 2 | 3 | 4 | Sample 2 MSBs | 2 | 3 | 4 |X X|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sample 3 MSBs | 3 | 4 |X X X X| Sample 4 MSBs | 4 |X X X X X X|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Finally a SIMD shift is used to shift each 16-bit value down by 6
   bits:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X| Sample 1 MSBs | 1 |X X X X X X| Sample 2 MSBs | 2 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |X X X X X X| Sample 3 MSBs | 3 |X X X X X X| Sample 4 MSBs | 4 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Completing the conversion of the first four samples of the block from
   this format to a padded 10-bit in 16-bit format more convenient for
   processing in software.  The conversion can then be continued with
   samples 5,6,7, and 8.

   A larger register length for the SIMD operations enables more samples
   to be decoded simultaneously.  This algorithm has been used
   effectively with 128-bit (16 octet) registers.

7.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4175]  Gharai, L. and C. Perkins, "RTP Payload Format for
              Uncompressed Video", RFC 4175, DOI 10.17487/RFC4175,
              September 2005, <https://www.rfc-editor.org/info/rfc4175>.

Author's Address











Weaver                  Expires October 29, 2021               [Page 15]

Internet-Draft                 PEF-Format                     April 2021


   James P. Weaver
   BBC R&D
   Floor 5
   Dock House
   MediaCity UK
   Salford  M50 2LH
   United Kingdom

   Email: james.barrett@bbc.co.uk










































Weaver                  Expires October 29, 2021               [Page 16]