Applying PVQ Outside Daala
draft-cho-netvc-applypvq-02

Abstract

This document describes the Perceptual Vector Quantization (PVQ) outside of the Daala video codec, where PVQ was originally developed. It discusses the issues arising while integrating PVQ into a traditional video codec, AV1.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on May 4, 2017.

Copyright Notice

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Background
• 2. Integration of PVQ into non-Daala codec, AV1
• 2.1. Signaling Skip for Partition and Transform Block
• 2.2. Issues
• 3. Performance of PVQ in AV1
• 3.1. Coding Gain
• 3.2. Speed
• 4. Future Work
• 5. Development Repository
• 6. Acknowledgements
• 7. IANA Considerations
• 8. Informative References
• Author's Address

1. Background

Perceptual Vector Quantization (PVQ)  [Perceptual-VQ][I-D.valin-netvc-pvq] has been proposed as a quantization and coefficient coding tool for an internet video codec. PVQ was originally developed for Daala video codec <https://xiph.org/daala/> [PVQ-demo], which does a gain-shape coding of transform coefficients instead of more traditional scalar quantization. (The original abbreviation of PVQ, "Pyramid Vector Quantizer", as in [I-D.valin-netvc-pvq] is now commonly expanded as "Perceptual Vector Quantization".)

The most distinguishing idea of PVQ is the way it references a predictor. With PVQ, we do not subtract the predictor from the input to produce a residual, which is then transformed and coded. Both the predictor and the input are transformed into the frequency domain. Then, PVQ applies a reflection to both the predictor and the input such that the prediction vector lies on one of the coordinate axes, and codes the angle between them. By not subtracting the predictor from the input, the gain of the predictor can be preserved and is explicitly coded, which is one of the benefits of PVQ. Since DC is not quantized by PVQ, the gain can be viewed as the amount of contrast in an image, which is an important perceptual parameter.

Also, an input block of transform coefficients is split into frequency bands based on their spatial orientation and scale. Then, each band is quantized by PVQ separately. The 'gain' of a band indicates the amount of contrast in the corresponding orientation and scale. It is simply the L2 norm of the band. The gain is non-linearly companded and then scalar quantized and coded. The remaining information in the band, the 'shape', is then defined as a point on the surface of a unit hypersphere.

Another benefit of PVQ is activity masking based on the gain, which automatically controls the quantization resolution based on the image contrast without any signaling. For example, for a smooth image area (i.e. low contrast thus low gain), the resolution of quantization will increase, thus fewer qunatization errors will be shown. Succint summary on the benefits of PVQ can be found in the Section 2.4 of [Terriberry_16].

Since PVQ has only been used in the Daala video codec, which contains many non-traditional design elements, there has not been any chance to see the relative coding performance of PVQ compared to scalar quantization in a more traditional codec design. We have tried to apply PVQ in AV1 video codec, which is currently being developed by Alliance for Open Media (AOM) as an open source and royalty-free video codec. While most of benefits of using PVQ arise from the enhancement of subjective quality of video, compression results with activity masking enabled are not available yet in this draft because the required parameters, which were set for Daala, have not been adjusted to AV1 yet. These results were achieved optimizing solely for PSNR.

2. Integration of PVQ into non-Daala codec, AV1

Adopting PVQ in AV1 requires replacing both the scalar quantization step and the coefficient coding of AV1 with those of PVQ. In terms of inputs to PVQ and the usage of trasnforms as shown in Figure 1 and Figure 2, the biggest conceptual changes required in a traditional coding system, such as AV1, are

• Introduction of a transformed predictor both in encoder and decoder. For this, we apply a forward transform to the predictors, both intra-predicted pixels and inter-predicted (i.e., motion-compensated) pixels. This is because PVQ references the predictor in the transform domain, instead of using a pixel-domain residual as in traditional scalar quantization.
• Absence of a difference signal (i.e. residue) defined as "input source - predictor". Hence AV1 with PVQ does not do any 'subtraction' in order for an input to reference the predictor. Instead, PVQ takes a different approach to referencing the predictor which happens in the transform domain.

  input X --> +-------------+                 +-------------+
              | Subtraction | --> residue --> | Transform T |
predictor --> +-------------+     signal R    +-------------+
        P                                            |
     |                                               v
     v                                              T(R)
    [+]--> decoded X                                 |
     ^                                               |
     |                                               v
     |       +-----------+    +-----------+     +-----------+
decoded  <-- | Inverse   | <--| Inverse   | <-- | Scalar    |
      R      | Transform |    | Quantizer |  |  | Quantizer |
             +-----------+    +-----------+  |  +-----------+
                                             v 
                                       +-------------+
                         bitstream  <--| Coefficient |
                         of coded T(R) |       Coder |
                                       +-------------+

            +-------------+            +-----------+ 
  input X-->| Transform T |--> T(X)--> | PVQ       | 
            |_____________|            | Quantizer |  +-------------+
                                +----> +-----------+  | PVQ         |
            +-------------+     |            |------> | Coefficient |
predictor-->| Transform T |--> T(P)          v        | Coder       |
        P   |_____________|     |      +-----------+  +-------------+
                                |      | PVQ       |        |
                                +----> | Inverse   |        v
                                       | Quantizer |    bitstream
                                       +-----------+    of coded T(X)
                                              |
              +-----------+                   v
 decoded X <--| Inverse   | <--------- dequantized T(X)
              | Transform |
              +-----------+