Codec-Layer Feedback for the Real-time Transport Control Protocol (RTCP)
draft-realvnc-rtcp-codec-00

Abstract

This document specifies an extension to the messages defined in the Audio-Visual Profile with Feedback (AVPF), allowing for an RTP codec to send codec-specific feedback via RTCP. Currently, RTCP feedback messages have been defined only for generic capabilities which may be defined in the context of individual codecs, and some codecs have their own RTCP feedback messages. There is a need for a feedback message at the codec-layer, allowing other codecs to send codec-specific feedback.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on March 28, 2019.

Copyright Notice

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Introduction
• 2. Definitions
• 2.1. Glossary
• 2.2. Requirements Language
• 3. Use Cases
• 4. Protocol Overview
• 5. Format of RTCP Feedback Message
• 6. Semantics of the RTCP Feedback Message
• 7. SDP Definition
• 7.1. Extension of the rtcp-fb attribute
• 7.2. Offer/Answer
• 8. IANA Considerations
• 9. Security Considerations
• 10. References
• 10.1. Normative References
• 10.2. Informative References
• Author's Address

1. Introduction

The Extended RTP Profile for RTCP-Based Feedback (RTP/AVPF, [RFC4585]) introduced the concept of feedback messages, which may be used by RTP peers to exchange information controlling an RTP stream. The framework provided by RTP/AVPF divides such feedback into Transport-Layer (RTPFB) and Payload-Specific feedback (PSFB) messages. These include generic capabilities such as "Slice Loss Indication" (SLI), which are widely applicable to many different codecs, yet require interpretation in the context of each codec, since the notion of "slice" is codec-specific. Since then, further Codec-Control messages have been defined in the same framework, which further expand the generic messages defined for the RTP/AVPF control layer, for example "Full Intra Request" from [RFC5104], which follows the same model as SLI.

However, these generic messages do not enable truly codec-specific feedback to be exchanged. There is currently no payload-specific feedback number assigned for codec-defined feedback which is not covered by the existing mechanisms (SLI, FIR, and others). Some codecs have resorted to defining their own top-level feedback messages (such as the H.271 Video Back Channel Message, [RFC5104]), yet the space of assigned numbers for PSFB messages is scarce, and it is not reasonable to allocate further numbers for messages that are strongly tied to specific codecs.

Some applications may make use of the Application-Layer Feedback capability from [RFC4585], however this is unsatisfactory for widespread use, since a codec may be used by more than one application, at which point collisions may occur in the use of the application-layer feedback.

This document therefore requests that a PSFB number be reserved for codec-layer feedback. Such feedback is specific to an individual codec, as identified by the payload type identifier carried in the feedback message. The contents and semantics of such a feedback message are entirely defined by the codec, in the same way that codecs have free control over the contents of the RTP payload.

2. Definitions

2.1. Glossary

AVPF -

The extended RTP profile for RTCP-based feedback

CCM -

Codec Control Message [RFC5104]

CLF -

Codec-Layer Feedback

FCI -

Feedback Control Information [RFC4585]

FIR -

Full Intra Request

PSFB -

Payload-Specific Feedback [RFC4585]

SLI -

Slice Loss Indication

2.2. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

3. Use Cases

We briefly outline here some use-cases for codec-specific feedback messages. In each case, there is no clear existing RTCP message which may be used.

Codec-specific quality control.

Some codecs may wish to allow control of the quality level via feedback from the receiver. Such quality control would not fit well into a generic message, since the codec's control parameters (quantization tables, dithering level, text quality threshold) may differ widely from the control parameters for other codecs. The ability for the codec to define its own PSFB messages, without requiring a codec-specific allocation of a new PSFB message, would be helpful.

Codec-specific picture control.

Some codecs may wish to allow the receiver to specify to the sender a preferred color palette to use for the transmitted picture, perhaps to ensure the image is usable in a constrained viewing environment. Different codecs however encode color data in different ways (sRGB, YUV, YDbDr, and many more). In the case of a codec designed to support palettized encodings for constrained viewing environments, it would be helpful for the codec to be able to define its own feedback messages to provide such a capability.

Codec-specific decoder control.

There are codecs in which the decoder requires access to certain control data in order to begin decoding, for example the RTP embedding of H.264 [RFC6184] has special considerations in the transmission of Picture Parameter Set (PPS) NALUs, with the recommendation that these units be exchanged reliably over an out-of-band channel (such as SDP exchange). However, for other codecs, the decoder control data may be less easily exchanged over such a channel: either because it is larger in size, or changes more frequently, or may be re-ordered with respect to the RTP stream (for codecs where the decoder does not need to block decoding until the very latest control information is received). In these cases, it may be preferable to exchange the decoder-control data over RTCP, so that it does not participate in the generic RTP control such as retransmissions. For a codec with specific decoder-control data, it may be helpful to enable the codec to make use of RTCP as well as RTP messages.

4. Protocol Overview

This document extends the RTCP feedback messages defined in the RTP/ AVPF [RFC4585] by defining an RTCP Codec-Layer Feedback (CLF) message. The RTCP CLF message contains a header identifying the codec by means of the payload type identifier, and the remaining contents and semantics of the message are defined by the codec.

A CLF RTCP message may be sent from the sender to the receiver, or from the receiver to the sender, depending on the codec.

5. Format of RTCP Feedback Message

As specified by section 6.1 of [RFC4585], Payload-Specific FB messages are identified by the RTCP packet type value PSFB (206).

This memo defines the CLF message, which is identified by RTCP packet type values PT=PSFB and FMT=XXX.

The Feedback Control Information (FCI) for the CLF is of variable-length, as determined by the codec which defines the CLF message. At least four octets of FCI must be provided. The length of the CLF feedback message MUST be set to 2+N/4, where N is the length in octets of the CLF message.

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|P| Payload Type|    CLF data, defined by the codec...          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         ...   | Padding (opt) | Padding count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     

Syntax of the Codec-Layer Feedback (CLF) message

P:

1 bit
Indicates whether the CLF message is padded. All CLF messages must be a multiple of 4 octets long; if the P bit is set to 1, then the final octet contains a padding count of between 1 and 3, indicating the number of final padding octets in the message (including the padding count).

Payload Type:

7 bits
Indicates the RTP payload type in the context of which the native CLF data MUST be interpreted.

6. Semantics of the RTCP Feedback Message

Within the common packet header for feedback messages (as defined in Section 6.1 of [RFC4585]), the "SSRC of packet sender" field indicates the source of the feedback message, and the "SSRC of media source" indicates the SSRC of the sender end of the stream.

A CLF message MUST NOT be used for codecs that do not define the contents and semantics of the CLF message.

A CLF message MUST NOT be sent in a direction which is not defined by the codec: a CLF message MUST NOT be sent by the sender to the receiver unless the codec defines the CLF message in that direction, and a CLF message MUST NOT be sent by the receiver to the sender unless the codec defines the CLF message in that direction.

CLF messages are sent according to timing rules defined by the codec. A mixer or translator MAY choose to forward the message, but SHOULD ensure that such forwarded messages are not sent at a rate greater than the receiver's bandwidth (for example, a mixer which receives feedback messages from a unicast peer, and forwards those messages to a multicast session may be forced to drop packets); such a mixer SHOULD understand the semantics of the CLF message for the payload being forwarded, to ensure that the dropped packets do not prevent correct operation of the codec. A mixer SHOULD NOT drop CLF packets which it does not understand. A mixer MAY choose not to forward CLF messages at all, by declaring that it does not support CLF messages (for example, by using an SDP declaration to state that it does not accept CLF messages for the given codec).

Since mixers may not support CLF messages for any particular codec, and may not support CLF messages at all, codecs SHOULD NOT mandate that CLF support is required for the correct operation of the codec.

7. SDP Definition

Section 4 of [RFC4585] defines a new SDP [RFC4566] attribute, rtcp-fb, that may be used to negotiate the capability to handle specific AVPF commands and indications. The ABNF for rtcp-fb is described in section 4.2 of [RFC4585]. In this section, we extend the rtcp-fb attribute to include the CLF message.

7.1. Extension of the rtcp-fb attribute

AVPF specifies that the rtcp-fb attribute must only be used as a media level attribute and must not be provided at session level. All the rules described in [RFC4585] for rtcp-fb attribute relating to payload type and to multiple rtcp-fb attributes in a session description also apply to the new feedback message defined in this memo.

In [RFC5104], a feedback value "ccm" is defined, which indicates the support of codec control using RTCP feedback messages. The "ccm" feedback value is used with a new parameter defined in this memo, "clf", to indicate support for Codec-Layer Feedback (CLF).

In the ABNF for rtcp-fb-ccm-param defined in [RFC5104], there is a placeholder token to allow defining new CCM message parameters. "clf" is defined as a new CCM parameter in this document.

Inclusion of the CLF message type in an SDP description indicates support for sending and receiving the CLF message, depending on whether the codec defines the messages in each direction.

The "clf" parameter SHOULD NOT be used with rtcp-fb-pt set to "*", to indicate support for the CLF message for all codecs; instead support for CLF messages SHOULD be declared for each supported codec individually.

For example, to indicate support for receiving CLF messages, the sender may declare the following in SDP:

v=0
o=alice 12345 12345 IN IP4 host.example.com
s=Media with CLF feedback
t=0 0
c=IN IP4 host.example.com
m=video 23456 RTP/AVPF 98
a=rtpmap:98 urn:codec-name/90000
a=rtcp-fb:98 ccm clf

7.2. Offer/Answer

The Offer/Answer [RFC3264] implications for codec layer feedback feedback messages are similar to those described in [RFC5104]. The offerer MAY indicate the capability to support selected CLF messages. The answerer MUST remove all CLF parameters corresponding to the CLF message and codec combinations that it does not wish to support in this particular media session (for example, because it does not implement the CLF message for the codec in question). The answerer MUST NOT add new clf parameters in addition to what has been offered.

As in [RFC5104], only if CLF is jointly declared for a codec by both offer and the answer, may CLF messages be used.

Including a CLF parameter in an offer or answer indicates that the party (offerer or answerer) is at least capable of receiving the corresponding CLF message and acting upon it. It is not mandated to send CLF messages of any negotiated type, unless the acknowledgement or response is a mandatory part of receiving the CLF message.

8. IANA Considerations

A new value "clf" must be registered with IANA in the "Codec Control Messages" registry, located at time of publication at: http://www.iana.org/assignments/sdp-parameters

The new value is as follows:

• Value name: clf
• Long name: Codec-Layer Feedback
• Usable with: ccm
• Reference: RFC-realvnc-codec-layer-feedback

The following value must be registered as a FMT value in the "FMT Values for PSFB Payload Types" registry located at the time of publication at: http://www.iana.org/assignments/rtp-parameters

PSFB range

9. Security Considerations

The CLF message is defined within the context of the existing security model for RTP/RTCP sessions. As such, it does not enable significant changes to the existing security model.

Since the behaviour of CLF messages is undefined by this specification, detailed considerations must be provided by each codec specification which defines such a message.

To guard against malicious injection of CLF data, the CLF messages should be treated as untrusted external data, in the same way that an RTP payload itself would be. Authentication and integrity checks may be used where appropriate to ensure that third-parties may not inject or modify CLF messages, for example using SRTP and the SAVFP profile to provide security.

The CLF message format itself is not believed to carry security risks additional to those already incurred by RTP/RTCP peers.

10. References

10.1. Normative References

10.2. Informative References

Author's Address