Grouping of Adjacent Media in the Session Description Protocol
draft-jennings-mmusic-adjacent-grouping-01

Abstract

Applications, such as multi-screen video conferencing systems or advertisement boards, often have multiple audio and video streams that are organized to be rendered side by side or in a grid. This specification uses the RFC 5888 grouping framework to define new semantics for grouping the media streams to be rendered side by side or in a grid and indicating their relative ordering.

Status of this Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

This Internet-Draft will expire on April 2, 2011.

Copyright Notice

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1.  Introduction
2.  Terminology
3.  Adjacent Media Grouping
    3.1.  Requirements
    3.2.  "ADJ" Grouping Semantics
    3.3.  Grouping for SSRC-Multiplexed RTP Streams
    3.4.  SDP Offer/Answer Model Considerations
4.  SDP Examples
    4.1.  Horizontal Layout
    4.2.  Grid Layout
5.  Security Considerations
6.  IANA Considerations
    6.1.  Registration of SDP Attributes
    6.2.  Registration of Grouping Semantics
7.  Acknowledgments
8.  Open Issues
9.  References
    9.1.  Normative References
    9.2.  Informative References
§  Authors' Addresses

1.  Introduction

There are many situations where applications create media streams that are meant do be rendered adjacent to each other. A common example is a multi-screen video conferencing system. Other examples are several video monitors placed side by side to display signs, and audio streams from a linear array of microphones. The Session Description Protocol (SDP) [RFC4566] (Handley, M., Jacobson, V., and C. Perkins, “SDP: Session Description Protocol,” July 2006.) allows negotiation of multiple media streams but does not have a way to carry the ordering information to indicate which media stream is adjacent to which one.

This specification introduces new grouping semantics, using the SDP grouping framework defined in [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.), that indicate media streams are adjacent, and the adjacency order is defined by the order of the entries in the group.

2.  Terminology

This specification uses all the terms defined in [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.) and will not make sense unless you have read [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.). The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this specification are to be interpreted as described in [RFC2119] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).

3.  Adjacent Media Grouping

3.1.  Requirements

Editor's note: If there are any requirements, we should list them here.

3.2.  "ADJ" Grouping Semantics

This specification defines new grouping semantics of "ADJ" that indicate the media streams in this group are meant to be played or displayed adjacently. Furthermore, the order of media streams in the group indicates the adjacency order.

N media streams could be in a linear horizontal layout, in which case we use a grid size of 1 x N. Alternatively, N media streams could be in a linear vertical layout, in which case we use a grid size of N x 1. In these configurations, the first stream in the group MUST be the one corresponding to the left most and top most output unit, respectively. In a more general grid size of N x M, we can group K (where K <= N x M) media streams starting from the one corresponding to the top-left output unit, and then doing a continuous horizontal scanning of the grid row by row (i.e., scanning first the top row from left to right, and then the second row from left to right, and so on).

To indicate the dimensions of the layout grid in SDP, we define a new session-level attribute. The syntax for the new attribute in ABNF [RFC5234] (Crocker, D. and P. Overell, “Augmented BNF for Syntax Specifications: ABNF,” January 2008.) is as follows:

     media-grid-dims-line = "a=media-grid-dims:" row "x" column CRLF

     row    = %x31-39 *DIGIT
     column = %x31-39 *DIGIT

The REQUIRED parameters "row" and "column" indicate the number of rows and columns for this media grid. They both MUST be an integer larger than zero.

If the "media-grid-dims" attribute does not exist in the SDP, then a 1 x N horizontal linear layout MUST be assumed.

Open Issue: I think this should be a session-level attribute, however what if I wanna define two grids? Maybe put an identifier field?

Per [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.), there MAY be more than one adjacent media group in a single SDP session.

Editor's note: Should we use "output unit" or "input unit" here?

Open Issue: What if the offerer wants to send 32 streams in a grid size of 64? Consider that he wants to send a separate video to each white-colored unit in a chessboard-like grid (skipping the black-colored units)? These streams are still adjacent but they have a "space" in between them. Does he need to define empty m lines for the black-colored units? Or should we maybe define an optional "bitmap" parameter for the media-grid-dims attribute? E.g., bitmap=101010 will indicate that I have 3 active streams and one empty screen between each. Given the use cases, it seems best to go with the simplest solution and just not support this type of variable spaced layout.

3.3.  Grouping for SSRC-Multiplexed RTP Streams

Editor's note: We should also define the grouping semantics for SSRC multiplexed streams (i.e., a=ssrc-group:ADJ) as in [RFC5576] (Lennox, J., Ott, J., and T. Schierl, “Source-Specific Media Attributes in the Session Description Protocol (SDP),” June 2009.).

3.4.  SDP Offer/Answer Model Considerations

When offering adjacent media grouping using SDP in an Offer/Answer model [RFC3264] (Rosenberg, J. and H. Schulzrinne, “An Offer/Answer Model with Session Description Protocol (SDP),” June 2002.), the following considerations apply.

A node that is receiving an offer from a sender may or may not understand line grouping. It is also possible that the node understands line grouping but it does not understand the "ADJ" semantics. From the viewpoint of the sender of the offer, these cases are indistinguishable.

When a node is offered a session with the "ADJ" grouping semantics but it does not support line grouping or the adjacent media grouping semantics, as per [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.), the node responds to the offer either (1) with an answer that ignores the grouping attribute or (2) with a refusal to the request (e.g., 488 Not Acceptable Here or 606 Not Acceptable in SIP).

In the first case, the original sender of the offer must send a new offer without any adjacent media grouping. In the second case, if the sender of the offer still wishes to establish the session, it should retry the request with an offer without the adjacent media grouping. This behavior is specified in [RFC5888] (Camarillo, G. and H. Schulzrinne, “The Session Description Protocol (SDP) Grouping Framework,” June 2010.).

The SIP offer MUST contain the sender's desired layout. The answer MAY contain the desired layout of the streams that the system sending the answer will be sending to the system that sent the offer.

Editor's note: Does it have to be a SIP offer/answer? Or should we rather just say "offer/answer"?

4.  SDP Examples

This section provides SDP examples showing how to use the adjacent media grouping.

4.1.  Horizontal Layout

A video conferencing system with three screens and six audio channels sends a SIP offer to a video conferencing system with two screens that support stereo audio. The following figure shows a top-down view of the room with the three screen system that is sending the SIP offer. Screen A is the left most screen for the user in this room but should be displayed as the rightmost screen for the user at the far end. The M1 to M6 indicate the placement of the audio microphones for the six streams of audio.

Editor Note: Clear up confusion about if right or left is from user or camera point of view. Suggest we change all of this to be from camera point of view.

  Screen A      Screen B     Screen C
[----------][------------][------------]
 M1      M2  M3        M4  M5        M6


                 User

Assume the SDP mid values for the screens are sa, sb, and sc, for Screens A through C respectively, and the audio streams have mid values m1 through m6. The offer contains the following in the SDP:

    a=group:ADJ sc sb sa
    a=group:ADJ m6 m5 m4 m3 m2 m1

There might be other media streams, such as presentation video, that are not part of any "ADJ" group. If the far end decides to only accept the video from Screen A and Screen B and takes the audio only from M1 and M6, the answer will contain the following in the SDP:

    a=group:ADJ sb sa
    a=group:ADJ m6 m1

As a note to implementors, consider the case where each screen had two media flows that were in the same FID group. In this case all the media streams are still listed in the ADJ group and the order of two streams in the same FID group can be arbitrarily picked as they will be displayed on the same device.

Editor's note: In the above SDPs, maybe put "sa sb sc" rather than the reverse order?

TBD - Put in full SDP for examples.

4.2.  Grid Layout

A system if providing 15 video streams to a wall of screens. The wall has 5 columns and 3 rows of screens.

TBD

5.  Security Considerations

Like all SDP, integrity of this information is important. When carrying SDP in SIP, mechanisms such as Transport Layer Security (TLS) can provide hop by hop confidentiality and integrity. The receiver SHOULD do an integrity check on SDP and follow the security considerations of SDP [RFC4566] (Handley, M., Jacobson, V., and C. Perkins, “SDP: Session Description Protocol,” July 2006.) to trust only SDP from trusted sources. End-to-end integrity can be provided by [RFC4474] (Peterson, J. and C. Jennings, “Enhancements for Authenticated Identity Management in the Session Initiation Protocol (SIP),” August 2006.).

6.  IANA Considerations

Note to RFC Editor: Please replace [RFC-AAAA] with the RFC number for this specification.

6.1.  Registration of SDP Attributes

This document registers a new attribute name in SDP.

SDP Attribute ("att-field"):
     Attribute name:     media-grid-dims
     Long form:          2-D media grid dimensions
     Type of name:       att-field
     Type of attribute:  Session level
     Subject to charset: No
     Purpose:            Specifies the dimensions for a media grid
     Reference:          [RFC-AAAA]
     Values:             See [RFC-AAAA]

6.2.  Registration of Grouping Semantics

This document, following the Standards Action policy from [RFC5226] (Narten, T. and H. Alvestrand, “Guidelines for Writing an IANA Considerations Section in RFCs,” May 2008.), registers the following semantics with IANA in the "Semantics for the "group" SDP Attribute" registry under SDP Parameters:

Semantics                         Token Reference
--------------------------------- ----- -----------
Adjacent Media                    ADJ   [RFC-AAAA]

7.  Acknowledgments

The authors would like to thank Flemming Andreasen, Allyn Romanow, Roni Even, Hakon Dahle, Ingemar Johansson, Peter Musgrave, and Geir Arne Sandbakken for their review comments.

8.  Open Issues

More example can be added.

9.  References

9.1. Normative References

9.2. Informative References

Authors' Addresses