| Network Working Group | P. Kyzivat |
| Internet-Draft | L. Xiao |
| Intended status: Standards Track | C. Groves |
| Expires: January 16, 2014 | Huawei |
| R. Hansen | |
| Cisco Systems | |
| July 15, 2013 |
CLUE Signaling
draft-kyzivat-clue-signaling-04
This document specifies how signaling is conducted in the course of CLUE sessions. This includes how SIP/SDP signaling is applied to CLUE sessions as well as defining a CLUE-specific signaling protocol that complements SIP/SDP and supports negotiation of CLUE application level data.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 16, 2014.
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
This document specifies how signaling is conducted in the course of CLUE sessions. This includes how SIP/SDP signaling is applied to CLUE sessions as well as defining a CLUE-specific signaling protocol that complements SIP/SDP and supports negotiation of CLUE application level data.
[Yes, this is a dup of the abstract for now. Eventually it should say more.]
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
This document draws liberally from the terminology defined in the CLUE Framework [I-D.ietf-clue-framework].
Other terms introduced here:
The CLUE Framework [I-D.ietf-clue-framework] mentions a CLUE-specific protocol for the exchange of ADVERTISEMENT and CONFIGURE messages, but gives little detail. The Data Model [I-D.presta-clue-data-model-schema] specifies a model and XML representation for CLUE-related data, but doesn't currently specify exactly what data belongs in each message, or how messages are sequenced. This document provides the detail missing from those documents.
There must be some provision for identifying incompatible protocol versions.
NOTE: We probably don't want to have incompatible versions. Typically changes will be introduced in a backward compatible way. But a time may come when this isn't possible, and we should be prepared for that. This is more likely to occur before an RFC is published. While it is probably unwise to deploy a product based on a draft, there will certainly be prototypes developed for testing, and those tests may lead to a need for incompatible change. So whatever the mechanism is, it should be applicable to changes that occur from draft to draft, as well as after an RFC has been published.
There must be some provision for dealing with optional-to-implement features in the specification, and/or for backward compatible extensions to the protocol. These are superficially different, but in practice they are more-or-less equivalent. To an implementation of the base protocol and some extensions, those extensions must be viewed as optional-to-implement features in peers.
One decision is whether extensions may be implemented mix-and-match, or whether there is a sequence of extensions, and one extension may only be supported if all the prior extensions have been supported.
Both version and options can be negotiated. Some mechanisms may work for both, while others are only appropriate for one or the other. Some possibilities:
The CLUE channel is reliable, so there is no need for acknowledgement to guarantee delivery. But there is still a need for application-to-application acknowledgement to report that the message has been received, parsed, and found to be of an acceptable format. One possibility is to introduce separate ACK and NAK messages. Another possibility is to add a confirmation element to each CLUE message, so that confirmation can be piggybacked on the basic messages. Some alternatives follow. [OTHER PROPOSALS WELCOME.]
The characteristics of this approach are:
The characteristics of this approach are:
The general format of every message is:
(The exact representation is TBD - by XML experts.)
There are loose ends to resolve here. In particular, how to acknowledge messages after NAKing one.
There needs to be a mechanism to report errors with other messages. The details of form, content, and usage still need to be specified, and need to be tuned to the details of the protocol. This could use distinct messages or be incorporated into the other messages. Errors this message must be able to report include:
Each message exchanged within a CLUE session could contain a complete description of the state it wishes to achieve. Or each message could describe just the changes that it wishes to make to the current state. Or the protocol could support both message forms. Which direction to pursue is TBD.
[Paul: while this does need to be decided, it is fundamentally just an optimization. IMO it does not have major impact on the other parts of this document, so I would prefer to continue deferring it until we are so far along with the remainder of the document that we can no longer defer it.]
There is a very basic introduction to this topic in section 4 (Overview) of the CLUE Framework [I-D.ietf-clue-framework]. After removing extraneous material it would look like:
+-----------+ +-----------+
| Endpoint1 | | Endpoint2 |
+----+------+ +-----+-----+
| |
| ADVERTISEMENT 1 |
|*********************************>|
| ADVERTISEMENT 2 |
|<*********************************|
| |
| CONFIGURE 1 |
|<*********************************|
| CONFIGURE 2 |
|*********************************>|
| |
But we need much more than this, to show multiple CONFIGUREs per ADVERTISEMENT, interleaving of ADVERTISEMENTs and CONFIGUREs in both directions, etc.
Message sequencing needs to be described at two levels:
Once a CLUE session has been established, ADVERTISEMENTs and CONFIGUREs exchanged, and media is flowing, a provider may experience a change in state that has an effect on what it wishes or is able to provide. In this case it may need to alter what it is sending and/or send a new ADVERTISEMENT. In some cases it will be necessary to alter what is being sent without first sending a new ADVERTISEMENT and waiting for a CONFIGURE conforming to it.
The following is a non-exhaustive list of situations and recommended actions:
If the Consumer for some reason looses the CLUE state information how does it ask for an Advertisement from the provider? There could be multiple possibilities. A error code approach? However error codes would typically be associated with a NACK so it may not be good for a Config message. Maybe send a message which means “send me a complete update”. An alternative may be to release the connection or just do new signaling to establish a new CLUE session.
CLUE messages are transported over a bidirectional CLUE channel. In a two-party CLUE session, a CLUE channel connects the two endpoints. In a CLUE conference, each endpoint has a CLUE channel connecting it to an MCU. (In conferences with cascaded mixers [RFC4353], two MCUs will be connected by a CLUE channel.)
The transport mechanism used for CLUE messages is DTLS/SCTP as specified in [I-D.tuexen-tsvwg-sctp-dtls-encaps] and [I-D.ietf-mmusic-sctp-sdp]. A CLUE channel consists of one SCTP stream in each direction over a DTLS/SCTP session. The mechanism for establishing the DTLS/SCTP session is described in Section 4.
The CLUE channel will usually be offered during the initial SIP INVITE, and remain connected for the duration of the CLUE/SIP session. However this need not be the case. The CLUE channel may be established mid-session after desire and capability for CLUE have been determined, and the CLUE channel may be dropped mid-call if the desire and/or capability to support it is lost.
There may be cases when it becomes necessary to "reset" the CLUE channel. This by be as a result of an error on the underlying SCTP association, a need to change the endpoint address of the SCTP association, loss of CLUE protocol state, or something else TBD.
The precise mechanisms used to determine when a reset is required, and how to accomplish it and return to a well defined state are TBS.
We will need to specify behavior in the face of transport errors that are so severe that they can't be managed via CLUE messaging within the CLUE channel. Some errors of this sort are:
The worst case is to drop the entire CLUE call. Another possibility is to fall back to legacy compatibility mode. Or perhaps a "reset" can be done on the protocol. E.g. this might be accomplished by sending a new O/A and establishing a replacement SCTP association. Or a new CLUE channel might be established within the existing SCTP association.
CLUE messages are encoded in XML. The Data Model [I-D.presta-clue-data-model-schema] defines many/most of the elements from which CLUE messages are composed. This document specifies an XML schema that contains an element definition for each CLUE message, with much of the content of those elements being drawn from the Data Model.
This message contains XML representations of captures, capture scenes, encoding groups, and simultaneous sets using the types defined for those in the Data Model [I-D.presta-clue-data-model-schema].
The XML definition for this is element <advertisement> in section Section 3.7
[[ Currently this does not contain any representation of encodings. It assumes those will be defined in SDP. ]]
This message optionally contains an XML representations of captureEncodings using the type defined in the Data Model [I-D.presta-clue-data-model-schema]. A configure message with no captureEncodings indicates that no captures are requested.
[[ It currently also contains a reference to the request number of the advertisement it is based upon. Whether this should be present, or if it should implicitly reference the most recently acknowledged advertisement is TBD. ]]
The XML definition for this is element <configure> in section Section 3.7
Need for, and details of, the ACK message are TBD.
The XML element <response> in section Section 3.7 could serve as the representation, either with no reason element, or a reason element with a special value.
Need for, and details of, the NACK message are TBD.
The XML element <response> in section Section 3.7 could serve to as the representation, with the reason element providing the details. Then the code value in the reason element should map to the errors in section Section 3.2.3.
[[ The following is a first cut at a schema for the actual messages in the clue protocol. It uses <encodingGroups> from the data model but not <encodings>. Rather, it assumes that encodings are described in SDP as m-lines with a text identifier, and that the identifier has the same value as the encodingIDs embedded in the <encodingGroups>. If we stick with this the data model should be adjusted to agree, but until then it should "work". The SDP encoding of the identifier is proposed to be 'a=label:ID', though 'a=mid:ID' is another candidate. ]]
For now there only <advertisement> and <configure> are defined. More messages will be needed for acknowledgment.
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema
targetNamespace="urn:ietf:params:xml:ns:clue-message"
xmlns:tns="urn:ietf:params:xml:ns:clue-message"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:dm="urn:ietf:params:xml:ns:clue-info"
xmlns="urn:ietf:params:xml:ns:clue-message"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<!-- Import data model schema -->
<xs:import namespace="urn:ietf:params:xml:ns:clue-info"
schemaLocation="clue-data-model-04-wip.xsd"/>
<!-- ELEMENT DEFINITIONS -->
<xs:element name="advertisement" type="advertisementMessageType"/>
<xs:element name="configure" type="configureMessageType"/>
<xs:element name="response" type="responseMessageType"/>
<!-- CLUE MESSAGE TYPE -->
<xs:complexType name="clueMessageType" abstract="true">
<xs:sequence>
<!-- mandatory fields -->
<!-- TBS: version info -->
</xs:sequence>
</xs:complexType>
<!-- CLUE REQUEST MESSAGE TYPE -->
<xs:complexType name="clueRequestMessageType" abstract="true">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="requestNumber" type="xs:integer"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- CLUE RESPONSE MESSAGE TYPE -->
<xs:complexType name="clueResponseMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="requestNumber" type="xs:integer"/>
<!-- optional fields -->
<xs:element name="reason" type="reasonType" minOccurs="0"/>
<xs:any namespace="##other"
processContents="lax" minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- CLUE ADVERTISEMENT MESSAGE TYPE -->
<xs:complexType name="advertisementMessageType">
<xs:complexContent>
<xs:extension base="clueRequestMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="mediaCaptures"
type="dm:mediaCapturesType"/>
<xs:element name="encodingGroups"
type="dm:encodingGroupsType"/>
<!-- The encodings are defined via identifiers in the SDP,
referenced in encodingGroups -->
<xs:element name="captureScenes"
type="dm:captureScenesType"/>
<!-- optional fields -->
<xs:element name="simultaneousSets"
type="dm:simultaneousSetsType" minOccurs="0"/>
<xs:any namespace="##other"
processContents="lax" minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- CLUE CONFIGURE MESSAGE TYPE -->
<xs:complexType name="configureMessageType">
<xs:complexContent>
<xs:extension base="clueRequestMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="advertisementNumber" type="xs:integer"/>
<!-- advertisementNumber is requestNumber
of the advertisement-->
<!-- optional fields -->
<xs:element name="captureEncodings"
type="dm:captureEncodingsType" minOccurs="0"/>
<xs:any namespace="##other"
processContents="lax" minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- REASON TYPE -->
<xs:complexType name="reasonType">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute type="xs:short" name="code" use="required"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:schema>
Message framing is provided by the SCTP transport protocol. Each CLUE message is carried in one SCTP message.
The CLUE channel is usually offered in the first SIP O/A exchange between two parties in an intended CLUE session. The offer of the CLUE channel is the indicator that this SIP session is proposing to establish a CLUE session.
(However it is also acceptable to start with a non-CLUE SIP session and upgrade it to a CLUE session later.)
The mechanism for negotiating a DTLS/SCTP connection is specified in [I-D.ietf-mmusic-sctp-sdp]. We need to specify how to select the specific pair of SCTP streams that comprise the CLUE channel.
The presence of an active m-line for the CLUE channel in an SDP offer is an indication that the offer that the sender is CLUE-capable and hence can understand CLUE-specific syntax.
Many CLUE constructs have no good analog in SDP. Entities such as 'captures', which describe spatial and other properties of a capture source such as a camera, are not tied directly to RTP streams, do not have negotiated properties and would prove a significant challenge to represent in SDP syntax (while also greatly increasing the size of the SDP).
However, two entities defined in the CLUE Framework [I-D.ietf-clue-framework] are a much closer fit for SDP: Encodings and Encoding Groups. Both describe RTP media properties and limitations, though unlike most SDP usage they describe the sender's capabilities, not the receiver's. Representing encodings in CLUE splits media limitations across two protocols, and risks duplicated and potentially contradictory information being sent in CLUE and SDP. As such we are exploring representing this information in SDP, with the decision to convey them in the CLUE messages only to be made if the SDP approach proves impractical.
This draft presents an attempt to describe CLUE encodings in SDP. As a decision has not yet been reached on how multiplexed RTP streams are to be expressed in SDP, at this stage the draft does so without multiplexing, using existing SDP attributes, with a seperate "m" line and hence port per unidirectional RTP stream. This is done with the understanding that when a decision is reached on new syntax for multiplexing RTP streams in SDP the CLUE SDP signaling will be modified to use it. Further, the framework document states that the multiplexing of streams by an implementation is optional, and in the case of a disaggregated system, with media streams going to different addresses, may not be possible.
With the current scheme of using existing syntax, an encoding is specified in SDP as a unicast "m" line, which MUST be marked as sendonly with the "a=sendonly" attribute or as inactive with the "a=inactive" attribute. The encoder capabilities of the stream are defined here using existing syntax; for instance, for H.264 see Table 6 in [RFC6184] for a list of valid parameters for representing encoder sender stream limits.
Every "m" line representing a CLUE encoding SHOULD contain a "label" attribute as defined in [RFC4574]. This label is used to identify the encoding by the sender in CLUE Advertisement messages and by the receiver in CLUE Configure messages.
A receiver who wishes to receive a CLUE stream via this encoding requires a matching "a=recvonly" "m" line. As well as the normal restrictions defined in [RFC3264] media MUST NOT be sent on this stream until the sender has received a valid CLUE Configure message specifying the capture to be used for this stream.
As per the previous section, there would be advantages to conveying encoding group information in SDP. However, with current SDP syntax there is no way to express the encoding group limits defined in the Data Model [I-D.presta-clue-data-model-schema]. As such the current draft keeps encoding groups as part of the Advertisement message for the time being.
In many cases an implementation may wish to mix media channels that are under CLUE control with those that are not. It may want to ensure that there are non-CLUE streams for purposes of interoperability, or that can provide media from the start of the call before CLUE negotiation completes, or because the implementation wants CLUE-controlled video but traditional audio, or for any other reasons.
Which "m" lines in an SDP body are under control of the CLUE channel is signalled via the SDP Grouping Framework [RFC5888]. Devices that wish to negotiate CLUE MUST support the grouping framework.
A new semantic for the "group" session-level attribute, "CLUE", is used to signal which "m" lines are under the control of a CLUE channel. As per the framework, all of the "m" lines of a session description that uses "group" MUST be identified with a "mid" attribute whether they are controlled by CLUE or not. The "mid" id of any "m" lines controlled by a CLUE channel MUST be included in the "CLUE" group attribute alongside the "mid" id of the CLUE channel controlling them.
The CLUE group MUST NOT include more than one "m" line for a CLUE channel. If a CLUE channel is part of the CLUE group attribute other media "m" lines included in the group are under the control of that CLUE channel; media MUST NOT be sent or received on these "m" lines until the CLUE channel has been negotiated and negotiation has taken place as defined in this document. If no CLUE channel is part of the CLUE group attribute then media MUST NOT be sent or received on these "m" lines.
"m" lines not specified as under CLUE control follow normal rules for media streams negotiated in SDP as defined in documents such as [RFC3264].
An SDP MAY include more than one group attribute with the "CLUE" semantic. An "mid" id for a given "m" line MUST NOT be included in more than one CLUE group.
A CLUE-capable device sending an initial SDP offer SHOULD include an "m" line for the CLUE channel, but SHOULD NOT include any other CLUE-controlled "m" lines. Once each side of the call is aware that the other side is CLUE-capable a new O/A exchange MAY be used to add CLUE-controlled "m" lines.
Information about media streams in CLUE is split between two message types: SDP, which defines media addresses and limits, and the CLUE channel, which defines properties of capture devices available, scene information and additional constraints. As a result certain operations, such as advertising support for a new transmissible capture with associated stream, cannot be performed atomically, as they require changes to both SDP and CLUE messaging.
This section defines how the negotiation of the two protocols interact, provides some recommendations on dealing with intermediary stages in non-atomic operations, and mandates additional constraints on when CLUE-configured media can be sent.
To avoid complicated state machines with the potential to reach invalid states if messages were to be lost, or be rewritten en-route by middle boxes, the current proposal is that SDP and CLUE messages are independent. The state of the CLUE channel does not restrict when an implementation may send a new SDP offer or answer, and likewise the implementation's ability to send a new CLUE Advertisement or Configure message is not restricted by the results of or the state of the most recent SDP negotiation.
The primary implication of this is that a device may receive an SDP with a CLUE encoding it does not yet have capture information for, or receive a CLUE Configure message specifying a capture encoding for which the far end has not negotiated a media stream in SDP.
CLUE messages contain an EncodingID which is used to identify a specific encoding in SDP. The non-atomic nature of CLUE negotiation means that a sender may wish to send a new Advertisement before the corresponding SDP message. As such the sender of the CLUE message MAY include an EncodingID which does not currently match an extant id in SDP.
Generally, implementations that receive messages for which they have incomplete information SHOULD wait until they have the corresponding information they lack before sending messages to make changes related to that information. For instance, an implementation that receives a new SDP offer with three new "a=sendonly" CLUE "m" lines that has not received the corresponding CLUE Advertisement providing the capture information for those streams SHOULD NOT include corresponding "a=recvonly" lines in its answer, but instead should make a new SDP offer when and if a new Advertisement arrives with captures relevant to those encodings.
Because of the constraints of offer/answer and because new SDP negotiations are generally more 'costly' than sending a new CLUE message, implementations needing to make changes to both channels SHOULD prioritize sending the updated CLUE message over sending the new SDP message. The aim is for the recipient to receive the CLUE changes before the SDP changes, allowing the recipient to send their SDP answers without incomplete information, reducing the number of new SDP offers required.
While SDP and CLUE message states do not impose constraints on each other, both impose constraints on the sending of media - media MUST NOT be sent unless it has been negotiated in both CLUE and SDP: an implementation MUST NOT send a specific CLUE capture encoding unless its most recent SDP exchange contains an active media channel for that encoding AND the far end has sent a CLUE Configure message specifying a valid capture for that encoding.
This example illustrates a call between two CLUE-capable endpoints. Alice, initiating the call, is a system with three cameras and three screens. Bob, receiving the call, is a system with two cameras and two screens. A call-flow diagram is presented, followed by an summary of each message.
To manage the size of this section only video is considered, and SDP snippets only illustrate video 'm' lines. ACKs are not discussed.
+----------+ +-----------+
| Alice | | Bob |
| | | |
+----+-----+ +-----+-----+
| |
| |
| INVITE 1 (BASIC SDP+COMEDIA) |
|--------------------------------->|
| |
| |
| 200 OK 2 (BASIC SDP+COMEDIA) |
|<---------------------------------|
| |
| |
| ACK 1 |
|--------------------------------->|
| |
| |
| |
|<########### MEDIA 1 ############>|
| 1 video A->B, 1 video B->A |
|<################################>|
| |
| |
| |
|<================================>|
| CLUE CTRL CHANNEL ESTABLISHED |
|<================================>|
| |
| |
| ADVERTISEMENT 1 |
|*********************************>|
| |
| |
| ADVERTISEMENT 2 |
|<*********************************|
| |
| |
| INVITE 2 (+3 sendonly) |
|--------------------------------->|
| |
| |
| CONFIGURE 1 |
|<*********************************|
| |
| |
| 200 OK 2 (+2 recvonly) |
|<---------------------------------|
| |
| |
| ACK 2 |
|--------------------------------->|
| |
| |
| |
|<########### MEDIA 2 ############>|
| 2 video A->B, 1 video B->A |
|<################################>|
| |
| |
| INVITE 3 (+2 sendonly) |
|<---------------------------------|
| |
| |
| CONFIGURE 3 |
|*********************************>|
| |
| |
| 200 OK 3 (+2 recvonly) |
|--------------------------------->|
| |
| |
| |
| ACK 3 |
|<---------------------------------|
| |
| |
| |
|<########### MEDIA 3 ############>|
| 2 video A->B, 2 video B->A |
|<################################>|
| |
| |
| |
v v
...
a=group:CLUE 3
...
m=video 6002 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:2
In INVITE 1, Alice sends Bob a SIP INVITE including in the SDP body the basilar audio and video capabilities ("BASIC SDP") and the information needed for opening a control channel to be used for CLUE protocol messages exchange, according to what is envisioned in the COMEDIA approach ("COMEDIA") for DTLS/SCTP channel [I-D.ietf-mmusic-sctp-sdp]. A snippet of the SDP showing the grouping attribute and the video m-line are shown below (mid 3 represents the CLUE channel):
Bob responds with a similar SDP (200 OK 1); due to their similiarity no SDP snippet is shown here. Alice and Bob are each able to send a single audio and video stream (whether they choose to send this initial media before CLUE has been negotiated is implementation-dependent). This is illustrated as MEDIA 1.
With the successful initial O/A Alice and Bob are also free to negotiate the CLUE channel. Once this is successfully established CLUE negotiation can begin. This is illustrated as CLUE CHANNEL ESTABLISHED.
Alice now sends her CLUE Advertisement (ADVERTISEMENT 1). She advertises three static captures representing her three cameras. She also includes switched captures suitable for two- and one-screen systems. All of these captures are in a single capture scene, with suitable capture scene entries to tell Bob that he should either subscribe to the three static captures, the two switched capture view or the one switched capture view. Alice has no simultaneity constraints, so includes all six captures in one simultaneous set. Finally, Alice includes an encoding group with three encoding IDs: "enc1", "enc2" and "enc3". These encoding ids aren't currently valid, but will match the next SDP offer she sends.
Bob received ADVERTISEMENT 1 but does not yet send a Configure message, because he has not yet received Alice's encoding information, so as yet he does not know if she will have sufficient resources to send him the two streams he ideally wants at a quality he is happy with.
Bob also sends his CLUE Advertisement (ADVERTISEMENT 2). He advertises two static captures representing his cameras. He also includes a single composed capture for single-screen systems, in which he will composite the two camera views into a single video stream. All three captures are in a single capture scene, with suitable capture scene entries to tell Alice that she should either subscribe to the two static captures, or the single composed capture. Bob also has no simultaneity constraints, so includes all three captures in one simultaneous set. Bob also includes a single encoding group with two encoding IDs: "foo" and "bar".
Similarly, Alices receives ADVERTISEMENT 2 but does not yet send a Configure message, because she has not yet received Bob's encoding information.
...
a=group:CLUE 3 4 5 6
...
m=video 6002 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:2
...
m=video 6004 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=mid:4
a=label:enc1
m=video 6006 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=mid:5
a=label:enc2
m=video 6008 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=mid:6
a=label:enc3
Alice now sends INVITE 2. She maintains the sendrecv audio, video and CLUE m-lines, and she adds three new sendonly m-lines to represents the maximum three encodings she can send. Each of these m-lines has a label corresponding to one of the encoding ids from ADVERTISEMENT 1. Each also has its mid added to the grouping attribute to show they are controlled by the CLUE channel. A snippet of the SDP showing the grouping attribute and the video m-lines are shown below (mid 3 represents the CLUE channel):
Bob now has all the information he needs to decide which streams to configure. As such he now sends CONFIGURE 1. This requests the pair of switched captures that represent Alice's scene, and he configures them with encoder ids "enc1" and "enc2".
Alice receives Bob's message CONFIGURE 1 but does not yet send the capture encodings specified, because at this stage Bob hasn't negotiated the ability to receive these streams in SDP.
...
a=group:CLUE 11 12 100
...
m=video 58722 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:10
...
m=video 58724 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:11
m=video 58726 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:12
m=video 0 RTP/AVP 96
Bob now sends his SDP answer as part of 200 OK 2. Alongside his original audio, video and CLUE m-lines he includes two active recvonly m-lines and a zeroed m-line for the third. He adds their mid values to the grouping attribute to show they are controlled by the CLUE channel. A snippet of the SDP showing the grouping attribute and the video m-lines are shown below (mid 100 represents the CLUE channel):
On receiving 200 OK 2 from Bob Alice is now able to send the two streams of video Bob requested - this is illustrated as MEDIA 2.
...
a=group:CLUE 11 12 13 14 100
...
m=video 58722 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:10
...
m=video 58724 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:11
m=video 58726 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:12
m=video 0 RTP/AVP 96
m=video 58728 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=label:foo
a=mid:13
m=video 58730 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=label:bar
a=mid:14
The constraints of offer/answer meant that Bob could not include his encoder information as new m-lines in 200 OK 2. As such Bob now sends INVITE 3 to generate a new offer. Along with all the streams from 200 OK 2 Bob also includes two new sendonly streams. Each stream has a label corresponding to the encoding ids in his ADVERTISEMENT 2 message. He also adds their mid values to the grouping attribute to show they are controlled by the CLUE channel. A snippet of the SDP showing the grouping attribute and the video m-lines are shown below (mid 100 represents the CLUE channel):
Having received this Alice now has all the information she needs to send CONFIGURE 2. She requests the two static captures from Bob, to be sent on encodings "foo" and "bar".
Bob receives Alice's message CONFIGURE 2 but does not yet send the capture encodings specified, because Alice hasn't yet negotiated the ability to receive these streams in SDP.
...
a=group:CLUE 3 4 5 7 8
...
m=video 6002 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:2
...
m=video 6004 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=mid:4
a=label:enc1
m=video 6006 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016
a=sendonly
a=mid:5
a=label:enc2
m=video 0 RTP/AVP 96
m=video 6010 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:7
m=video 6012 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=recvonly
a=mid:8
Alice now sends 200 OK 3, matching two recvonly m-lines to Bob's new sendonly lines. She includes their mid values in the grouping attribute to show they are controlled by the CLUE channel. A snippet of the SDP showing the grouping attribute and the video m-lines are shown below (mid 3 represents the CLUE channel):
Finally, on receiving 200 OK 3 Bob is now able to send the two streams of video Alice requested - this is illustrated as MEDIA 3.
Both sides of the call are now sending multiple video streams with their sources defined via CLUE negotiation. As the call progresses either side can send new Advertisement or Configure or new SDP negotiation to add, remove or change what they have available or want to receive.
In this brief example Alice is a CLUE-capable endpoint making a call to Bob, who is not CLUE-capable, i.e., it is not able to use the CLUE protocol.
+----------+ +-----------+
| EP1 | | EP2 |
| | | |
+----+-----+ +-----+-----+
| |
| |
| INVITE 1 (BASIC SDP+COMEDIA) |
|--------------------------------->|
| |
| |
| 200 0K 1 (BASIC SDP+*NO*COMEDIA) |
|<---------------------------------|
| |
| |
| ACK 1 |
|--------------------------------->|
| |
| |
| |
|<########### MEDIA 1 ############>|
| 1 video A->B, 1 video B->A |
|<################################>|
| |
| |
| |
| |
v v
...
a=group:CLUE 3
...
m=video 6002 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600
a=sendrecv
a=mid:2
In INVITE 1, Alice sends Bob a SIP INVITE including in the SDP body the basilar audio and video capabilities ("BASIC SDP") and the information needed for opening a control channel to be used for CLUE protocol messages exchange, according to what is envisioned in the COMEDIA approach ("COMEDIA") for DTLS/SCTP channel [I-D.ietf-mmusic-sctp-sdp]. A snippet of the SDP showing the grouping attribute and the video m-line are shown below (mid 3 represents the CLUE channel):
Bob is not CLUE capable, and hence does not recognize the "CLUE" semantic for the grouping attribute, not does he support the CLUE channel. He responds with an answer with audio and video, but with the CLUE channel zeroed.
From the lack of the CLUE channel Alice understands that Bob does not support CLUE, or does not wish to use it. Both sides are now able to send a single audio and video stream to each other. Alice at this point begins to send her fallback video: in this case likely a switched view from whichever camera shows the current loudest participant on her side.
The current proposal calls for a new "CLUE" semantic for the SDP Grouping Framework [RFC5888].
Any other SDP extensions required to support CLUE signaling should also be specified here. Then we will need to take action within MMUSIC to make those happen. This section should be empty and removed before this document becomes an RFC.
NOTE: The RTP mapping document [I-D.even-clue-rtp-mapping] is also likely to call for SDP extensions. We will have to reconcile how to coordinate these two documents.
(Placeholder) This may be unremarkable. If so we can drop it.
This may just describe how the degenerate form of the general mechanisms work for legacy devices. Or it may describe special case handling that we mandate as part of CLUE. Or it may just discuss non-normative things for implementors should consider.
We may want to rule this out of scope for now. But we should be thinking about this.
Here are issues pertinent to signaling that need resolution. Resolution will probably result in changes somewhere in this document, but may also impact other documents.
The team focusing on this draft consists of: Roni Even, Rob Hansen, Christer Holmberg, Paul Kyzivat, Simon Pietro-Romano, Roberta Presta.
Christian Groves has contributed detailed comments and suggestions.
The author list should be updated as people contribute substantial text to this document.
TBD
TBD
These were developed by Lennard Xiao, Christian Groves and Paul, so added Lennard and Christian as authors.
| [RFC4353] | Rosenberg, J., "A Framework for Conferencing with the Session Initiation Protocol (SIP)", RFC 4353, February 2006. |
| [RFC3264] | Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. |
| [RFC6184] | Wang, Y.-K., Even, R., Kristensen, T. and R. Jesup, "RTP Payload Format for H.264 Video", RFC 6184, May 2011. |
| [I-D.even-clue-sdp-clue-relation] | Even, R., "Signalling of CLUE and SDP offer/answer", Internet-Draft draft-even-clue-sdp-clue-relation-01, October 2012. |
| [I-D.even-clue-rtp-mapping] | Even, R. and J. Lennox, "Mapping RTP streams to CLUE media captures", Internet-Draft draft-even-clue-rtp-mapping-05, February 2013. |
| [I-D.hansen-clue-sdp-interaction] | Hansen, R., "SDP and CLUE message interactions", Internet-Draft draft-hansen-clue-sdp-interaction-00, February 2013. |