| TOC |
|
This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”
The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.
This Internet-Draft will expire on March 13, 2010.
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.
This document defines a set of metrics and their usage to evaluate the performance of end-to-end Session Initiation Protocol (SIP) based services in both production and testing environments. The purpose of this document is to combine a standard set of common metrics, allowing interoperable performance measurements, easing the comparison of industry implementations.
1.
Introduction and Scope
2.
Terminology
3.
Time Interval Measurement and Reporting
4.
SIP Performance Metrics
4.1.
Registration Request Delay (RRD)
4.2.
Ineffective Registration Attempts (IRA)
4.3.
Session Request Delay (SRD)
4.3.1.
Successful Session Setup SRD
4.3.2.
Failed Session Setup SRD
4.3.3.
Instant Messaging
4.4.
Session Disconnect Delay (SDD)
4.4.1.
Successful session completion SDD
4.4.2.
Failed session completion SDD
4.5.
Session Duration Time (SDT)
4.5.1.
Successful session duration SDT
4.5.2.
Failed session completion SDT
4.6.
Hops per Request (HpR)
4.7.
Session Establishment Ratio (SER)
4.7.1.
Instant Messaging
4.8.
Session Establishment Effectiveness Ratio (SEER)
4.9.
Session Defects Ratio (SDR)
4.10.
Ineffective Session Attempts (ISA)
4.11.
Session Disconnect Failures (SDF)
4.12.
Session Completion Ratio (SCR)
4.12.1.
Successful Session Completion
4.12.2.
Failed Session Completion
4.13.
Session Success Ratio (SSR)
5.
Metric Correlations
6.
Additional Considerations
6.1.
Back-to-back User Agent (B2BUA)
6.2.
Authorization and Authentication
6.3.
Forking
6.4.
Data Collection
6.5.
Testing Documentation
7.
Conclusions
8.
IANA Considerations
9.
Security Considerations
10.
Contributors
11.
Acknowledgements
12.
References
12.1.
Normative References
12.2.
Informative References
§
Authors' Addresses
| TOC |
SIP has become a widely-used standard among many service providers, vendors, and end users. Although there are many different standards for measuring the performance of signaling protocols, none of them specifically address SIP.
The scope of this document is limited to the definitions of a standard set of metrics for measuring and reporting SIP performance from an end-to-end perspective. The metrics introduce a common foundation for understanding and quantifying performance expectations between service providers, vendors, and the users of services based on SIP. The intended audience for this document can be found among network operators, who often collect information on the responsiveness of the network to customer requests for services.
Measurements of the metrics described in this document are affected by variables external to SIP. The following is a non-exhaustive list of examples:
Note that some metrics in this document may not apply to all applications of SIP. This document defines a list of pertinent metrics, which may be used individually or as a set based on the usage of SIP within the context of a given service.
The metrics defined in this document DO NOT take into consideration the impairment or failure of actual application processing of a request or response. The metrics do not distinguish application processing time from other sources of delay, such as packet transfer delay.
Metrics designed to quantify single device application processing performance are beyond the scope of this document.
This document does not provide any numerical objectives or acceptance threshold values for the SIP performance metrics defined below, as these items are beyond the scope of IETF activities, in general.
The metrics defined in this document are applicable in scenarios where the SIP messages launched (into a network under test) are dedicated messages for testing purposes, or where the messages are user-initiated and a portion of the live traffic present. These two scenarios are sometimes referred to as active and passive measurement, respectively.
| TOC |
The following terms and conventions will be used throughout this document:
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.) [RFC2119].
End-to-End - This is described as two or more elements utilized for initiating a request, receiving the request, and responding to the request. It encompasses elements as necessary to be involved in a session dialog between the originating user agent client (UAC), destination user agent server (UAS), and any interim proxies (may also include back-to-back user agent's (B2BUA's)). This may be relative to a single operator's set of elements or extend to encompass all elements (if beyond a single operator's network) associated with a session.
Session - As described in RFC 3261 (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261], SIP is used primarily to request, create, and conclude sessions. "These sessions include Internet telephone calls, multimedia distribution, and multimedia conferences." The metrics within this document measure the performance associated with the processes necessary to establish these sessions; therefore, they are titled as: Session Request Delay, Session Disconnect Delay, etc. Although the titles of many of the metrics include this term, they are specifically measuring the signaling aspects only. Each session is identified by a unique Call-ID.
Session Establishment - Session establishment occurs when a 200 OK response from the UAS has been received, in response to a corresponding UAC's INVITE setup request, indicating the session setup request was successful.
Session Setup - As referenced within the sub-sections of 4.2 in this document, session setup is the set of messages and included parameters directly related to the process of a UAC requesting to establish a session with a corresponding UAS. This is also described as a set of steps in order to establish "ringing" [RFC3261] (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.).
| TOC |
Many of the metrics defined in this memo utilize a clock to assess the time interval between two events. This section defines time- related terms and reporting requirements.
T1 - start time
This is the time instant (when a request is sent) that begins a continuous time interval. T1 occurs when the designated request has been processed by the SIP application and the first bit of the request packet has been sent from the UA or proxy (and is externally observable at some logical or physical interface).
T1 represents the time at which each request-response test begins, and SHALL be used to designate the time-of-day when a particular measurement was conducted (e.g., The Session Request Delay at "T1" and (some specific UA interface) was measured to be X ms.)
T4 - end time
This is the time instant that concludes the continuous time interval begun when the related request is sent. T4 occurs when the last bit of the designated response is received by the SIP application at the requesting device (and is externally observable at some logical or physical interface).
Note: The designations T2 and T3 are reserved for future use at another interface involved in satisfying a request.
Section 10.1 of RFC 2330 (Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, “Framework for IP Performance Metrics,” May 1998.) [RFC2330] describes time-related issues in measurements, and defines the errors that can be attributed to the clock themselves. These definitions are used in the material below.
Time of Day Accuracy
As defined above, T1 is associated with the start of a request and also serves as the time-of-day stamp associated with a single specific measurement. The clock offset [RFC2330] (Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, “Framework for IP Performance Metrics,” May 1998.) is the difference between T1 and a recognized primary source of time, such as UTC (offset = T1 - UTC).
When measurement results will be correlated with other results or information using time-of-day stamps, then the time clock that supplies T1 SHOULD be synchronized to a primary time source, to minimize the clock's offset. The clocks used at the different measurement points SHOULD be synchronized to each other, to minimize the relative offset (as defined in RFC2330). The clock's offset and the relative offset MUST be reported with each measurement.
Time Interval Accuracy
The accuracy of the T4-T1 interval is also critical to maintain and report. The difference between a clock's offsets at T1 and T4 is one source of error for the measurement and is associated with the clock's skew [RFC2330] (Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, “Framework for IP Performance Metrics,” May 1998.).
A stable and reasonably accurate clock is needed to make the time interval measurements required by this memo. This source of error SHOULD be constrained to less than +/- 1 ms, implying 1 part per 1000 frequency accuracy for a 1 second interval. This implies greater stability is required as the length of the T4-T1 increases, in order to constrain the error to be less than +/- 1ms.
There are several other important aspects of clock operation:
The physical operation of reading time from a clock may be constrained by the delay to service the interrupt. Therefore, if the accuracy of the time stamp read at T1 or T4 includes the interrupt delay, this source of error SHOULD be known and included in the error assessment.
| TOC |
In regards to all of the following metrics, T1 begins with the first associated SIP message sent by the UAC or UAS, and is not reset if the UAC or UAS must retransmit the same request, with the same Call-ID, multiple times. The first associated SIP message indicates the T1 associated with the user or application expectation relative to the request.
Some metrics are calculated based on the final response message. These metrics do not take into consideration route advances to additional signaling functions based on "final" failure responses. In these unique cases, the final response related to the initial setup attempt SHOULD be utilized for input to the metric.
In regards to all of the metrics, the accuracy and granularity of the output values are related to the accuracy and granularity of the input values.
The following metrics may be utilized for many different SIP applications.
| TOC |
Registration Request Delay (RRD) is a measurement of the delay in responding to a UAC REGISTER request. RRD SHALL be measured and reported only for successful REGISTER requests, while Ineffective Registration Attempts (Section 4.2) SHALL be reported for failures. This metric is measured at the UAC. The output value of this metric is numerical and SHOULD be stated in units of milliseconds. The RRD is calculated using the following formula:
RRD = Time of Final Response - Time of REGISTER Request
In a successful registration attempt, RRD is defined as the time interval from the first bit of the initial REGISTER message containing the necessary information is passed by the originating UAC to the intended registrar until the last bit of the 200 OK is received indicating the registration attempt has completed successfully. This dialog includes an expected authentication challenge prior to receiving the 200 OK as described in the following registration flow examples.
The following flow provides an example of identifiable events necessary for inputs in calculating RRD during a successful registration completion:
UA1 Registrar
| |
|REGISTER |
T1---->|--------------------->|
/\ | 401|
|| |<---------------------|
RRD |REGISTER |
|| |--------------------->|
\/ | 200|
T4---->|<---------------------|
| |
| TOC |
Ineffective registration attempts are utilized to detect failures or impairments causing an inability for a registrar to receive a UAC REGISTER request. This metric is measured at the UAC. The output value of this metric is numerical and SHOULD be reported as a percentage of registration attempts.
This metric is calculated as a percentage of total REGISTER requests. The IRA is calculated using the following formula:
# of IRA
IRA % = ----------------------------- x 100
Total # of REGISTER Requests
A failed registration attempt is defined as a final failure response to the initial REGISTER request. It usually indicates a failure received from the destination registrar, interim proxies, or due to a timeout of the REGISTER request at the originating UA. A failure response is described as a 4XX (excluding 401, 402, and 407 non- failure challenge response codes), 5XX, or possible 6XX message. A timeout failure is identified by the timer F expiring. IRA may be used to detect problems in downstream signaling functions, which may be impairing the REGISTER message from reaching the intended registrar; or, it may indicate a registrar has become overloaded and is unable to respond to the request.
The following flow provides a timeout example of an identifiable event necessary for input as a failed registration attempt:
UA1 Registrar
| |
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
| |
Failure ---->|***Timer F Expires |
| |
In the previous message flow the UAC retries a REGISTER request multiple times before the timer, indicating the failure, expires. Only the first REGISTER request MUST used for input to the calculation and an IRA. Subsequent REGISTER retries are identified by the same Call-ID and MUST be ignored for purposes of metric calculation. This ensures an accurate representation of the metric output.
The following flow provides a registrar servicing failure example of an identifiable event necessary for input as a failed registration attempt:
UA1 Registrar
| |
|REGISTER |
|--------------------->|
| |
| |
| |
| |
| 503|
Failure ---->|<---------------------|
| |
| TOC |
Session Request Delay is utilized to detect failures or impairments causing delays in responding to a UA session request. SRD is measured for both successful and failed session setup requests as this metric usually relates to a user experience; however, SRD for session requests ending in a failure MUST NOT be combined in the same result with successful requests. The duration associated with success and failure responses will likely vary substantially, and the desired output time associated with each will be significantly different in many cases. This metric is similar to Post-Selection Delay [E.721] (ITU-T, “Series E: Overall Network Operation, Telephone Service, Service Operation and Human Factors,” May 1999.) or Post-Dial Delay (PDD) in telephony applications of SIP, and it is measured at the UAC only. The output value of this metric MUST indicate whether the output is for successful or failed session requests and SHOULD be stated in units of seconds. The SRD is calculated using the following formula:
SRD = Time of Status Indicative Response - Time of INVITE
| TOC |
In a successful request attempt, SRD is defined as the time interval from the first bit of the initial INVITE message containing the necessary information is sent by the originating agent or user to the intended mediation or destination agent until the last bit of the first provisional response is received indicating an audible or visual status of the initial session setup request. In SIP, the message indicating status would be a non-100 Trying provisional message received in response to an INVITE request. In some cases, a non-100 Trying provisional message is not received, but rather a 200 message is received as the first status message instead. In these situations, the 200 message would be used to calculate the interval.
The following flow provides an example of identifiable events necessary for inputs in calculating SRD during a successful session setup request without a redirect (i.e. 3XX message):
UA1 UA2
| |
|INVITE |
T1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 180|
T4---->|<---------------------|
| |
The following flow provides an example of identifiable events necessary for inputs in calculating SRD during a successful session setup with a redirect (e.g. 302 Moved Temporarily):
UA1 Redirect Server UA2
| | |
|INVITE | |
T1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 180|
T4---->|<-------------------------------------------|
| TOC |
In a failed request attempt, SRD is defined as the time interval from the first bit of the initial INVITE message containing the necessary information sent by the originating agent or user to the intended mediation or destination agent until the last bit of the first provisional response or a failure indication response. A failure response is described as a 4XX (excluding 401, 402, and 407 non-failure challenge response codes), 5XX, or possible 6XX message. SRD may be used to detect problems in downstream signaling functions, which may be impairing the INVITE message from reaching the intended UA. While this metric calculates the delay associated with a failed session request, the metric Ineffective Session Attempts (Section 4.10) is used for calculating a ratio of session attempt failures.
The following flow provides an example of identifiable events necessary for inputs in calculating SRD during a failed session setup attempt without a redirect (i.e. 3XX message):
UA1 UA2
| |
|INVITE |
T1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 480|
T4---->|<---------------------|
| |
The following flow provides an example of identifiable events necessary for inputs in calculating SRD during a failed session setup attempt with a redirect (e.g. 302 Moved Temporarily):
UA1 Redirect Server UA2
| | |
|INVITE | |
T1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 480|
T4---->|<-------------------------------------------|
| TOC |
This metric is also applicable to MESSAGE [RFC3428] (Campbell, B., Rosenberg, J., Schulzrinne, H., Huitema, C., and D. Gurle, “Session Initiation Protocol (SIP) Extension for Instant Messaging,” December 2002.) requests. In the above metric, INVITE can be replaced with MESSAGE to provide SRD for instant messaging (IM). The dialog will vary slightly as described in RFC 3428 (Campbell, B., Rosenberg, J., Schulzrinne, H., Huitema, C., and D. Gurle, “Session Initiation Protocol (SIP) Extension for Instant Messaging,” December 2002.) [RFC3428]. The inputs for this metric SHOULD be utilized regardless of whether a prior SIP dialog was utilized to setup the session. In that case both the SIP dialog and the MESSAGE requests are measured independently.
The following flow provides an example of identifiable events necessary for inputs in calculating SRD during a successful session MESSAGE request:
UA1 UA2
| |
|MESSAGE |
T1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 200|
T4---->|<---------------------|
| |
Failure requests occur similarly as to those described in section 4.2.2 with MESSAGE in replacement of INVITE as the IM session request method.
| TOC |
This metric is utilized to detect failures or impairments delaying the time necessary to end a session. It can be measured from both a UAC and UAS perspective. SDD is measured for both successful and failed session completions. The output value of this metric is numerical and SHOULD be stated in units of milliseconds. The SDD is calculated using the following formula:
SDD = Time of 2XX or Timeout - Time of Completion Message (BYE)
| TOC |
In a successful session completion, SDD is defined as the interval between the first bit of the sent session completion message, such as a BYE, and the last bit of the subsequently received 2XX acknowledgment. The following flows provide an example of identifiable events necessary for inputs in calculating SDD during a successful session completion:
Measuring SDD at the UAC -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
|BYE |
T1---->|--------------------->|
/\ | |
|| | |
SDD | |
|| | |
\/ | 200|
T4---->|<---------------------|
Measuring SDD at the UAS -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
| BYE|
|<---------------------|<----T1
| | /\
| | ||
| | SDD
| | ||
|200 | \/
|--------------------->|<----T4
| TOC |
In some cases, no response is received after a session completion message is sent and potentially retried. In this case, SDD is defined as the interval between the first bit of the sent session completion message, such as a BYE, and the resulting Timer F expiration. The following flows provide an example of identifiable events necessary for inputs in calculating SDD during a failed session completion attempt:
Measuring SDD at the UAC -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
|BYE |
T1---->|--------------------->|
/\ |BYE |
|| |--------------------->|
SDD |BYE |
|| |--------------------->|
\/ | |
T4---->|***Timer F Expires |
Measuring SDD at the UAS -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
| BYE|
|<---------------------|<----T1
| BYE| /\
|<---------------------| ||
| BYE| SDD
|<---------------------| ||
| | \/
| Timer F Expires***|<----T4
| TOC |
This metric is used to detect problems (e.g. poor audio quality) causing short session durations. SDT is measured for both successful and failed session completions. It can be measured from both a UAC and UAS perspective. This metric is similar to Call Hold Time, and is traditionally calculated as Average Call Hold Time (ACHT) in telephony applications of SIP. The output value of this metric is numerical and SHOULD be stated in units of seconds. The SDT is calculated using the following formula:
SDT = Time of BYE or Timeout - Time of 200 OK response to INVITE
| TOC |
In a successful session completion, SDT is calculated as an average and is defined as the duration of a dialog defined by the interval from receipt of the first bit of a 200 OK response to an INVITE and receipt of the last bit of an associated BYE message indicating dialog completion.
The following flows provide an example of identifiable events necessary for inputs in calculating SDT during a successful session completion (The message flows are changed between the UAC and UAS to provide varying examples.):
Measuring SDT at the UAC -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
T1---->|<---------------------|
/\ |ACK |
|| |--------------------->|
|| | |
SDT | |
|| | |
|| | |
\/ |BYE |
T4---->|--------------------->|
| |
When measuring SDT at the UAS, it is defined by the interval from sending the first bit of a 200 OK response to an INVITE and receipt of the last bit of an associated BYE message indicating dialog completion. If the UAS initiates the BYE, then it is defined by the interval from sending the first bit of a 200 OK response to an INVITE and sending the first bit of an associated BYE message indicating dialog completion. This is illustrated in the following example message flow -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----T1
|ACK | /\
|--------------------->| ||
| | ||
| | SDT
| | ||
| | ||
| BYE| \/
|<---------------------|<----T4
| |
(In these two examples, T1 is the same even if the UAC/UAS receives the BYE instead of sending it.)
| TOC |
In some cases, no response is received after a session completion message is sent and potentially retried. In this case, SDT is defined as the interval between receiving the first bit of a 200 OK response to an INVITE, and the resulting Timer F expiration. The following flows provide an example of identifiable events necessary for inputs in calculating SDT during a failed session completion attempt:
Measuring SDT at the UAC -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
T1---->|<---------------------|
/\ |BYE |
|| |--------------------->|
|| |BYE |
SDT |--------------------->|
|| |BYE |
|| |--------------------->|
\/ | |
T4---->|***Timer F Expires |
When measuring SDT at the UAS, SDT is defined as the interval between sending the first bit of a 200 OK response to an INVITE, and the resulting Timer F expiration. This is illustrated in the following example message flow -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----T1
| BYE| /\
|<---------------------| ||
| BYE| ||
|<---------------------| SDT
| BYE| ||
|<---------------------| ||
| | \/
| Timer F Expires***|<----T4
| TOC |
This metric is used to indicate potential inefficient routing and to detect failure occurrences related to the number of elements traversed by a single SIP INVITE or MESSAGE request. HpR is defined as the number of hops traversed by an INVITE or MESSAGE request. This metric requires the Max-Forwards value to be captured at both the originating UAC or proxy and the terminating UAS or proxy perspective as relative to the end-to-end network under measurement. The output value of this metric is measured in a numerical value indicating a number of hops.
The following variables MUST be captured for use in the HpR formula:
a = Initial INVITE/MESSAGE "Max-Forwards" value
b = Initial INVITE/MESSAGE received by terminating UAS "Max- Forwards" value
c = # of Hops for INVITE/MESSAGE requests
Using the variables defined above, the HpR is calculated using the following formula:
c = a - b
The following dialog provides an example describing the inputs necessary for this calculation. Although this example is of an INVITE SIP dialog request, a MESSAGE request is similar in its use of the Max-Forwards header. (The dialog continuation was omitted for clarity):
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE (F4) | | |
|--------------->|INVITE (F5) | |
| 100|--------------->|INVITE (F6) |
|<---------------| 100|--------------->|
| |<---------------| |
Message Details (Only the message details of the INVITE messages have been included for clarity. Also, some headers after Max-Forwards have been omitted for additional clarity.):
(F4) INVITE UA1 -> Proxy 1
INVITE sip:ua2@biloxi.example.com SIP/2.0
Via: SIP/2.0/TCP
client.atlanta.example.com:5060;branch=z9hG4bK74bf9
Max-Forwards: 70
Route: <sip:ss1.atlanta.example.com;lr>
From: UA1 <sip:ua1@atlanta.example.com>;tag=9fxced76sl
To: UA2 <sip:ua2@biloxi.example.com>
(F5) INVITE Proxy 1 -> Proxy 2
INVITE sip:ua2@biloxi.example.com SIP/2.0
Via: SIP/2.0/TCP
ss1.atlanta.example.com:5060;branch=z9hG4bK2d4790.1
Via: SIP/2.0/TCP
client.atlanta.example.com:5060;branch=z9hG4bK74bf9
;received=192.0.2.101
Max-Forwards: 69
Record-Route: <sip:ss1.atlanta.example.com;lr>
From: UA1 <sip:ua1@atlanta.example.com>;tag=9fxced76sl
To: UA2 <sip:ua2@biloxi.example.com>
(F6) INVITE Proxy 2 -> UA2
INVITE sip:ua2@client.biloxi.example.com SIP/2.0
Via: SIP/2.0/TCP ss2.biloxi.example.com:5060;branch=z9hG4bK721e4.1
Via: SIP/2.0/TCP
ss1.atlanta.example.com:5060;branch=z9hG4bK2d4790.1
;received=192.0.2.111
Via: SIP/2.0/TCP
client.atlanta.example.com:5060;branch=z9hG4bK74bf9
;received=192.0.2.101
Max-Forwards: 68
Record-Route: <sip:ss2.biloxi.example.com;lr>,
<sip:ss1.atlanta.example.com;lr>
From: UA1 <sip:ua1@atlanta.example.com>;tag=9fxced76sl
To: UA2 <sip:ua2@biloxi.example.com>
| TOC |
This metric is used to detect the ability of a terminating UA or downstream proxy to successfully establish sessions per new session INVITE requests. SER is defined as the number of new session INVITE requests resulting in a 200 OK response, to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response. This metric is similar to Answer Seizure Ratio (ASR) [E.411] (ITU-T, “Series E: Overall Network Operation, Telephone Service, Service Operation and Human Factors,” March 2000.) in telephony applications of SIP. It is measured at the UAC only. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successfully established sessions. The SER is calculated using the following formula:
# of INVITE Requests w/ associated 200 OK
SER = --------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
The following flow provides an example of identifiable events necessary for inputs in determining session establishment as described above:
UA1 UA2
| |
|INVITE |
+----------->|------------------>|
| | 180|
| |<------------------|
Session Established | |
| | |
| | 200|
+----------->|<------------------|
| |
| TOC |
This metric is also applicable to MESSAGE [RFC3428] (Campbell, B., Rosenberg, J., Schulzrinne, H., Huitema, C., and D. Gurle, “Session Initiation Protocol (SIP) Extension for Instant Messaging,” December 2002.) requests. In the above metric, INVITE can be replaced with MESSAGE to provide SER for IM. The dialog will vary slightly as described in RFC 3428 (Campbell, B., Rosenberg, J., Schulzrinne, H., Huitema, C., and D. Gurle, “Session Initiation Protocol (SIP) Extension for Instant Messaging,” December 2002.) [RFC3428].
The following flow provides an example of identifiable events necessary for inputs in calculating SER for MESSAGE requests:
UA1 UA2
| |
|MESSAGE |
+----------->|------------------>|
| | |
Session Established | |
| | 200|
+----------->|<------------------|
| |
| TOC |
This metric is complimentary to SER, but is intended to exclude the potential effects of the terminating UAS from the metric. SEER is defined as the number of INVITE requests resulting in a 200 OK response and INVITE requests resulting in a 480, 486, or 600; to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX, 401, 402, and 407 response. This metric is similar to Network Effectiveness Ratio (NER) [E.411] (ITU-T, “Series E: Overall Network Operation, Telephone Service, Service Operation and Human Factors,” March 2000.) in telephony applications of SIP. It is measured at the UAC only. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successfully established sessions less common UAS failures.
In order to simplify the formula, the following variable is used to summarize multiple SIP responses:
a = 3XX, 401, 402, and 407
The SEER is calculated using the following formula:
# of INVITE Requests w/ associated 200 OK, 480, 486, or 600
SEER = -------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 'a' Response)
Reference the example flow is Section 4.7.
| TOC |
Session defects provide a subset of SIP failure responses, which consistently indicate a failure in dialog processing. Defects are necessary to provide input to calculations such as Defects per Million (DPM) or other similar metrics. These failure responses are in response to initial session setup requests, such as a new INVITE. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of defective sessions. The following failure responses provide a guideline for defective criterion:
This set of failure responses was derived through correlating more granular ISUP failure responses as described in RFC 3398 (Camarillo, G., Roach, A., Peterson, J., and L. Ong, “Integrated Services Digital Network (ISDN) User Part (ISUP) to Session Initiation Protocol (SIP) Mapping,” December 2002.) [RFC3398]. The SDR is calculated using the following formula:
# of INVITE Requests w/ associated 500, 503, or 504
SDR = ----------------------------------------------------- x 100
Total # of INVITE Requests
| TOC |
Ineffective session attempts occur when a proxy or agent internally releases a setup request with a failed or overloaded condition. This metric is similar to Ineffective Machine Attempts (IMA) in telephony applications of SIP, and was adopted from Telcordia GR-512-CORE [GR‑512] (Telcordia, “LSSGR: Reliability, Section 12,” January 1998.). The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of ineffective session attempts. The following failure responses provide a guideline for this criterion:
This set was derived in a similar manner as described in Section 4.9, in addition 408 failure responses is indicative a overloaded state with a downstream element.
This metric is calculated as a percentage of total session setup requests. The ISA is calculated using the following formula:
# of ISA
ISA % = ----------------------------- x 100
Total # of Session Requests
| TOC |
Session disconnect failures occur when an active session is terminated due to a failure condition that can be identified by a REASON header [RFC3326] (Schulzrinne, H., Oran, D., and G. Camarillo, “The Reason Header Field for the Session Initiation Protocol (SIP),” December 2002.) in a BYE message. This occurs, for example, when a user agent (UA) is controlling an IP or TDM (Time Division Multiplexing) media gateway, and the media gateway notifies the UA of a failure condition causing the loss of media related to an established session. The UA will release the session with a BYE, but SHOULD include a REASON header indicating the session was disconnected abnormally. The REASON value is utilized to determine the disconnect was a failure. This metric is similar to Cutoff Calls (CC) in telephony applications of SIP, and was adopted from Telcordia GR-512-CORE [GR‑512] (Telcordia, “LSSGR: Reliability, Section 12,” January 1998.). The input variables for this metric are captured from the originating UAC or proxy perspective as relative to the end- to-end network under measurement. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of session disconnect failures.
This metric is calculated as a percentage of total session completed successfully as defined in Section 3.5. The SDF is calculated using the following formula:
# of SDF's
SDF % = ------------------------------- x 100
Total # of Session Requests
| TOC |
A session completion is defined as a SIP dialog, which completes without failing due to a lack of response from an intended proxy or UA. This metric is only used when at least one proxy is involved in the dialog. This metric is similar to Call Completion Ratio (CCR) in telephony applications of SIP. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successfully completed sessions.
This metric is calculated as a percentage of total sessions completed successfully. The SCR is calculated using the following formula:
# of Successfully Completed Sessions
SCR % = --------------------------------------- x 100
Total # of Session Requests
| TOC |
A session completes successfully when it begins with a setup request and ends with a session completion message.
The following dialog [RFC3665] (Johnston, A., Donovan, S., Sparks, R., Cunningham, C., and K. Summers, “Session Initiation Protocol (SIP) Basic Call Flow Examples,” December 2003.) provides an example describing the necessary events of a successful session completion:
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | | 180|
| | 180 |<---------------|
| 180|<---------------| |
|<---------------| | 200|
| | 200|<---------------|
| 200|<---------------| |
|<---------------| | |
|ACK | | |
|--------------->|ACK | |
| |--------------->|ACK |
| | |--------------->|
| Both Way RTP Media |
|<================================================>|
| | | BYE|
| | BYE|<---------------|
| BYE|<---------------| |
|<---------------| | |
|200 | | |
|--------------->|200 | |
| |--------------->|200 |
| | |--------------->|
| | | |
| TOC |
Session completion fails when an INVITE is sent from a UAC, but there is no indication the INVITE reached the intended UAS. This can also occur if the intended UAS does not respond to the UAC or the response never reaches the UAC associated with the session.
The following dialog provides an example describing the necessary events of an unsuccessful session completion:
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | |INVITE |
| | |--------------->|
| | | |
| | |INVITE |
| | |--------------->|
| | | |
| | 408| |
| 408|<---------------| |
|<---------------|ACK | |
| |--------------->| |
|ACK | | |
|--------------->| | |
| TOC |
Session success ratio is defined as the percentage of successfully completed sessions compared to sessions, which fail due to ISA or SDF. This metric is also known as Call Success Ratio (CSR) in telephony applications of SIP. The output value of this metric is numerical and SHOULD be adjusted to indicate a percentage of successful sessions. The SSR is calculated using the following formula:
SSR = 100% - (ISA% + SDF%)
| TOC |
These metrics may be used to determine the performance of a domain and/or user. This would be to provide a metric relative to one or more dimensions. The following is a subset of dimensions for providing further granularity per metric:
| TOC |
| TOC |
A B2BUA may impact the ability to collect these metrics with an end- to-end perspective. It is necessary to realize a B2BUA may act as an originating UAC and terminating UAS or it may act as a proxy. In some cases, it may be necessary to consider information collected from both sides of the B2BUA in order to determine the end-to-end perspective. In other cases, the B2BUA may act simply as a proxy allowing data to be derived as necessary for the input into any of the listed calculations.
| TOC |
During the process of setting up a SIP dialog, various authentication methods may be utilized. These authentication methods will add to the duration as measured by the metrics, and the length of time will vary based on those methods. The failures of these authentication methods will also be captured by these metrics, since SIP is ultimately used to indicate the success or failure of the authorization and/or authentication attempt. The metrics in section 3 are inclusive of the duration associated with this process, even if the method is external to the SIP protocol. This was included purposefully, due to its inherent impact on the protocol and the subsequent SIP dialogs.
| TOC |
Forking SHOULD be considered when determining the messages associated with the input values for the described metrics. If all of the forked dialogs were used in the metric calculations, the numbers would skew dramatically. There are two different points of forking, which MUST be considered. First, forking may occur at a proxy downstream from the UAC that is being used for metric input values. Since, the downstream proxy is responsible for forking a message and then only sending the accepted response to the UAC, the UAC will only see messages as indicated in the described metrics. Second, in the cases where the observed UAC or proxy is forking the messages, then it MUST utilize the first INVITE or set of INVITE messages sent and the first accepted 200 OK. A tag will identify this dialog as distinct from the other 200 OK responses, which are acknowledged and an immediate BYE is sent. The application responsible for capturing and/or understanding the input values MUST utilize this tag to distinguish between dialogs.
| TOC |
The input necessary for these calculations may be collected in a number of different manners. It may be collected or retrieved from call detail records (CDR) or raw signaling information generated by a proxy or UA. When using records, time synchronization MUST be considered between applicable elements.
The information may also be transmitted through the use of network management protocols like Simple Network Management Protocol (SNMP) and via future extensions to the SIP Management Information Base (MIB) modules [RFC4780] (Lingle, K., Mule, J-F., Maeng, J., and D. Walker, “Management Information Base for the Session Initiation Protocol (SIP),” April 2007.), or through a potential undefined new performance metric event package [RFC3265] (Roach, A., “Session Initiation Protocol (SIP)-Specific Event Notification,” June 2002.) retrieved via SUBSCRIBE requests.
Data may be collected for a sample of calls or all calls, and may also be derived from test call scenarios. These metrics are flexible based on the needs of the application.
| TOC |
In some cases, these metrics will be used to provide output values to signify the performance level of a specific SIP-based element. When using these metrics in a test environment, the environment MUST be accurately documented for the purposes of replicating any output values in future testing and/or validation.
| TOC |
The proposed guideline provides a description of common performance metrics, and their defined use with SIP. The use of these metrics will provide a common viewpoint across all vendors, service providers, and users. These metrics will likely be utilized in production SIP environments for providing input regarding Key Performance Indicators (KPI) and Service Level Agreement (SLA) indications; however, they may also be used for testing end-to-end SIP-based service environments.
| TOC |
There are no IANA considerations at this time.
| TOC |
Security SHOULD be considered in the aspect of securing the relative data utilized in providing input to the above calculations. All other aspects of security SHOULD be considered as described in RFC 3261 (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261].
| TOC |
The following people made substantial contributions to this work:
Carol Davids Illinois Institute of Technology Marian Delkinov Ericsson Adam Uzelac Global Crossing Jean-Francois Mule CableLabs Rich Terpstra Level 3 Communications
| TOC |
We would like to thank John Hearty and Dean Bayless for their efforts in reviewing the draft and providing insight regarding clarification of certain aspects described throughout the draft. We also thank Dan Romascanu for his insightful comments and Vijay Gurbani for agreeing to perform the role of document shepherd.
| TOC |
| TOC |
| [E.411] | ITU-T, “Series E: Overall Network Operation, Telephone Service, Service Operation and Human Factors,” E.411 , March 2000. |
| [E.721] | ITU-T, “Series E: Overall Network Operation, Telephone Service, Service Operation and Human Factors,” E.411 , May 1999. |
| [GR-512] | Telcordia, “LSSGR: Reliability, Section 12,” GR-512-CORE Issue 2, January 1998. |
| [RFC2119] | Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” BCP 14, RFC 2119, March 1997 (TXT, HTML, XML). |
| [RFC3261] | Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” RFC 3261, June 2002 (TXT). |
| [RFC3265] | Roach, A., “Session Initiation Protocol (SIP)-Specific Event Notification,” RFC 3265, June 2002 (TXT). |
| [RFC3326] | Schulzrinne, H., Oran, D., and G. Camarillo, “The Reason Header Field for the Session Initiation Protocol (SIP),” RFC 3326, December 2002 (TXT). |
| [RFC3428] | Campbell, B., Rosenberg, J., Schulzrinne, H., Huitema, C., and D. Gurle, “Session Initiation Protocol (SIP) Extension for Instant Messaging,” RFC 3428, December 2002 (TXT). |
| [RFC3665] | Johnston, A., Donovan, S., Sparks, R., Cunningham, C., and K. Summers, “Session Initiation Protocol (SIP) Basic Call Flow Examples,” BCP 75, RFC 3665, December 2003 (TXT). |
| [RFC4780] | Lingle, K., Mule, J-F., Maeng, J., and D. Walker, “Management Information Base for the Session Initiation Protocol (SIP),” RFC 4780, April 2007 (TXT). |
| TOC |
| [RFC2330] | Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, “Framework for IP Performance Metrics,” RFC 2330, May 1998 (TXT, HTML, XML). |
| [RFC3398] | Camarillo, G., Roach, A., Peterson, J., and L. Ong, “Integrated Services Digital Network (ISDN) User Part (ISUP) to Session Initiation Protocol (SIP) Mapping,” RFC 3398, December 2002 (TXT). |
| TOC |
| Daryl Malas | |
| CableLabs | |
| 858 Coal Creek Circle | |
| Louisville, CO 80027 | |
| US | |
| Phone: | +1 303 661 3302 |
| Email: | d.malas@cablelabs.com |
| Al Morton | |
| AT&T Labs | |
| 200 Laurel Avenue South | |
| Middletown, NJ 07748 | |
| US | |
| Phone: | +1 732 420 1571 |
| Email: | acmorton@att.com |