The Common Log Format (CLF) for the Session Initiation Protocol (SIP)
DOCNAME

Abstract

Well-known web servers such as Apache and web proxies like Squid support event logging using a common log format. The logs produced using these de-facto standard formats are invaluable to system administrators for trouble-shooting a server and tool writers to craft tools that mine the log files and produce reports and trends. Furthermore, these log files can also be used to train anomaly detection systems and feed events into a security event management system. The Session Initiation Protocol does not have a common log format, and as a result, each server supports a distinct log format that makes it unnecessarily complex to produce tools to do trend analysis and security detection. We propose a common log file format for SIP servers that can be used uniformly by proxies, registrars, redirect servers as well as back-to-back user agents.

Status of this Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

This Internet-Draft will expire on April 28, 2011.

Copyright Notice

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1.  Terminology
2.  Introduction
3.  Problem statement
4.  What SIP CLF is and what it is not
5.  Alternative approaches to SIP CLF
    5.1.  SIP CLF and CDRs
    5.2.  SIP CLF and Wireshark packet capture
6.  Motivation and use cases
7.  Challenges in establishing a SIP CLF
8.  Data model
    8.1.  SIP CLF mandatory fields
    8.2.  Mandatory fields and SIP entities
9.  Examples
    9.1.  UAC registration
    9.2.  Direct call between Alice and Bob
    9.3.  Single downstream branch call
    9.4.  Forked call
10.  Security Considerations
11.  Operational guidance
12.  IANA Considerations
13.  Acknowledgments
14.  References
    14.1.  Normative References
    14.2.  Informative References
§  Authors' Addresses

1.  Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.) [RFC2119].

RFC 3261 (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261] defines additional terms used in this document that are specific to the SIP domain such as "proxy"; "registrar"; "redirect server"; "user agent server" or "UAS"; "user agent client" or "UAC"; "back-to-back user agent" or "B2BUA"; "dialog"; "transaction"; "server transaction".

This document uses the term "SIP Server" that is defined to include the following SIP entities: user agent server, registrar, redirect server, a SIP proxy in the role of user agent server, and a B2BUA in the role of a user agent server.

2.  Introduction

Servers executing on Internet hosts produce log records as part of their normal operations. A log record is, in essence, a summary of an application layer protocol data unit (PDU), that captures in precise terms an event that was processed by the server. These log records serve many purposes, including analysis and troubleshooting.

Well-known web servers such as Apache and Squid support event logging using a Common Log Format (CLF), the common structure for logging requests and responses serviced by the web server. It can be argued that a good part of the success of Apache has been its CLF because it allowed third parties to produce tools that analyzed the data and generated traffic reports and trends. The Apache CLF has been so successful that not only did it become the de-facto standard in producing logging data for web servers, but also many commercial web servers can be configured to produce logs in this format. An example of Apache CLF is depicted next:

          %h      %l     %u       %t   \"%r\"   %s    %b
     remotehost rfc931 authuser [date] request status bytes

remotehost:

Remote hostname (or IP number if DNS hostname is not available, or if DNSLookup is Off.

rfc931:

The remote logname of the user.

authuser:

The username by which the user has authenticated himself.

[date]:

Date and time of the request.

request:

The request line exactly as it came from the client.

status:

The HTTP status code returned to the client.

bytes:

The content-length of the document transferred.

The inspiration for the SIP CLF is the Apache CLF. However, the state machinery for a HTTP transaction is much simpler than that of the SIP transaction (as evidenced in Section 7 (Challenges in establishing a SIP CLF)). The SIP CLF needs to do considerably more.

3.  Problem statement

The Session Initiation Protocol (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261](SIP) is an Internet multimedia session signaling protocol that is increasingly used for other services besides session establishment. A typical deployment of SIP in an enterprise will consist of SIP entities from multiple vendors. Currently, if these entities are capable of producing a log file of the transactions being handled by them, the log files are produced in a proprietary format. The result of multiplicity of the log file formats is the inability of the support staff to easily trace a call from one entity to another, or even to craft common tools that will perform trend analysis, debugging and troubleshooting problems uniformly across the SIP entities of multiple vendors.

SIP does not currently have a CLF format and this document serves to provide the rationale to establish a SIP CLF and identifies the required minimal information that must appear in any SIP CLF record.

4.  What SIP CLF is and what it is not

The SIP CLF is a standardized manner of producing a log file. This format can be used by SIP clients, SIP Servers, proxies, and B2BUAs. The SIP CLF is simply an easily digestible log of currently occurring events and past transactions. It contains enough information to allow humans and automata to derive relationships between discrete transactions handled at a SIP entity or to search for a certain dialog or a related set of transactions.

The SIP CLF is amenable to quick parsing (i.e., well-delimited fields) and it is platform and operating system neutral.

The SIP CLF is amenable to easy parsing and lends itself well to creating other innovative tools.

The SIP CLF is not a billing tool. It is not expected that enterprises will bill customers based on SIP CLF. The SIP CLF records events at the signaling layer only and does not attempt to correlate the veracity of these events with the media layer. Thus, it cannot be used to trigger customer billing.

The SIP CLF is not a quality of service (QoS) measurement tool. If QoS is defined as measuring the mean opinion score (MOS) of the received media, then SIP CLF does not aid in this task since it does not summarize events at the media layer.

5.  Alternative approaches to SIP CLF

It is perhaps tempting to consider other approaches --- which though not standardized, are in wide enough use in networks today --- to determine whether or not a SIP CLF would benefit a SIP network consisting of multi-vendor products. The two existing approaches that approximate what SIP CLF does are Call Detail Records (CDRs) and Wireshark packet sniffing.

5.1.  SIP CLF and CDRs

CDRs are used in operator networks widely and with the adoption of SIP, standardization bodies such as 3GPP have subsequently defined SIP-related CDRs as well. Today, CDRs are used to implement the functionality approximated by SIP CLF, however, there are important differences.

One, SIP CLF operates natively at the transaction layer and maintains enough information in the information elements being logged that dialog-related data can be subsequently derived from the transaction logs. Thus, esoteric SIP fields and parameters like the To header, including tags; the From header, including tags, the CSeq number, etc. are logged in SIP CLF. By contrast, a CDR is used mostly for charging and thus saves information to facilitate that very aspect. A CDR will most certainly log the public user identification of a party requesting a service (which may not correspond to the From header) and the public user identification of the party called party (which may not correspond to the To header.) Furthermore, the sequence numbers maintained by the CDR may not correspond to the SIP CSeq header. Thus it will be hard to piece together the state of a dialog through a sequence of CDR records.

Two, a CDR record will, in all probability, be generated at a SIP entity performing some form of proxy-like functionality of a B2BUA providing some service. By contrast, SIP CLF is light- weight enough that it can be generated by a canonical SIP user agent server and user agent client as well, including those that execute on resource constrained devices (mobile phones).

Finally, SIP is also being deployed outside of operator- managed VoIP networks. Universities, research laboratories, and small-to-medium size companies are deploying SIP-based VoIP solutions on networks owned and managed by them. Much of the latter constituencies will not have an interest in generating CDRs, but they will like to have a concise representation of the messages being handled by the SIP entities in a common format.

5.2.  SIP CLF and Wireshark packet capture

Wireshark is a popular raw packet capture tool. It contains filters that can understand SIP at the protocol level and break down a captured message into its individual header components. While Wireshark is appropriate to capture and view discrete SIP messages, it does not suffice to serve in the same capacity as SIP CLF for two reasons.

First, all SIP entities that need to save SIP CLF records would require a Wireshark library for different operating systems and configurations to link into. Second, and more importantly, if the SIP messages are exchanged over a TLS-oriented transport, Wireshark will be unable to decrypt them and render them as individual SIP headers.

6.  Motivation and use cases

As SIP becomes pervasive in multiple business domains and ubiquitous in academic and research environments, it is beneficial to establish a CLF for the following reasons:

Common reference for interpreting events:

In a laboratory environment or an enterprise service offering there will typically be SIP entities from multiple vendors participating in routing requests. Absent a CLF format, each entity will produce output records in a native format making it hard to establish commonality for tools that operate on the log file.

Writing common tools:

A CLF format allows independent tool providers to craft tools and applications that interpret the CLF data to produce insightful trend analysis and detailed traffic reports. The format should be such that it retains the ability to be read by humans and processed using traditional Unix text processing tools.

Session correlation across diverse processing elements:

In operational SIP networks, a request will typically be processed by more than one SIP server. A SIP CLF will allow the network operator to trace the progression of the request (or a set of requests) as they traverse through the different servers to establish a concise diagnostic trail of a SIP session.

Note that tracing the request through a set of servers is considerably less challenging if all the servers belong to the same administrative domain.

Message correlation across transactions:

A SIP CLF can enable a quick lookup of all messages that comprise a transaction (e.g., "Find all messages corresponding to server transaction X, including all forked branches.")

Message correlation across dialogs:

A SIP CLF can correlate transactions that comprise a dialog (e.g., "Find all messages for dialog created by Call-ID C, From tag F and To tag T.")

Trend analysis:

A SIP CLF allows an administrator to collect data and spot patterns or trends in the information (e.g., "What is the domain where the most sessions are routed to between 9:00 AM and 12:00 PM?")

Train anomaly detection systems:

A SIP CLF will allow for the training of anomaly detection systems that once trained can monitor the CLF file to trigger an alarm on the subsequent deviations from accepted patterns in the data set. Currently, anomaly detection systems monitor the network and parse raw packets that comprise a SIP message -- a process that is unsuitable for anomaly detection systems [rieck2008] (Rieck, K., Wahl, S., Laskov, P., Domschitz, P., and K-R. Muller, “A Self-learning System for Detection of Anomalous SIP Messages,” 2008.). With all the necessary event data at their disposal, network operations managers and information technology operation managers are in a much better position to correlate, aggregate, and prioritize log data to maintain situational awareness.

Testing:

A SIP CLF allows for automatic testing of SIP equipment by writing tools that can parse a SIP CLF file to ensure behavior of a device under test.

Troubleshooting:

A SIP CLF can enable cursory trouble shooting of a SIP entity (e.g., "How long did it take to generate a final response for the INVITE associated with Call-ID X?")

Offline analysis:

A SIP CLF allows for offline analysis of the data gathered. Once a SIP CLF file has been generated, it can be transported (subject to the security considerations in Section 10 (Security Considerations)) to a host with appropriate computing resources to perform subsequent analysis.

Real-time monitoring:

A SIP CLF allows administrators to visually notice the events occurring at a SIP entity in real-time providing accurate situational awareness.

7.  Challenges in establishing a SIP CLF

Establishing a CLF for SIP is a challenging task. The behavior of a SIP entity is more complex when compared to the equivalent HTTP entity.

Base protocol services such as parallel or serial forking elicit multiple final responses. Ensuing delays between sending a request and receiving a final response all add complexity when considering what fields should comprise a CLF and in what manner. Furthermore, unlike HTTP, SIP groups multiple discrete transactions into a dialog, and these transactions may arrive at a varying inter-arrival rate at a proxy. For example, the BYE transaction usually arrives much after the corresponding INVITE transaction was received, serviced and expunged from the transaction list. Nonetheless, it is advantageous to relate these transactions such that automata or a human monitoring the log file can construct a set consisting of related transactions.

ACK requests in SIP need careful consideration as well. In SIP, an ACK is a special method that is associated with an INVITE only. It does not require a response, and furthermore, if it is acknowledging a non-2xx response, then the ACK is considered part of the original INVITE transaction. If it is acknowledging a 2xx-class response, then the ACK is a separate transaction consisting of a request only (i.e., there is not a response for an ACK request.) CANCEL is another method that is tied to an INVITE transaction, but unlike ACK, the CANCEL request elicits a final response.

While most requests elicit a response immediately, the INVITE request in SIP can pend at a proxy as it forks branches downstream or at a user agent server while it alerts the user. RFC 3261 (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261] instructs the server transaction to send a 1xx-class provisional response if a final response is delayed for more than 200 ms. A SIP CLF log file needs to include such provisional responses because they help train automata associated with anomaly detection systems and provide some positive feedback for a human observer monitoring the log file.

Finally, beyond supporting native SIP actors such as proxies, registrars, redirect servers, and user agent servers (UAS), it is beneficial to derive a CLF format that supports back-to-back user agent (B2BUA) behavior, which may vary considerably depending on the specific nature of the B2BUA.

8.  Data model

The minimal SIP CLF fields are defined below. Some of these fields contain URIs. If the URI contains an escaped character (""%" HEX HEX" mechanism), the escaped character MUST be logged as received.

OPEN ISSUE: 4K limit for mandatory headers. There has been some discussion in the WG mailing list (see http://www.ietf.org/mail-archive/ web/sip-clf/current/msg00238.html) to have a limit for headers. 4K has been mentioned. What are folks thinking here ... is this limit for URIs only? For all headers that are part of SIP CLF? For each header that is part of SIP CLF? For the entire SIP CLF record?

The SIP CLF record does not contain any provisions for logging bodies, whether or not the body appears as part of a SIP message, separated by a CRLF, or the body appears as the special hname "body" in a SIP URI. In SIP, bodies can be of arbitrary length and composition, resulting in unwieldy and overtly verbose SIP CLF records. If required, certain aspects of the SIP body can be logged as extension elements; however, whether or not to do so depends on the specific logging implementation.

8.1.  SIP CLF mandatory fields

The following SIP CLF fields are defined as minimal information that MUST appear in any SIP CLF record:

Timestamp -

Date and time of the request or response represented as the number of seconds and milliseconds since the Unix epoch.

Message type -

An indicator on whether the SIP message is a request or a response. The allowable values for this field are 'R' (for Request) and 'r' (for response).

Directionality -

An indicator on whether the SIP message is received by the SIP entity or sent by the SIP entity. The allowable values for this field are 's' (for message sent) and 'r' (for message received).

Transport -

The transport over which a SIP message is sent or received. The allowable values for the transport are governed by the "transport" production rule in Section 25.1 of RFC3261 (Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, “SIP: Session Initiation Protocol,” June 2002.) [RFC3261].

Source-address -

The DNS name or IP address of the sender of the SIP message.

Source-port -

The source port number of the sender of the SIP message.

Destination-address -

The DNS name or IP address of the recipient of the SIP message.

Destination-port -

The port number of the recipient of the SIP message.

From -

The From URI. Whilst one may question the value of the From URI in light of RFC4744 (Peterson, J. and C. Jennings, “Enhancements for Authenticated Identity Management in the Session Initiation Protocol (SIP),” August 2006.) [RFC4474], the From URI, nonetheless, imparts some information. For one, the From tag is important and, in the case of a REGISTER request, the From URI can provide information on whether this was a third-party registration or a first-party one. It is not necessary to log any URI parameters.

From-tag -

The tag parameter of the From header.

To -

The To URI. It is not necessary to log any URI parameters.

To-tag -

The tag parameter of the To header. Note that the tag parameter will be absent in the initial request that forms a dialog.

Callid -

The Call-ID.

CSeq-Method -

The method from the CSeq header.

CSeq-Number -

The number from the CSeq header.

R-URI -

The Request-URI, including any URI parameters.

Status -

The SIP response status code.

SIP Proxies may fork, creating several client transactions that correlate to a single server transaction. Responses arriving on these client transactions, or new requests (CANCEL, ACK) sent on the client transaction need log file entries that correlate with a server transaction. Similarly, a B2BUA may create one or more client transactions in response to an incoming request. These transactions will require correlation as well. The last two data model elements provide this correlation.

Server-Txn -

Server transaction identification code - the transaction identifier associated with the server transaction. Implementations can reuse the server transaction identifier (the topmost branch-id of the incoming request, with or without the magic cookie), or they could generate a unique identification string for a server transaction (this identifier needs to be locally unique to the server only.) This identifier is used to correlate ACKs and CANCELs to an INVITE transaction; it is also used to aid in forking as explained later in this section. (See Section 9 (Examples) for usage.)

Client-Txn -

Client transaction identification code - this field is used to associate client transactions with a server transaction for forking proxies or B2BUAs. Upon forking, implementations can reuse the value they inserted into the topmost Via header's branch parameter, or they can generate a unique identification string for the client transaction. (See Section 9 (Examples) for usage.)

Finally, the SIP CLF should be extensible such that future SIP methods, headers and bodies can be represented as well. Besides the mandatory fields listed above, any other SIP header that needs to be logged will appear as an ordered pair of header field name and value.

This data model applies to all SIP entities --- a UAC, UAS, Proxy, a B2BUA, registrar and redirect server. Note that a B2BUA is a degenerate case of a proxy and as such the SIP CLF fields prescribed for a proxy is equally applicable to the B2BUA. Similarly, registrars and redirect servers are a degenerate case of a UAS, and as such the SIP CLF fields prescribed for a UAS is equally applicable to registrars and redirect servers.

The next section specifies the individual SIP CLF data model elements that form a log record for specific instance of a SIP entity. We limit our specification to using the minimum data model elements. It is understood that a SIP CLF record is extensible using extension mechanisms appropriate to the specific representation used to generate the SIP CLF record. This document, however, does not prescribe a specific representation format and it limits the discussion to the mandatory data elements described above.

8.2.  Mandatory fields and SIP entities

Each SIP CLF record MUST consist of all the mandatory data model elements outlined in Section 8.1 (SIP CLF mandatory fields). This document does not specify a representation of a logging format; it is expected that other documents will do so. Each SIP CLF record MUST contain the mandatory elements shown below:

      Timestamp, Message type, Directionality, CSeq-Method,
      CSeq-Number, Transport, R-URI, Destination-address,
      Destination-port, Source-address, Source-port, To,
      To-tag, From, From-tag, Call-ID, Status, Server-Txn,
      Client-Txn

Table 1 summarizes how the mandatory fields are logged by a UAC, UAS, or UAC-half, UAS-half of a SIP proxy and B2BUA. In the table below:

R:

implies that the field is logged when a request is handled by that SIP entity.

r:

implies that the field is logged when a response is handled by that SIP entity.

-:

implies that the field is not applicable to that SIP entity.

SIP CLF fields logged per entity

9.  Examples

The examples use only the mandatory data elements defined in Section 8.1 (SIP CLF mandatory fields). Extension elements are not considered. The examples below use the template defined in Section 8.2 (Mandatory fields and SIP entities) when logging a SIP CLF record. When a given mandatory field is not applicable to a SIP entity as determined by Table 1, we use the horizontal dash ("-") to represent it.

There are five principals in the examples below. They are Alice, the initiator of requests. Alice's user agent uses IPv4 address 198.51.100.1, port 5060. P1 is a proxy that Alice's request traverse on their way to Bob, the recipient of the requests. P1 also acts as a registrar to Alice. P1 uses an IPv4 address of 198.51.100.10, port 5060. Bob has two instances of his user agent running on different hosts. The first instance uses an IPv4 address of 203.0.113.1, port 5060 and the second instance uses an IPv6 address of 2001:db8::9, port 5060. P2 is a proxy responsible for Bob's domain. Table 2 summarizes these addresses.

Principal to IP address asignment

Illustrative examples of SIP CLF follow.

9.1.  UAC registration

Alice sends a registration registrar P1 and receives a 2xx-class response. The register requests causes Alice's UAC to produce a log record shown below. The mandatory data model elements correspond to those listed in Table 1.

     Timestamp: 1275930743.699
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 1
     CSeq-Method: REGISTER
     R-URI: sip:example.com
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:example.com
     To-tag: -
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f81-d4-f6@example.com
     Status: -
     Server-Txn: -
     Client-Txn: c-tr-1

After some time, Alice's UAC will receive a response from the registrar. The response causes Alice's agent to produce a log record shown below. The mandatory data elements correspond to those listed in Table 1.

     Timestamp: 1275930744.100
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 1
     CSeq-Method: REGISTER
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:example.com
     To-tag: reg-1-xtr
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f81-d4-f6@example.com
     Status: 100
     Server-Txn: -
     Client-Txn: c-tr-1

9.2.  Direct call between Alice and Bob

In this example, Alice sends a session initiation request directly to Bob's agent (instance 1.) Bob's agent accepts the session invitation. We first present the SIP CLF logging from Alice's UAC point of view. In line 1, Alice's user agent sends out the INVITE. Shortly, it receives a "180 Ringing" (line 2), followed by a "200 OK" response (line 3). Upon the receipt of the 2xx-class response, Alice's user agent sends out an ACK request (line 4).

     Timestamp: 1275930743.699
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 32
     CSeq-Method: INVITE
     R-URI: sip:bob@bob1.example.net
     Destination-address: 203.0.113.1
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f82-d4-f7@example.com
     Status: -
     Server-Txn: -
     Client-Txn: c-1-xt6

     Timestamp: 1275930745.002
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 32
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.1.100
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b-in6-iu
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f82-d4-f7@example.com
     Status: 180
     Server-Txn: -
     Client-Txn: c-1-xt6

     Timestamp: 1275930746.100
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 32
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.1.100
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b-in6-iu
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f82-d4-f7@example.com
     Status: 200
     Server-Txn: -
     Client-Txn: c-1-xt6

     Timestamp: 1275930746.120
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 32
     CSeq-Method: ACK
     R-URI: sip:bob@bob1.example.net
     Destination-address: 203.0.113.1
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b-in6-iu
     From: sip:alice@example.com
     From-tag: 76yhh
     Call-ID: f82-d4-f7@example.com
     Status: -
     Server-Txn: -
     Client-Txn: c-1-xt6

9.3.  Single downstream branch call

In this example, Alice sends a session invitation request to Bob through proxy P1, which inserts a Record-Route header causing subsequent requests between Alice and Bob to traverse the proxy. The SIP CLF log records correspond to the viewpoint of P1. The line numbers below refer to Figure 1 (Simple proxy-aided call flow)

     Alice             P1             Bob
      +---INV--------->|               |  Line 1
      |                |               |
      |<---------100---+               |  Line 2
      |                |               |
      |                +---INV-------->|  Line 3
      |                |               |
      |                |<--------100---+  Line 4
      |                |               |
      |                |<--------180---+  Line 5
      |                |               |
      |<---------180---+               |  Line 6
      |                |               |
      |                |<--------200---+  Line 7
      |                |               |
      |<---------200---+               |  Line 8
      |                |               |
      +---ACK--------->|               |  Line 9
      |                |               |
      |                |---ACK-------->|  Line 10

1    Timestamp: 1275930743.699
     Message Type: R
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: sip:bob@example.net
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: -
     Server-Txn: s-x-tr
     Client-Txn: -

Note that at this point P1 has created a server transaction identification code and populated the SIP CLF field Server-Txn with it. P1 has not yet created a client transaction identification code, thus Client-Txn contains a "-".

2    Timestamp: 1275930744.001
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 100
     Server-Txn: s-x-tr
     Client-Txn: -

In line 3 below, P1 has created a client transaction identification code for the downstream branch and populated the SIP CLF field Client-Txn.

3    Timestamp: 1275930744.998
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: sip:bob@bob1.example.net
     Destination-address: 203.0.113.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: -
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

4    Timestamp: 1275930745.200
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 100
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

5    Timestamp: 1275930745.800
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 180
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

6    Timestamp: 1275930746.009
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 180
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

7    Timestamp: 1275930747.120
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 200
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

8    Timestamp: 1275930747.300
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: 200
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

9    Timestamp: 1275930749.100
     Message Type: R
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: ACK
     R-URI: sip:bob@example.net
     Destination-address: 198.51.100.10
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: -
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

10   Timestamp: 1275930749.100
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: ACK
     R-URI: sip:bob@bob1.example.net
     Destination-address: 203.0.113.1
     Destination-port: 5060
     Source-address: 198.51.100.10
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: al-1
     Call-ID: tr-87h@example.com
     Status: -
     Server-Txn: s-x-tr
     Client-Txn: c-x-tr

9.4.  Forked call

In this example, Alice sends a session invitation to Bob's proxy, P2. P2 forks the session invitation request to two registered endpoints corresponding to Bob's address-of-record. Both endpoints respond with provisional responses. Shortly thereafter, one of Bob's user agent instances accepts the call, causing P2 to send a CANCEL request to the second user agent. P2 does not Record-Route, therefore the subsequent ACK request from Alice to Bob's user agent does not traverse through P2 (and is not shown below.)

Figure 2 (Forked call flow) depicts the call flow.

                        Bob            Bob
     Alice      P2   (Instance 1) (Instance 2)
      +---INV--->|          |         |  Line 1
      |          |          |         |
      |<---100---+          |         |  Line 2
      |          |          |         |
      |          +---INV--->|         |  Line 3
      |          |          |         |
      |          +---INV----+-------->|  Line 4
      |          |          |         |
      |          |<---100---+         |  Line 5
      |          |          |         |
      |          |<---------+---100---+  Line 6
      |          |          |         |
      |          |<---180---+---------+  Line 7
      |          |          |         |
      |<---180---+          |         |  Line 8
      |          |          |         |
      |          |<---180---+         |  Line 9
      |          |          |         |
      |<---180---+          |         |  Line 10
      |          |          |         |
      |          |<---200---+         |  Line 11
      |          |          |         |
      |<---200---+          |         |  Line 12
      |          |          |         |
      |          +---CANCEL-+-------->|  Line 13
      |          |          |         |
      |          |<---------+---487---+  Line 14
      |          |          |         |
      |          +---ACK----+-------->|  Line 15
      |          |          |         |
      |          |<---------+---200---+  Line 16

The SIP CLF log correspond to the viewpoint of P2. The fields logged are shown below; the line numbers refer to Figure 2 (Forked call flow).

1    Timestamp: 1275930743.699
     Message Type: R
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: sip:bob@example.net
     Destination-address: 203.0.113.200
     Destination-port: 5060
     Source-address: 198.51.100.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: -
     Server-Txn: s-1-tr
     Client-Txn: -

2    Timestamp: 1275930744.001
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 100
     Server-Txn: s-1-tr
     Client-Txn: -

3    Timestamp: 1275930744.998
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: sip:bob@bob1.example.net
     Destination-address: 203.0.113.1
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: -
     Server-Txn: s-1-tr
     Client-Txn: c-1-tr

4    Timestamp: 1275930745.500
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: sip:bob@bob2.example.net
     Destination-address: [2001:db8::9]
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: -
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: -
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

5    Timestamp: 1275930745.800
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1=-1
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com 100
     Status: 100
     Server-Txn: s-1-tr
     Client-Txn: c-1-tr

6    Timestamp: 1275930746.100
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: udp
     Source-address: [2001:db8::9]
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 100
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

7    Timestamp: 1275930746.700
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: udp
     Source-address: [2001:db8::9]
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 180
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

8    Timestamp: 1275930746.990
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 180
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

9    Timestamp: 1275930747.100
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com 100
     Status: 180
     Server-Txn: s-1-tr
     Client-Txn: c-1-tr

10   Timestamp: 1275930747.300
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 180
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

11   Timestamp: 1275930747.800
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: 5060
     Source-address: 203.0.113.1
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com 100
     Status: 200
     Server-Txn: s-1-tr
     Client-Txn: c-1-tr

12   Timestamp: 1275930748.000
     Message Type: r
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 198.51.100.1
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b1-1
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 200
     Server-Txn: s-1-tr
     Client-Txn: c-1-tr

13   Timestamp: 1275930748.201
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: CANCEL
     R-URI: sip:bob@bob2.example.net
     Destination-address: [2001:db8::9]
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: -
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

14   Timestamp: 1275930748.991
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: INVITE
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: udp
     Source-address: [2001:db8::9]
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 487
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

15   Timestamp: 1275930749.455
     Message Type: R
     Directionality: s
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: ACK
     R-URI: sip:bob@bob2.example.net
     Destination-address: [2001:db8::9]
     Destination-port: 5060
     Source-address: 203.0.113.200
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: -
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

16   Timestamp: 1275930750.001
     Message Type: r
     Directionality: r
     Transport: udp
     CSeq-Number: 43
     CSeq-Method: CANCEL
     R-URI: -
     Destination-address: 203.0.113.200
     Destination-port: udp
     Source-address: [2001:db8::9]
     Source-port: 5060
     To: sip:bob@example.net
     To-tag: b2-2
     From: sip:alice@example.com
     From-tag: a1-1
     Call-ID: tr-88h@example.com
     Status: 200
     Server-Txn: s-1-tr
     Client-Txn: c-2-tr

The above SIP CLF log makes it easy to search for a specific transaction or a state of the session. Searching for the string "c-1-tr" on the log records will readily yield the information that an INVITE was sent to sip:bob@bob1.example.com, it elicited a 100 followed by a 180 and then a 200. The absence of the ACK request signifies that the ACK was exchanged end-to-end.

Searching on "c-2-tr" yields a more complex scenario of sending an INVITE to sip:bob@bob2.example.net, receiving 100 and 180. However, the log makes it apparent that the request to sip:bob@bob2.example.net was subsequently CANCEL'ed before a final response was generated, and that the pending INVITE returned a 487. The ACK to the final non-2xx response and a 200 to the CANCEL request complete the exchange on that branch.

10.  Security Considerations

A log file by its nature reveals both the state of the entity producing it and the nature of the information being logged. To the extent that this state should not be publicly accessible and that the information is to be considered private, appropriate file and directory permissions attached to the log file should be used. The following threats may be considered for the log file while it is stored:

• An attacker may gain access to view the log file, or may surreptitiously make a copy of the log file for later viewing;
• An attacker may mount a replay attack by modifying existing records in the log file or inserting new records;
• An attacker may delete parts of --- or indeed, the whole --- file.

It is outside the scope of this document to specify how to protect the log file while it is stored on disk. However, operators may consider using common administrative features such as disk encryption and securing log files [schneier‑1] (Schneier, B. and J. Kelsey, “Secure audit logs to support computer forensics,” May 1999.). Operators may also consider hardening the machine on which the log files are stored by restricting physical access to the host as well as restricting access to the files themselves.

In the worst case, public access to the SIP log file provides the same information that an adversary can gain using network sniffing tools (assuming that the SIP traffic is in clear text.) If all SIP traffic on a network segment is encrypted, then as noted above, special attention must be directed to the file and directory permissions associated with the log file to preserve privacy such that only a privileged user can access the contents of the log file.

Transporting SIP CLF files across the network pose special challenges as well. The following threats may be considered for transferring log files or while transferring individual log records:

• An attacker may view the records;
• An attacker may modify the records in transit or insert previously captured records into the stream;
• An attacker may remove records in transit, or may stage a man- in-the-middle attack to deliver a partially or entirely falsified log file.

It is also outside the scope of this document to specify protection methods for log files or log records that are being transferred between hosts. However, operators may consider using common security protocols described in [RFC3552] (Rescorla, E. and B. Korver, “Guidelines for Writing RFC Text on Security Considerations,” July 2003.) to transfer log files or individual records. Alternatively, the log file may be transferred through bulk methods that also guarantees integrity, or at least detects and alerts to modification attempts.

The SIP CLF represents the minimum fields that lend themselves to trend analysis and serve as information that may be deemed useful. Other formats can be defined that include more headers (and the body) from Section 8.1 (SIP CLF mandatory fields). However, where to draw a judicial line regarding the inclusion of non-mandatory headers can be challenging. Clearly, the more information a SIP entity logs, the longer time the logging process will take, the more disk space the log entry will consume, and the more potentially sensitive information could be breached. Therefore, adequate tradeoffs should be taken in account when logging more fields than the ones recommended in Section 8.1 (SIP CLF mandatory fields).

Implementers need to pay particular attention to buffer handling when reading or writing log files. SIP CLF entries can be unbounded in length. It would be reasonable for a full dump of a SIP message to be thousands of octets long. This is of particular importance to CLF log parsers, as a SIP CLF log writers may add one or more extension fields to the message to be logged.

11.  Operational guidance

SIP CLF log files will take up substantive amount of disk space depending on traffic volume at a processing entity and the amount of information being logged. As such, any enterprise using SIP CLF should establish operational procedures for file rollovers as appropriate to the needs of the organization.

Listing such operational guidelines in this document is out of scope for this work.

12.  IANA Considerations

This document does not require any considerations from IANA.

13.  Acknowledgments

Members of the sipping, dispatch, ipfix and syslog working groups provided invaluable input to the formulation of the draft. These include Benoit Claise, Spencer Dawkins, John Elwell, David Harrington, Christer Holmberg, Hadriel Kaplan, Atsushi Kobayashi, Jiri Kuthan, Scott Lawrence, Chris Lonvick, Simon Perreault, Adam Roach, Dan Romascanu, Robert Sparks, Brian Trammell, Dale Worley, Theo Zourzouvillys and others that we have undoubtedly, but inadvertently, missed.

Rainer Gerhards, David Harrington, Cullen Jennings and Gonzalo Salgueiro helped tremendously in discussions related to arriving at the beginnings of a data model.

14.  References

14.1. Normative References

14.2. Informative References

Authors' Addresses