Internet DRAFT - draft-ietf-bmwg-ca-bench-meth
draft-ietf-bmwg-ca-bench-meth
Internet Engineering Task Force M. Hamilton
Internet-Draft Ixia
Intended status: Informational S. Banks
Expires: August 5, 2013 Aerohive Networks
Feb 2013
Benchmarking Methodology for Content-Aware Network Devices
draft-ietf-bmwg-ca-bench-meth-04
Abstract
This document defines a set of test scenarios and metrics that can be
used to benchmark content-aware network devices. The scenarios in
the following document are intended to more accurately predict the
performance of these devices when subjected to dynamic traffic
patterns. This document will operate within the constraints of the
Benchmarking Working Group charter, namely black box characterization
in a laboratory environment.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 5, 2013.
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
Hamilton & Banks Expires August 5, 2013 [Page 1]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1. Test Considerations . . . . . . . . . . . . . . . . . . . 6
3.2. Clients and Servers . . . . . . . . . . . . . . . . . . . 6
3.3. Traffic Generation Requirements . . . . . . . . . . . . . 6
3.4. Discussion of Network Limitations . . . . . . . . . . . . 6
3.5. Framework for Traffic Specification . . . . . . . . . . . 8
3.6. Multiple Client/Server Testing . . . . . . . . . . . . . . 8
3.7. Device Configuration Considerations . . . . . . . . . . . 8
3.7.1. Network Addressing . . . . . . . . . . . . . . . . . . 9
3.7.2. Network Address Translation . . . . . . . . . . . . . 9
3.7.3. TCP Stack Considerations . . . . . . . . . . . . . . . 9
3.7.4. Other Considerations . . . . . . . . . . . . . . . . . 9
4. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . . . 9
4.1. Maximum Application Session Establishment Rate . . . . . . 10
4.1.1. Objective . . . . . . . . . . . . . . . . . . . . . . 10
4.1.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 10
4.1.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 10
4.1.4. Measurement . . . . . . . . . . . . . . . . . . . . . 10
4.1.4.1. Maximum Application Flow Rate . . . . . . . . . . 10
4.1.4.2. Application Flow Duration . . . . . . . . . . . . 11
4.1.4.3. Application Efficiency . . . . . . . . . . . . . . 11
4.1.4.4. Application Flow Latency . . . . . . . . . . . . . 11
4.2. Application Throughput . . . . . . . . . . . . . . . . . . 11
4.2.1. Objective . . . . . . . . . . . . . . . . . . . . . . 11
4.2.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 11
4.2.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 12
4.2.4. Measurement . . . . . . . . . . . . . . . . . . . . . 12
4.2.4.1. Maximum Throughput . . . . . . . . . . . . . . . . 12
4.2.4.2. Maximum Application Flow Rate . . . . . . . . . . 12
4.2.4.3. Application Flow Duration . . . . . . . . . . . . 12
4.2.4.4. Application Efficiency . . . . . . . . . . . . . . 12
4.2.4.5. Packet Loss . . . . . . . . . . . . . . . . . . . 12
4.2.4.6. Application Flow Latency . . . . . . . . . . . . . 12
4.3. Malformed Traffic Handling . . . . . . . . . . . . . . . . 13
4.3.1. Objective . . . . . . . . . . . . . . . . . . . . . . 13
4.3.2. Setup Parameters . . . . . . . . . . . . . . . . . . . 13
4.3.3. Procedure . . . . . . . . . . . . . . . . . . . . . . 13
4.3.4. Measurement . . . . . . . . . . . . . . . . . . . . . 13
Hamilton & Banks Expires August 5, 2013 [Page 2]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . 13
7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14
7.1. Normative References . . . . . . . . . . . . . . . . . . . 14
7.2. Informative References . . . . . . . . . . . . . . . . . . 15
7.3. URL References . . . . . . . . . . . . . . . . . . . . . . 15
Appendix A. Example Traffic Mix . . . . . . . . . . . . . . . . . 15
Appendix B. Malformed Traffic Algorithm . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19
Hamilton & Banks Expires August 5, 2013 [Page 3]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
1. Introduction
Content-aware and deep packet inspection (DPI) device deployments
have grown significantly in recent years. No longer are devices
simply using Ethernet and IP headers to make forwarding decisions.
This class of device now uses application-specific data to make these
decisions. For example, a web-application firewall (WAF) may use
search criteria upon the HTTP uniform resource indicator (URI)[1] to
decide whether a HTTP GET method may traverse the network. In the
case of lawful/legal intercept technology, a device could use the
phone number within the Session Description Protocol[14] to determine
whether a voice-over-IP phone may be allowed to connect. In addition
to the development of entirely new classes of devices, devices that
could historically be classified as 'stateless' or raw forwarding
devices are now performing DPI functionality. Devices such as core
and edge routers are now being developed with DPI functionality to
make more intelligent routing and forwarding decisions.
The Benchmarking Working Group (BMWG) has historically produced
Internet Drafts and Requests for Comment that are focused
specifically on creating output metrics that are derived from a very
specific and well-defined set of input parameters that are completely
and unequivocally reproducible from test bed to test bed. The end
goal of such methodologies is to, in the words of the RFC 2544 [2],
reduce "specsmanship" in the industry and hold vendors accountable
for performance claims.
The end goal of this methodology is to generate performance metrics
in a lab environment that will closely relate to actual observed
performance on production networks. By utilizing dynamic traffic
patterns relevant to modern networks, this methodology should be able
to closely tie laboratory and production metrics. It should be
further noted than any metrics acquired from production networks
SHOULD be captured according to the policies and procedures of the
IPPM or PMOL working groups.
An explicit non-goal of this document is to replace existing
methodology/terminology pairs such as RFC 2544 [2]/RFC 1242 [3] or
RFC 3511 [4]/RFC 2647 [5]. The explicit goal of this document is to
create a methodology more suited for modern devices while
complementing the data acquired using existing BMWG methodologies.
This document does not assume completely repeatable input stimulus.
The nature of application-driven networks is such that a single
dropped packet inherently changes the input stimulus from a network
perspective. While application flows will be specified in great
detail, it simply is not practical to require totally repeatable
input stimulus.
Hamilton & Banks Expires August 5, 2013 [Page 4]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [6].
2. Scope
Content-aware devices take many forms, shapes and architectures.
These devices are advanced network interconnect devices that inspect
deep into the application payload of network data packets to do
classification. They may be as simple as a firewall that uses
application data inspection for rule set enforcement, or they may
have advanced functionality such as performing protocol decoding and
validation, anti-virus, anti-spam and even application exploit
filtering. The document will universally call these devices
middleboxes, as defined by RFC 3234 [7].
This document is strictly focused on examining performance and
robustness across a focused set of metrics: throughput(min/max/avg/
sample std dev), transaction rates(successful/failed), application
response times, concurrent flows, and unidirectional packet latency.
None of the metrics captured through this methodology are specific to
a device and the results are DUT implementation independent.
Functional testing of the DUT is outside the scope of this
methodology.
Devices such as firewalls, intrusion detection and prevention
devices, wireless LAN controllers, application delivery controllers,
deep packet inspection devices, wide-area network(WAN) optimization
devices, and unified threat management systems generally fall into
the content-aware category. While this list may become obsolete,
these are a subset of devices that fall under this scope of testing.
3. Test Setup
This document will be applicable to most test configurations and will
not be confined to a discussion on specific test configurations.
Since each DUT/SUT will have their own unique configuration, users
SHOULD configure their device with the same parameters that would be
used in the actual deployment of the device or a typical deployment,
if the actual deployment is unknown. A summary of the DUT
configuration MUST be published with the final benchmarking results.
In order to improve repeatability, the published configuration
information SHOULD include command-line scripts used to configure the
DUT, if any, and SHOULD also include any configuration information
Hamilton & Banks Expires August 5, 2013 [Page 5]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
for the test equipment used."
3.1. Test Considerations
3.2. Clients and Servers
Content-aware device testing SHOULD involve multiple clients and
multiple servers. As with RFC 3511 [4], this methodology will use
the terms virtual clients/servers because both the client and server
will be represented by the tester and not actual clients/servers.
Similarly defined in RFC 3511 [4], a data source may emulate multiple
clients and/or servers within the context of the same test scenario.
The test report SHOULD indicate the number of virtual clients/servers
used during the test. IANA has reserved address ranges for
laboratory characterization. These are defined for IPv4 and IPv6 by
RFC 2544 Appendix C [2] and RFC 5180 Section 5.2 [8] respectively and
SHOULD be consulted prior to testing.
3.3. Traffic Generation Requirements
The explicit purposes of content-aware devices vary widely, but these
devices use information deeper inside the application flow to make
decisions and classify traffic. This methodology will utilize
traffic flows that resemble real application traffic without
utilizing captures from live production networks. Application Flows,
as defined in Section 1.1 RFC 2724 [9] are able to be well-defined
without simply referring to a network capture. An example traffic
template is defined and listed in Appendix A of this document. A
user of this methodology is free to utilize the example mix as
provided in the appendix. If a user of this methodology understands
the traffic patterns in their production network, that user MAY use
the template provided in Appendix A to describe a traffic mix
appropriate for their environment. In all cases, users MUST report
the traffic mix used in the test, and SHOULD report this using a
template similar to that in Appendix A.
The test tool SHOULD be able to create application flows between
every client and server, regardless of direction. The tester SHOULD
be able to open TCP connections on multiple destination ports and
SHOULD be able to direct UDP traffic to multiple destination ports.
3.4. Discussion of Network Limitations
Prior to executing the methodology as outlined in the following
sections, it is imperative to understand the implications of
utilizing representative application flows for the traffic content of
the benchmarking effort. One interesting aspect of utilizing
application flows is that each flow is inherently different from
Hamilton & Banks Expires August 5, 2013 [Page 6]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
every other application flow. The content of each flow will vary
from application to application, and in most cases, even varies
within the same type of application flow. The following description
of the methodology will individually benchmark every individual type
and subset of application flow, prior to performing similar tests
with a traffic mix as specified either by the example mix in
Appendix A, or as defined by the user of this methodology.
The purpose of this process is to ensure that any performance
implications that are discovered during the mixed testing aren't due
to the inherent physical network limitations. As an example of this
phenomena, it is useful to examine a network device inserted into a
single path, as illustrated in the following diagram.
+----------+
+---+ 1gE | DUT/ | 1gE +---+
|C/S|------| SUT |------|C/S|
+---+ +----------+ +---+
Simple Inline DUT Configuration
Figure 1: Simple Middle-box Example
For the purpose of this discussion, let's take a hypothetical
application flow that utilizes UDP for the transport layer. Assume
that the sample transaction we will be using to model this particular
flow requires 10 UDP datagrams to complete the transaction. For
simplicity, each datagram within the flow is exactly 64 bytes,
including associated Ethernet, IP, and UDP overhead. With any
network device,there are always three metrics which interact with
each other: number of concurrent application flows, number of
application flows per second, and layer-7 throughput.
Our example test bed is a single-path device connected by 1 gigabit
Ethernet links. The purpose of this benchmark effort is to quantify
the number of application flows per second that may be processed
through our device under test. Let's assume that the result from our
scenario is that the DUT is able to process 10,000 application flows
per second. The question is whether that ceiling is the actual
ceiling of the device, or if it is actually being limited by one of
the other metrics. If we do the appropriate math, 10000 flows per
second, with each flow at 640 total bytes means that we are achieving
an aggregate bitrate of roughly 49 Mbps. This is dramatically less
than the 1 gigabit physical link we are using. We can conclude that
10,000 flows per second is in fact the performance limit of the
device.
If we change the example slightly and increase the size of each
Hamilton & Banks Expires August 5, 2013 [Page 7]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
datagram to 1312 bytes, then it becomes necessary to recompute the
load. Assuming the same observed DUT limitation of 10,000 flows per
second, it must be ensured that this is an artifact of the DUT, and
not of physical limitations. For each flow, we'll require 104,960
bits. 10,000 flows per second implies a throughput of roughly 1 Gbps.
At this point, we cannot definitively answer whether the DUT is
actually limited to 10,000 flows per second. If we are able to
modify the scenario, and utilize 10 Gigabit interfaces, then perhaps
the flow per second ceiling will be reached at a higher number than
10,000.
This example illustrates why a user of this methodology SHOULD
benchmark each application variant individually to ensure that the
cause of a measured limit is fully understood
3.5. Framework for Traffic Specification
The following table SHOULD be specified for each application flow
variant.
o Data Exchanged By Flow, Bits
o Offered Percentage of Total Flows
o Transport Protocol(s)
o Destination Port(s)
3.6. Multiple Client/Server Testing
In actual network deployments, connections are being established
between multiple clients and multiple servers simultaneously. Device
vendors have been known to optimize the operation of their devices
for easily defined patterns. The connection sequence ordering
scenarios a device will see on a network will likely be much less
deterministic. In fact, many application flows have multiple layer 4
connections within a single flow, with client and server reversing
roles. Flow initiation SHOULD be in a pseudo-random manner across
ingress ports.
3.7. Device Configuration Considerations
The configuration of the DUT may have an effect on the observed
results of the following methodology. A comprehensive, but certainly
not exhaustive, list of potential considerations is listed below.
Hamilton & Banks Expires August 5, 2013 [Page 8]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
3.7.1. Network Addressing
The IANA has issued a range of IP addresses to the BMWG for purposes
of benchmarking. Please refer to RFC 2544 [2] and RFC 5180 [8] for
more details. If more IPv4 addresses are required than the RFC 2544
allotment provides, then allocations from the private address space
as defined in RFC 1918 [10] may be used.
3.7.2. Network Address Translation
Many content-aware devices are capable of performing Network Address
Translation (NAT)[5]. If the final deployment of the DUT will have
this functionality enabled, then the DUT SHOULD also have it enabled
during the execution of this methodology. It MAY be beneficial to
perform the test series in both modes in order to determine the
performance differential when using NAT. The test report SHOULD
indicate whether NAT was enabled during the testing process.
3.7.3. TCP Stack Considerations
The IETF has historically provided guidance and information on TCP
stack considerations. This methodology is strictly focused on
performance metrics at layers above 4, thus does not specifically
define any TCP stack configuration parameters of either the tester or
the DUTs. The TCP configuration of the tester MUST remain constant
across all DUTs in order to ensure comparable results. While the
following list of references is not exhaustive, each document
contains a relevant discussion on TCP stack considerations.
The general IETF TCP roadmap is defined in RFC 4614 [11] and
congestion control algorithms are discussed in Section 2 of RFC 3148
[12] with even more detailed references. TCP receive and congestion
window sizes are discussed in detail in RFC 6349 [13].
3.7.4. Other Considerations
Various content-aware devices will have widely varying feature sets.
In the interest of representative test results, the DUT features that
will likely be enabled in the final deployment SHOULD be used. This
methodology is not intended to advise on which features should be
enabled, but to suggest using actual deployment configurations.
4. Benchmarking Tests
Each of the following benchmark scenarios SHOULD be run with each of
the single application flow templates. Upon completion of all
iterations, the mixed test SHOULD be completed, subject to the
Hamilton & Banks Expires August 5, 2013 [Page 9]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
traffic mix as defined by the user.
4.1. Maximum Application Session Establishment Rate
4.1.1. Objective
To determine the maximum rate through which a device is able to
establish and complete application flows as defined by
draft-ietf-bmwg-ca-bench-term-00.
4.1.2. Setup Parameters
The following parameters SHOULD be used and reported for all tests:
For each application protocol in use during the test run, the table
provided in Section 3.5 SHOULD be published.
4.1.3. Procedure
The test SHOULD generate application network traffic that meets the
conditions of Section 3.3. The traffic pattern SHOULD begin with an
application flow rate of 10% of expected maximum. The test SHOULD be
configured to increase the attempt rate in units of 10% up through
110% of expected maximum. In the case where expected maximum is
limited by physical link rate as discovered through Appendix A, the
maximum rate will attempted will be 100% of expected maximum, or
"wire-speed performance". The duration of each loading phase SHOULD
be at least 30 seconds. This test MAY be repeated, each subsequent
iteration beginning at 5% of expected maximum and increasing session
establishment rate to 110% of the maximum observed from the previous
test run.
This procedure MAY be repeated any reasonable number of times with
the results being averaged together.
4.1.4. Measurement
The following metrics MAY be determined from this test, and SHOULD be
observed for each application protocol within the traffic mix:
4.1.4.1. Maximum Application Flow Rate
The test tool SHOULD report the maximum rate at which application
flows were completed, as defined by RFC 2647 [5], Section 3.7. This
rate SHOULD be reported individually for each application protocol
present within the traffic mix.
Hamilton & Banks Expires August 5, 2013 [Page 10]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
4.1.4.2. Application Flow Duration
The test tool SHOULD report the minimum, maximum and average
application duration, as defined by RFC 2647 [5], Section 3.9. This
duration SHOULD be reported individually for each application
protocol present within the traffic mix.
4.1.4.3. Application Efficiency
The test tool SHOULD report the application efficiency, similarly
defined for TCP by RFC 6349 [13].
Transmitted Bytes - Retransmitted Bytes
App Efficiency % = --------------------------------------- X 100
Transmitted Bytes
Figure 2: Application Efficiency Percent Calculation
Note than calculation less than 100% does not necessarily imply
noticeably degraded performance since certain applications utilize
algorithms to maintain a quality user experience in the face of data
loss.
4.1.4.4. Application Flow Latency
The test tool SHOULD report the minimum, maximum and average amount
of time an application flow member takes to traverse the DUT, as
defined by RFC 1242 [3], Section 3.8. This value SHOULD be reported
individually for each application protocol present within the traffic
mix.
4.2. Application Throughput
4.2.1. Objective
To determine the maximum rate through which a device is able to
forward bits when using application flows as defined in the previous
sections.
4.2.2. Setup Parameters
The same parameter reporting procedure as described in Section 4.1.2
SHOULD be used for all tests.
Hamilton & Banks Expires August 5, 2013 [Page 11]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
4.2.3. Procedure
This test will attempt to send application flows through the device
at a flow rate of 30% of the maximum, as observed in Section 4.1.
This procedure MAY be repeated with the results from each iteration
averaged together.
4.2.4. Measurement
The following metrics MAY be determined from this test, and SHOULD be
observed for each application protocol within the traffic mix:
4.2.4.1. Maximum Throughput
The test tool SHOULD report the minimum, maximum and average
application throughput.
4.2.4.2. Maximum Application Flow Rate
The test tool SHOULD report the maximum rate at which application
flows were completed, as defined by RFC 2647 [5], Section 3.7. This
rate SHOULD be reported individually for each application protocol
present within the traffic mix.
4.2.4.3. Application Flow Duration
The test tool SHOULD report the minimum, maximum and average
application duration, as defined by RFC 2647 [5], Section 3.9. This
duration SHOULD be reported individually for each application
protocol present within the traffic mix.
4.2.4.4. Application Efficiency
The test tool SHOULD report the application efficiency as defined in
Section 4.1.4.3.
4.2.4.5. Packet Loss
The test tool SHOULD report the number of packets lost or dropped
from source to destination.
4.2.4.6. Application Flow Latency
The test tool SHOULD report the minimum, maximum and average amount
of time an application flow member takes to traverse the DUT, as
defined by RFC 1242 [3], Section 3.13. This value SHOULD be reported
individually for each application protocol present within the traffic
mix.
Hamilton & Banks Expires August 5, 2013 [Page 12]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
4.3. Malformed Traffic Handling
4.3.1. Objective
To determine the effects on performance and stability that malformed
traffic may have on the DUT.
4.3.2. Setup Parameters
The same parameters SHOULD be used for Transport-Layer and
Application Layer Parameters previously specified in Section 4.1.2
and Section 4.2.2.
4.3.3. Procedure
This test will utilize the procedures specified previously in
Section 4.1.3 and Section 4.2.3. When performing the procedures
listed previously, the tester should generate malformed traffic at
all protocol layers. This is commonly known as fuzzed traffic.
Fuzzing techniques generally modify portions of packets, including
checksum errors, invalid protocol options, and improper protocol
conformance.
The process by which the tester SHOULD generate the malformed traffic
is outlined in detail in Appendix B.
4.3.4. Measurement
For each protocol present in the traffic mix, the metrics specified
by Section 4.1.4 and Section 4.2.4 MAY be determined. This data may
be used to ascertain the effects of fuzzed traffic on the DUT.
5. IANA Considerations
This memo includes no request to IANA.
All drafts are required to have an IANA considerations section (see
the update of RFC 2434 [15] for a guide). If the draft does not
require IANA to do anything, the section contains an explicit
statement that this is the case (as above). If there are no
requirements for IANA, the section will be removed during conversion
into an RFC by the RFC Editor.
6. Security Considerations
Benchmarking activities as described in this memo are limited to
Hamilton & Banks Expires August 5, 2013 [Page 13]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the other constraints
RFC 2544 [2].
The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test
traffic into a production network, or mis-route traffic to the test
management network
7. References
7.1. Normative References
[1] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986,
January 2005.
[2] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999.
[3] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991.
[4] Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
"Benchmarking Methodology for Firewall Performance", RFC 3511,
April 2003.
[5] Newman, D., "Benchmarking Terminology for Firewall
Performance", RFC 2647, August 1999.
[6] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[7] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues",
RFC 3234, February 2002.
[8] Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin,
"IPv6 Benchmarking Methodology for Network Interconnect
Devices", RFC 5180, May 2008.
[9] Handelman, S., Stibler, S., Brownlee, N., and G. Ruth, "RTFM:
New Attributes for Traffic Flow Measurement", RFC 2724,
October 1999.
[10] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and E.
Lear, "Address Allocation for Private Internets", BCP 5,
RFC 1918, February 1996.
Hamilton & Banks Expires August 5, 2013 [Page 14]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
[11] Duke, M., Braden, R., Eddy, W., and E. Blanton, "A Roadmap for
Transmission Control Protocol (TCP) Specification Documents",
RFC 4614, September 2006.
[12] Mathis, M. and M. Allman, "A Framework for Defining Empirical
Bulk Transfer Capacity Metrics", RFC 3148, July 2001.
[13] Constantine, B., Forget, G., Geib, R., and R. Schrage,
"Framework for TCP Throughput Testing", RFC 6349, August 2011.
7.2. Informative References
[14] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[15] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
7.3. URL References
[16] Sandvine Corporation, "http://www.sandvine.com/general/
document.download.asp?docID=58&sourceID=0", 2012.
Appendix A. Example Traffic Mix
This appendix shows an example case of a protocol mix that may be
used with this methodology. This mix closely represents the research
published by Sandvine [16] in their biannual report for the first
half of 2012 on North American fixed access service provider
networks.
+------------+------------------+--------------------+--------+
| Direction | Application Flow | Options | Value |
+------------+------------------+--------------------+--------+
| Upstream | BitTorrent | | |
| | | Avg Flow Size (L7) | 512 MB |
| | | Flow Percentage | 44.4% |
| | HTTP | | |
| | | Avg Flow Size (L7) | 128 kB |
| | | Flow Percentage | 7.3% |
| | Skype | | |
| | | Avg Flow Size (L7) | 8 MB |
| | | Flow Percentage | 4.9% |
| | SSL/TLS | | |
| | | Avg Flow Size (L7) | 128 kB |
| | | Flow Percentage | 3.2% |
| | Netflix | | |
Hamilton & Banks Expires August 5, 2013 [Page 15]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
| | | Avg Flow Size (L7) | 500 kB |
| | | Flow Percentage | 3.1% |
| | PPStream | | |
| | | Avg Flow Size (L7) | 500 MB |
| | | Flow Percentage | 2.2% |
| | YouTube | | |
| | | Avg Flow Size (L7) | 4 MB |
| | | Flow Percentage | 1.9% |
| | Facebook | | |
| | | Avg Flow Size (L7) | 2 MB |
| | | Flow Percentage | 1.9% |
| | Teredo | | |
| | | Avg Flow Size (L7) | 500 MB |
| | | Flow Percentage | 1.2% |
| | Apple iMessage | | |
| | | Avg Flow Size (L7) | 40 kB |
| | | Flow Percentage | 1.1% |
| | Bulk TCP | | |
| | | Avg Flow Size (L7) | 128 kB |
| | | Flow Percentage | 28.8% |
| Downstream | Netflix | | |
| | | Avg Flow Size (L7) | 512 MB |
| | | Flow Percentage | 32.9% |
| | YouTube | | |
| | | Avg Flow Size (L7) | 5 MB |
| | | Flow Percentage | 13.8% |
| | HTTP | | |
| | | Avg Flow Size (L7) | 1 MB |
| | | Flow Percentage | 12.1% |
| | BitTorrent | | |
| | | Avg Flow Size (L7) | 500 MB |
| | | Flow Percentage | 6.3% |
| | iTunes | | |
| | | Avg Flow Size (L7) | 32 MB |
| | | Flow Percentage | 3.8% |
| | Flash Video | | |
| | | Avg Flow Size (L7) | 100 MB |
| | | Flow Percentage | 2.6% |
| | MPEG | | |
| | | Avg Flow Size (L7) | 100 MB |
| | | Flow Percentage | 2.0% |
| | RTMP | | |
| | | Avg Flow Size (L7) | 50 MB |
| | | Flow Percentage | 2.0% |
| | Hulu | | |
| | | Avg Flow Size (L7) | 300 MB |
| | | Flow Percentage | 1.8% |
| | SSL/TLS | | |
Hamilton & Banks Expires August 5, 2013 [Page 16]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
| | | Avg Flow Size (L7) | 256 kB |
| | | Flow Percentage | 1.6% |
| | Bulk TCP | | |
| | | Avg Flow Size (L7) | 500 kB |
| | | Flow Percentage | 21.1% |
+------------+------------------+--------------------+--------+
Table 1: Example Traffic Pattern
Appendix B. Malformed Traffic Algorithm
Each application flow will be broken into multiple transport
segments, IP packets, and Ethernet frames. The malformed traffic
algorithm looks very similar to the IP Stack Integrity Checker
project at http://isic.sourceforge.net.
The algorithm is very simple and starts by defining each of the
fields within the TCP/IP stack that will be malformed during
transmission. The following table illustrates the Ethernet, IPv4,
IPv6, TCP, and UDP fields which are able to be malformed by the
algorithm. The first column lists the protocol, the second column
shows the actual header field name, with the third column showing the
percentage of packets that should have the field modified by the
malformation algorithm.
Hamilton & Banks Expires August 5, 2013 [Page 17]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
+--------------+--------------------------+-------------+
| Protocol | Header Field | Malformed % |
+--------------+--------------------------+-------------+
| Total Frames | | 1% |
| Ethernet | | |
| | Destination MAC | 0% |
| | Source MAC | 1% |
| | Ethertype | 1% |
| | CRC | 1% |
| IP Version 4 | | |
| | Version | 1% |
| | IHL | 1% |
| | Type of Service | 1% |
| | Total Length | 1% |
| | Identification | 1% |
| | Flags | 1% |
| | Fragment Offset | 1% |
| | Time to Live | 1% |
| | Protocol | 1% |
| | Header Checksum | 1% |
| | Source Address | 1% |
| | Destination Address | 1% |
| | Options | 1% |
| | Padding | 1% |
| UDP | | |
| | Source Port | 1% |
| | Destination Port | 1% |
| | Length | 1% |
| | Checksum | 1% |
| TCP | | |
| | Source Port | 1% |
| | Destination Port | 1% |
| | Sequence Number | 1% |
| | Acknowledgement Number | 1% |
| | Data Offset | 1% |
| | Reserved(3 bit) | 1% |
| | Flags(9 bit) | 1% |
| | Window Size | 1% |
| | Checksum | 1% |
| | Urgent Pointer | 1% |
| | Options(Variable Length) | 1% |
+--------------+--------------------------+-------------+
Table 2: Malformed Header Values
This algorithm is to be used across the regular application flows
used throughout the rest of the methodology. As each frame is
emitted from the test tool, a pseudo-random number generator will
Hamilton & Banks Expires August 5, 2013 [Page 18]
Internet-Draft Methodology for Content-Aware Devices Feb 2013
indicate whether the frame is to be malformed by creating a number
between 0 and 100. If the number is less than the percentage defined
in the table, then that frame will be malformed. If the frame is to
be malformed, then each of the headers in the table present within
the frame will follow the same process. If it is determined that a
header field should be malformed, the same pseudo-random number
generator will be used to create a random number for the specified
header field.
Authors' Addresses
Mike Hamilton
Ixia
Austin, TX 78730
US
Phone: +1 512 636 2303
Email: mhamilton@ixiacom.com
Sarah Banks
Aerohive Networks
San Jose, CA 95134
US
Email: sbanks@aerohive.com
Hamilton & Banks Expires August 5, 2013 [Page 19]
