Internet DRAFT - draft-cheng-sdn-switchtest
draft-cheng-sdn-switchtest
Network Working Group M. Cheng
Internet-Draft Y. Bao
Intended status: Standards Track BII Group Holdings Ltd.
Expires: June 24, 2017 December 21, 2016
SDN Switch Benchmark Test Cases Summary
draft-cheng-sdn-switchtest-00
Abstract
This document summarized the test cases for SDN switches, and
introduced the test methodologies for each case of them.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 24, 2017.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Cheng & Bao Expires June 24, 2017 [Page 1]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1. Case1 . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2. Case2 . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3. Case3 . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.4. Case4 . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.5. Case5 . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.6. Case6 . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.7. Case7 . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.8. Case8 . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.9. Case9 . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.10. Case10 . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.11. Case11 . . . . . . . . . . . . . . . . . . . . . . . . . 8
3. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9
1. Introduction
This document listed 11 test cases for SDN switch.
2. Test Cases
2.1. Case1
Test Case Name:Flow entry installing time
Test Purpose:Measure how much time does the switch under test test to
valid a flow entry.
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple
and an action OFPAT_OUTPUT to an output port
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow
entry
5. Send a matched TCP packet to the data plane, record this time as
Tt
6. Receive the packet at the output port, record this time as Tr
Cheng & Bao Expires June 24, 2017 [Page 2]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
7. The result time Tresult = Tr - Tt
8. Iterate this test by a pre-configured number(less than the known
maximum flow entry number) of flow entries installed already or the
maximum number of flow entries
2.2. Case2
Test Case Name:Flow table look up time
Test Purpose:Measure how much time does the switch under test take to
look up flow table for a matched packet
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple
and an action OFPAT_OUTPUT to an output port
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow
entry
5. Send a matched TCP packet to the data plane, record this time as
Tt
6. Receive the packet at the output port, record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test by a pre-configured number(less than the known
maximum flow entry number) of flow entries installed already or the
maximum number of flow entries
2.3. Case3
Test Case Name:Packet in message performance of table-miss
Test Purpose:Measures the time sending packet_in message after a
table-miss flow entry is triggered
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of the switch under test
Cheng & Bao Expires June 24, 2017 [Page 3]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
3. Add a table-miss flow entry matching all wildcards, and a default
action OFPAT_OUTPUT to port OFPP_CONTROLLER
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow
entry
5. Send a packet to data plane, record this time as Tt
6. Receive Packet_in messages and record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test with different datapath packet throughput
2.4. Case4
Test Case Name:Packet in message performance of none table-miss
Test Purpose:Measures the time sending packet_in message after a
normal flow entry is triggered
Test Methodology:
1. Setup connection between switch under test and test tool
2. remove all flow entries of switch
3. add a flow entry to table 0 with matching in_port and an action
OFPAT_OUTPUT to port OFPP_CONTROLLER
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow
entry
5. send a packet to the matched data plane port, record this time as
Tt
6. receive Packet_in messages and record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test with different datapath packet throughput
2.5. Case5
Test Case Name:Flow entry deleting time
Test Purpose:Measure how much time does the switch under test take to
delete a flow entry.
Cheng & Bao Expires June 24, 2017 [Page 4]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple
and an action OFPAT_OUTPUT to an output port
4. Keep sending TCP packets to dataplane ingress port
5. Wait enough time(e.g. 3 seconds) for switch until packets are
forwarded to the output port
6. Send a flow delete message, delete all flow entries, record this
time as Tt
7. wait until no packets received at the output port, record the time
of the last one received packet as Tr
8. The result time Tresult = Tr - Tt
9. Iterate this test for 100 flow entries and 1000 flow entries
installed already or the maximum number(if it is less than 1000) of
flow entries
2.6. Case6
Test Case Name:Flow entry modifing time
Test Purpose:Measure how much time does the switch under test take to
modify a flow entry.
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple
and an action OFPAT_OUTPUT to an output port
4. Keep sending TCP packets to dataplane ingress port
5. Wait enough time(e.g. 3 seconds) for switch until packets are
forwarded to the output port
Cheng & Bao Expires June 24, 2017 [Page 5]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
6. Send a flow modify message, output the packets to another port,
record this time as Tt
7. wait until packets received at the ohter output port, record the
time of the first received packet as Tr
8. The result time Tresult = Tr - Tt
9. Iterate this test for 100 flow entries and 1000 flow entries
installed already or the maximum number(if it is less than 1000) of
flow entries
2.7. Case7
Test Case Name:Table capacity
Test Purpose:Measure the maximum number of flow entries of a certain
table of a switch
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Send an empty body ofp_multipart_request with type
OFPMP_TABLE_FEATURES, to get the maximum flow entries of a specified
table
4. Keep sending flow_add messages with different priorities, actions
can be an output action to an output port
5. Wait until ofp_error_msg is received, with type
OFPET_FLOW_MOD_FAILED, and error code OFPFMFC_TABLE_FULL
6. Record the number of flow entries sent before the ofp_error_msg
is triggered as the test result
2.8. Case8
Test Case Name:Packet out message throughput/latency time
Test Purpose:Measure the Packet_out message processing throughput and
latency of the switch under test
Test Methodology:
1. Setup connection between switch under test and test tool
Cheng & Bao Expires June 24, 2017 [Page 6]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
2. Send packet_out messages with a certain length of packet in its
data field and action OFPAT_OUTPUT to an output port of the switch
under test, record the time as Tt
3. Receive the packet contains in the data field, record the time as
Tr
4. The result Tresult = Tr - Tt
5. Iterate this test for different Packet_out throughput and data
length
2.9. Case9
Test Case Name:Flow entry multipart_reply message test
Test Purpose:Measure flow statistics update time of a switch
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple
and an action OFPAT_OUTPUT to an output port
4. Keep sending an OFPT_MULTIPART_REQUEST message with type of
OFPMP_FLOW to request statistics of the flow added in step 3
5. Send a packet matches the flow added in step 3 to switch from
dataplane, record the time as Tt
6. Wait enough time(e.g. 3 seconds) for switch until flow statistics
are updated, record the time as Tr
7. The result Tresult = Tr - Tt
8. Iterate this test for different flows
2.10. Case10
Test Case Name:Port multipart_reply message test
Test Purpose:Measure port statistics update time of a switch
Test Methodology:
Cheng & Bao Expires June 24, 2017 [Page 7]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Send an OFPT_MULTIPART_REQUEST message with type of
OFPMP_PORT_STATS to request statistics of one port, record the
rx_packets and rx_bytes
4. Keep sending the OFPT_MULTIPART_REQUEST message of step 3.
5. Send a packet to switch from dataplane, record the time as Tt
6. Wait enough time(e.g. 3 seconds) for switch until port statistics
are updated, record the time as Tr
7. The result Tresult = Tr - Tt
8. Iterate this test for sending different amount and kind of
packets to switch
2.11. Case11
Test Case Name:Idle time out timer performance test
Test Purpose:Test the accuracy of idle timeout of flow entries
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Install a flow entry with idle timeout 5s and set the
OFPFF_SEND_FLOW_REM flag
4. Send a matching packet to ingress port within idle timeout and
record the time as Ts
5. Wait until the switch send an OFP_FLOW_REMOVED message to the
test tool, record the time as Te
6. The result time, Tresult = Te - Ts
7. Compare Tresult with the hard timeout(5s), the error percentage
equals (Tresult - 5s)/5s * 100%
8. Iterate this test with different number of installed flow entries
Cheng & Bao Expires June 24, 2017 [Page 8]
Internet-Draft SDN Switch Benchmark Test Cases December 2016
3. Acknowledgements
Funding for the RFC Editor function is currently provided by BII
Group.
Authors' Addresses
Mike Cheng
BII Group Holdings Ltd.
Beijing
P. R. China
Email: mikecheng@biigroup.com
Yaming Bao
BII Group Holdings Ltd.
Beijing
P. R. China
Email: ymbao@biigroup.cn
Cheng & Bao Expires June 24, 2017 [Page 9]
