| Network Working Group | M. Cheng |
| Internet-Draft | Y. Bao |
| Intended status: Standards Track | BII Group Holdings Ltd. |
| Expires: June 24, 2017 | December 21, 2016 |
SDN Switch Benchmark Test Cases Summary
draft-cheng-sdn-switchtest-00
This document summarized the test cases for SDN switches, and introduced the test methodologies for each case of them.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 24, 2017.
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
This document listed 11 test cases for SDN switch.
Test Case Name:Flow entry installing time
Test Purpose:Measure how much time does the switch under test test to valid a flow entry.
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple and an action OFPAT_OUTPUT to an output port
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow entry
5. Send a matched TCP packet to the data plane, record this time as Tt
6. Receive the packet at the output port, record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test by a pre-configured number(less than the known maximum flow entry number) of flow entries installed already or the maximum number of flow entries
Test Case Name:Flow table look up time
Test Purpose:Measure how much time does the switch under test take to look up flow table for a matched packet
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple and an action OFPAT_OUTPUT to an output port
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow entry
5. Send a matched TCP packet to the data plane, record this time as Tt
6. Receive the packet at the output port, record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test by a pre-configured number(less than the known maximum flow entry number) of flow entries installed already or the maximum number of flow entries
Test Case Name:Packet in message performance of table-miss
Test Purpose:Measures the time sending packet_in message after a table-miss flow entry is triggered
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of the switch under test
3. Add a table-miss flow entry matching all wildcards, and a default action OFPAT_OUTPUT to port OFPP_CONTROLLER
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow entry
5. Send a packet to data plane, record this time as Tt
6. Receive Packet_in messages and record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test with different datapath packet throughput
Test Case Name:Packet in message performance of none table-miss
Test Purpose:Measures the time sending packet_in message after a normal flow entry is triggered
Test Methodology:
1. Setup connection between switch under test and test tool
2. remove all flow entries of switch
3. add a flow entry to table 0 with matching in_port and an action OFPAT_OUTPUT to port OFPP_CONTROLLER
4. Wait enough time(e.g. 5 seconds) for switch to valid the flow entry
5. send a packet to the matched data plane port, record this time as Tt
6. receive Packet_in messages and record this time as Tr
7. The result time Tresult = Tr - Tt
8. Iterate this test with different datapath packet throughput
Test Case Name:Flow entry deleting time
Test Purpose:Measure how much time does the switch under test take to delete a flow entry.
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple and an action OFPAT_OUTPUT to an output port
4. Keep sending TCP packets to dataplane ingress port
5. Wait enough time(e.g. 3 seconds) for switch until packets are forwarded to the output port
6. Send a flow delete message, delete all flow entries, record this time as Tt
7. wait until no packets received at the output port, record the time of the last one received packet as Tr
8. The result time Tresult = Tr - Tt
9. Iterate this test for 100 flow entries and 1000 flow entries installed already or the maximum number(if it is less than 1000) of flow entries
Test Case Name:Flow entry modifing time
Test Purpose:Measure how much time does the switch under test take to modify a flow entry.
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple and an action OFPAT_OUTPUT to an output port
4. Keep sending TCP packets to dataplane ingress port
5. Wait enough time(e.g. 3 seconds) for switch until packets are forwarded to the output port
6. Send a flow modify message, output the packets to another port, record this time as Tt
7. wait until packets received at the ohter output port, record the time of the first received packet as Tr
8. The result time Tresult = Tr - Tt
9. Iterate this test for 100 flow entries and 1000 flow entries installed already or the maximum number(if it is less than 1000) of flow entries
Test Case Name:Table capacity
Test Purpose:Measure the maximum number of flow entries of a certain table of a switch
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Send an empty body ofp_multipart_request with type OFPMP_TABLE_FEATURES, to get the maximum flow entries of a specified table
4. Keep sending flow_add messages with different priorities, actions can be an output action to an output port
5. Wait until ofp_error_msg is received, with type OFPET_FLOW_MOD_FAILED, and error code OFPFMFC_TABLE_FULL
6. Record the number of flow entries sent before the ofp_error_msg is triggered as the test result
Test Case Name:Packet out message throughput/latency time
Test Purpose:Measure the Packet_out message processing throughput and latency of the switch under test
Test Methodology:
1. Setup connection between switch under test and test tool
2. Send packet_out messages with a certain length of packet in its data field and action OFPAT_OUTPUT to an output port of the switch under test, record the time as Tt
3. Receive the packet contains in the data field, record the time as Tr
4. The result Tresult = Tr - Tt
5. Iterate this test for different Packet_out throughput and data length
Test Case Name:Flow entry multipart_reply message test
Test Purpose:Measure flow statistics update time of a switch
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Add a flow entry to table 0 with a match field of TCP five-tuple and an action OFPAT_OUTPUT to an output port
4. Keep sending an OFPT_MULTIPART_REQUEST message with type of OFPMP_FLOW to request statistics of the flow added in step 3
5. Send a packet matches the flow added in step 3 to switch from dataplane, record the time as Tt
6. Wait enough time(e.g. 3 seconds) for switch until flow statistics are updated, record the time as Tr
7. The result Tresult = Tr - Tt
8. Iterate this test for different flows
Test Case Name:Port multipart_reply message test
Test Purpose:Measure port statistics update time of a switch
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Send an OFPT_MULTIPART_REQUEST message with type of OFPMP_PORT_STATS to request statistics of one port, record the rx_packets and rx_bytes
4. Keep sending the OFPT_MULTIPART_REQUEST message of step 3.
5. Send a packet to switch from dataplane, record the time as Tt
6. Wait enough time(e.g. 3 seconds) for switch until port statistics are updated, record the time as Tr
7. The result Tresult = Tr - Tt
8. Iterate this test for sending different amount and kind of packets to switch
Test Case Name:Idle time out timer performance test
Test Purpose:Test the accuracy of idle timeout of flow entries
Test Methodology:
1. Setup connection between switch under test and test tool
2. Remove all flow entries of switch
3. Install a flow entry with idle timeout 5s and set the OFPFF_SEND_FLOW_REM flag
4. Send a matching packet to ingress port within idle timeout and record the time as Ts
5. Wait until the switch send an OFP_FLOW_REMOVED message to the test tool, record the time as Te
6. The result time, Tresult = Te - Ts
7. Compare Tresult with the hard timeout(5s), the error percentage equals (Tresult - 5s)/5s * 100%
8. Iterate this test with different number of installed flow entries
Funding for the RFC Editor function is currently provided by BII Group.