Internet DRAFT - draft-liu-dclc-gap-virtual-test
draft-liu-dclc-gap-virtual-test
Network Working Group Vic Liu
Internet Draft Lingli Deng
Intended status: Informational Dapeng Liu
China Mobile
Expires: January 2015 July 5, 2014
Gap Analysis on Virtualized Network Test
draft-liu-dclc-gap-virtual-test-00.txt
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. This document may not be modified,
and derivative works of it may not be created, and it may not be
published except as an Internet-Draft.
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. This document may not be modified,
and derivative works of it may not be created, except to publish it
as an RFC and to translate it into languages other than English.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
Vic Liu Expires January 5, 2015 [Page 1]
Internet-Draft PS for VxLAN Performance July 2014
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on January 5, 2009.
Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document.
Abstract
In virtual network, the VSwitch become an important element in the
network. It is in charge for both forward management and virtual
network function. We figure out that to evaluate the performance of
VSwitch is not like the test on hardware (like using RFC2544). This
draft introduce the gap of testing for virtual network performance.
On chapter 2 we introduce the test practices which is the virtual
test bad setup and benchmark test. On chapter three, we analysis the
gap of virtual network performance test.
Vic Liu Expires January 5, 2015 [Page 2]
Internet-Draft PS for VxLAN Performance July 2014
Table of Contents
1. Introduction ................................................ 3
2. Virtual network test......................................... 3
2.1. Test Bed Setup ......................................... 3
2.2. Benchmark test on virtualized network .................. 6
3. Gap analysis for virtual network test ....................... 7
3.1. Test methodology ....................................... 7
3.2. Throughput ............................................. 7
3.3. Latency ................................................ 8
3.4. CPU .................................................... 8
3.5. Number of test.......................................... 9
4. Security Considerations..................................... 10
5. IANA Considerations ........................................ 10
6. Conclusions ................................................ 10
7. References ................................................. 10
7.1. Normative References................................... 10
7.2. Informative References................................. 10
8. Acknowledgments ............................................ 10
1. Introduction
In virtual network, the VSwitch become an important element in the
network. It is in charge for both forward management and virtual
network function. We figure out that to evaluate the performance of
VSwitch is not like the test on hardware (like using RFC2544). This
draft introduce the gap of testing for virtual network performance.
On chapter 2 we introduce the test practices which is the virtual
test bad setup and benchmark test. On chapter three, we analysis the
gap of virtual network performance test.
2. Virtual network test practices
2.1. Test Bed Setup
The test bed is constituted by two physical server with 10GE NIC, a
test center, a 10GE TOR switch for test traffic and a 1GE TOR switch
for management.
Vic Liu Expires January 5, 2015 [Page 3]
Internet-Draft PS for VxLAN Performance July 2014
----------------------
|Test Center PHY 10GE*2|
----------------------
||
||
----------
=====| 10GE TOR |=======
|| ---------- ||
|| ||
|| ||
------------------- -------------------
| -------------- | | -------------- |
| |V-switch(VTEP)| | | |V-switch(VTEP)| |
| -------------- | | -------------- |
| | | | | | | |
| ----- ----- | | ----- ----- |
| |TCVM1| |TCVM2|| | |TCVM1| |TCVM2||
| ----- ----- | | ----- ----- |
------------------- -------------------
Server1 Server2
Vic Liu Expires January 5, 2015 [Page 4]
Internet-Draft PS for VxLAN Performance July 2014
Two Dell server are R710XD (CPU: E5-2460) and R710 (CPU: E5-2430)
with a pair of 10GE NIC. And in the server we allocate 2 vCPU and 8G
memory to each Test Center Virtual Machine (TCVM).
In traffic model A: We use a physical test center connect to each
server to verify the benchmark of each server.
----------------------
|Test Center PHY 10GE*2|
----------------------
||
||
-------------------
| -------------- |
| |V-switch(VTEP)| |
| -------------- |
| | | |
| ----- ----- |
| |TCVM1| |TCVM2||
| ----- ----- |
-------------------
Server1
In traffic model B: We use the benchmark to test the performance of
VxLAN.
----------
=====| 10GE TOR |=======
|| ---------- ||
|| ||
|| ||
------------------- -------------------
| -------------- | | -------------- |
| |V-switch(VTEP)| | | |V-switch(VTEP)| |
| -------------- | | -------------- |
| | | | | | | |
| ----- ----- | | ----- ----- |
| |TCVM1| |TCVM2|| | |TCVM1| |TCVM2||
| ----- ----- | | ----- ----- |
------------------- -------------------
Server1 Server2
Vic Liu Expires January 5, 2015 [Page 5]
Internet-Draft PS for VxLAN Performance July 2014
2.2. Benchmark test on virtualized network
The reason we need a benchmark test is we realized that the
virtualized network is different from physical network device. We
cannot use test methodology like RFC 2544. The performance is not
linear growth with traffic we generate. It has an inflection point.
We allocate the Test Center Virtual Machine (TCVM) with one v CPU
and 4G memory to get the benchmark, we use traffic model A and get
the result table below:
Server 1: CPU E5-2430
------------------------------------------------------------
| Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem|
------------------------------------------------------------
| 0 | 0 | 495 | 2817 | 370 | 495 |
------------------------------------------------------------
| 128 | 0.40 | 4704 | 2817 | 4541 | 495 |
------------------------------------------------------------
| 256 | 0.66 | 4830 | 2824 | 4519 | 495 |
------------------------------------------------------------
| 512 | 1.43 | 5161 | 2818 | 4870 | 495 |
------------------------------------------------------------
| 1024| 2.62 | 5131 | 2819 | 4782 | 495 |
------------------------------------------------------------
| 1518| 3.66 | 4957 | 2820 | 4585 | 495 |
------------------------------------------------------------
Server 2: CPU E5-2620
------------------------------------------------------------
| Byte| Rate(GE)| Server CPU MHZ |Server Mem| VM CPU| VM Mem|
------------------------------------------------------------
| 0 | 0 | 188 | 2712 | 59 | 493 |
------------------------------------------------------------
| 128 | 0.46 | 4648 | 2703 | 4100 | 494 |
------------------------------------------------------------
| 256 | 0.70 | 4552 | 2704 | 4077 | 493 |
------------------------------------------------------------
| 512 | 1.50 | 4521 | 2703 | 3958 | 493 |
------------------------------------------------------------
| 1024| 3.10 | 4559 | 2702 | 3938 | 493 |
------------------------------------------------------------
| 1518| 5.46 | 5195 | 2704 | 4377 | 493 |
------------------------------------------------------------
Vic Liu Expires January 5, 2015 [Page 6]
Internet-Draft PS for VxLAN Performance July 2014
3. Gap analysis for virtual network test
3.1. Test methodology
There is a huge difference between testing on hardware network and
virtual network. On virtual network, the Device under Test (DUT) is
the VSwitch or hypervisor (or the protocol such as VxLAN
encapsulation by VSwitch) while the test machine which is a VM
connect to the DUT shares the same resources of the physical server.
In virtual network, tester and the DUT (visual switches) are in one
server (physically converged), so the CPU and MEM share the same
resources. Theoretically, the tester's operation may has some
influences on the DUT's performances. However, for the specialty of
virtualization, this method is the only way to assess the truth of
assessment method.
3.2. Throughput
The throughput generate to test is also different from hardware
switch (for example, RFC 2544). We realize that the throughput is
very hard to reach the line rate by the TCVM. There is an inflection
point as the CPU over loaded. For example, if we generate 1Gb
traffic, it will 100% receive. But if we generate 10Gb traffic, it
can only generate 530Mb because CPU is overloaded.
Besides, the generate traffic is related to the packet length. For
example, 128bit can only generate 0.4Gb traffic and 1518bit can
generate 4Gb traffic. The test concept on traditional physical
switches is not apply to virtual switch test. In traditional
throughput, the capability of the switch can reach line rate at any
bytes while virtual network cannot.
Under the background of existing technology, when we mean to test
the virtual switch's throughput, the concept of traditional physical
switch will not be applicable. The traditional throughput indicates
the switches' largest transmit capability, for certain selected
bytes and selected cycle under zero-packet-lose conditions. But in
virtual environment, the fluctuant of performance on virtual network
will be much greater than dedicated physical devices. In the same
time, because the DUT and the tester cannot be separated, which only
proved the DUT realize same network performances under certain
circumstances, it also means the DUT may achieve higher capability.
Therefore, we change the throughout in virtual environment to actual
throughput, hoping in future, as the improvement of technique, the
actual throughput will approach the theoretical throughput
gradually.
Vic Liu Expires January 5, 2015 [Page 7]
Internet-Draft PS for VxLAN Performance July 2014
Of course, under actual condition, this throughout have certain
referential meanings. In most cases, common throughput application
cannot compare with professional tester, so for virtual application
and data center's deployment, the actual throughout already have
great refinance value.
3.3. Latency
Physical tester's time reference from its own clock or other time
source, such as GPS, which can achieve the accuracy of 10ns. In
virtual network circumstances, the virtual tester gets its reference
time from Linux systems. But the clock on Linux of different server
or VM can't synchronized accuracy due to current method. Although VM
of some higher versions of CentOS or Fedora can achieve the accuracy
of 1ms, if the network can provide better NTP connections, the
result will be better.
In the future, we may consider some other ways to have a better
synchronization of the time to improve the accuracy of the test.
3.4. CPU
The operation of DUT (VSwitch) can increase the CPU load of host
server. While we find that the performance of the VSwitch and tester
measure the index below:
a. CPU type
We test the VSwitch performance based on two servers which CPU are
Intel E-2430 and Intel E-2620.
At 1518 bytes, E-2620 can throughput 5.46Gb/s while E-2430 is
3.66Gb/s. And the E-2620 consume 5195 Mhz CPU and E2430 is 4957
Mhz. The better type of CPU can generate more traffic with almost
same CPU consumption.
b.vCPU allocation for tester
Because of the tester is a VM, the allocation of vCPU for the tester
may affect the performance. Besides, the vCPU can be allocated for
VM like tester, but it can't allocated for the DUT (VSwitch). The
reason is the VSwitch is established in the hypervisor which is a
globe supervisory control for all the resource in the host server.
Vic Liu Expires January 5, 2015 [Page 8]
Internet-Draft PS for VxLAN Performance July 2014
The table below show the throughput performance of different vCPU
allocation on one server.
----------------------------------
| Byte| 1*vCPU | 2*vCPU | 8*vCPU |
----------------------------------
| 0 | 0 | 0 | 0 |
----------------------------------
| 128 | 0.40 | 0.46 | 0.46 |
----------------------------------
| 256 | 0.66 | 0.84 | 0.84 |
----------------------------------
| 512 | 1.43 | 1.56 | 1.56 |
----------------------------------
| 1024| 2.62 | 2.88 | 2.88 |
----------------------------------
| 1518| 3.66 | 4.00 | 4.00 |
----------------------------------
3.5. Number of test port
We need test port to establish test flow. The test port is locate as
a vNIC on the tester. As we know the tester is a VM which has
resource limitation. We realized that we can add more vNIC or tester
VM to measure the test performance.
The table below show the performance of different test case:
------------------------------------------------------------------
| Byte| 1*n 1*t | 2*n 1*t | 4*n 1*t | 1*n 2*t | 2*n 2*t | 4*n 2*t |
------------------------------------------------------------------
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
------------------------------------------------------------------
| 128 | 0.46 | 0.65 | 0.65 | 0.93 | 1.01 | 0.97 |
------------------------------------------------------------------
| 256 | 0.84 | 1.03 | 1.01 | 1.47 | 1.68 | 1.51 |
------------------------------------------------------------------
| 512 | 1.56 | 2.16 | 2.16 | 2.77 | 3.68 | 3.08 |
------------------------------------------------------------------
| 1024| 2.88 | 4.22 | 4.24 | 4.93 | 6.67 | 6.18 |
------------------------------------------------------------------
| 1518| 4.00 | 6.22 | 6.08 | 6.84 | 9.45 | 9.53 |
------------------------------------------------------------------
Vic Liu Expires January 5, 2015 [Page 9]
Internet-Draft PS for VxLAN Performance July 2014
In the table, the n represent for number of vNIC and t represent for number of tester VM. There are 6 test case show that the throughput of one tester VM with 1/2/4 vNICs and two tester VM with 1/2/4 vNICs. We can see that as the number of test port increase the throughput also increased. This will affect the test result of DUT.
4. Security Considerations
5. IANA Considerations
6. Conclusions
7. References
7.1. Normative References
[1] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[2] Crocker, D. and Overell, P.(Editors), "Augmented BNF for
Syntax Specifications: ABNF", RFC 2234, Internet Mail
Consortium and Demon Internet Ltd., November 1997.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for
Syntax Specifications: ABNF", RFC 2234, Internet Mail
Consortium and Demon Internet Ltd., November 1997.
7.2. Informative References
[3] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in TCP
and Its Effect on Busy Servers", Proc. Infocom 1999 pp. 1573-
1583.
[Fab1999] Faber, T., Touch, J. and W. Yue, "The TIME-WAIT state in
TCP and Its Effect on Busy Servers", Proc. Infocom 1999
pp. 1573-1583.
8. Acknowledgments
Vic Liu Expires January 5, 2015 [Page 10]
Internet-Draft PS for VxLAN Performance July 2014
This document was prepared using 2-Word-v2.0.template.dot.
Authors' Addresses
Vic Liu
China Mobile
32 Xuanwumen West Ave, Beijing, China
Email: liuzhiheng@chinamobile.com
Lingli Deng
China Mobile
32 Xuanwumen West Ave, Beijing, China
Email: denglingli@chinamobile.com
Dapeng Liu
China Mobile
32 Xuanwumen West Ave, Beijing, China
Email: liudapeng@chinamobile.com
Vic Liu Expires January 5, 2015 [Page 11]