Internet DRAFT - draft-ou-cdn-vm-migration-scheme
draft-ou-cdn-vm-migration-scheme
Network Working Group Liang. Ou
Internet-Draft Huamin. Jin
Intended status: Informational China Telecom
Expires: April 16, 2013 M. Chen
Huawei Technologies Co., Ltd
October 13, 2012
Content Delivery Network Based Virtual Machine Migration Scheme
draft-ou-cdn-vm-migration-scheme-00
Abstract
Virtual machine migration is a critical requirement in today's data
centers to support services dynamically deployed in distributed
infrastructures. To keep the uniform environment required by
application software, large number of networking based solutions have
been presented to provide data center interconnection for virtual
resource management. Aiming at simplifying networking complexities
in off-site service deployment, this document presents a framework
based on content delivery network for supporting virtual machine
migration over the legacy wide area network.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 16, 2013.
Copyright Notice
Ou, et al. Expires April 16, 2013 [Page 1]
Internet-Draft CDN based VM Migration scheme October 2012
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. CDN based Migration Architecture . . . . . . . . . . . . . . . 4
3.1. Functional Architecture . . . . . . . . . . . . . . . . . 4
3.2. Protocols . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3. Mapping Tables in RMG . . . . . . . . . . . . . . . . . . 7
4. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.1. Batch Migration . . . . . . . . . . . . . . . . . . . . . 8
4.2. Multi-nodes Backup . . . . . . . . . . . . . . . . . . . . 9
4.3. Virtual Desktop Migration . . . . . . . . . . . . . . . . 11
5. Implementation Consideration . . . . . . . . . . . . . . . . . 11
5.1. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2. Experimental Work . . . . . . . . . . . . . . . . . . . . 12
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12
7. Security Considerations . . . . . . . . . . . . . . . . . . . 12
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12
9. Normative References . . . . . . . . . . . . . . . . . . . . . 12
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Ou, et al. Expires April 16, 2013 [Page 2]
Internet-Draft CDN based VM Migration scheme October 2012
1. Introduction
In an infrastructure-as-a-service (IaaS) system, IT infrastructure is
deployed in a provider's data center as virtual machine (VM).
Managing a VM's full life-cycle mainly includes setting up networks
dynamically for groups of VMs and their storage requirements, such as
VM disk image deployment or on-the-fly software environment creation.
With IaaS cloud growing popularity, VM migration technology has been
widely used in off-site database backup, new services deployment and
server system maintenance or upgrade.
Basically, VM migration is categorized into two types: live-migration
(real-time) and off-line migration (non real-time). The former
migration procedure is typically controlled by VM manager, which
involves real-time status synchronization, including networks, memory
and storage, between two hosts where VMs locate. Due to strict
requirement to network performance, most former methods (application-
level or process-level) are only limited within a data center instead
of data center interconnection (DCI) scope to prevent poor migration
performance or system exception. Not migrating VM in run-time, the
later VM migration can be implemented by sending VM image files
(static disk image and dynamic snapshot) from source host to
destination. That is to say, file-level data transfer between hosts
is efficient enough for non real-time VM migration when network
quality is favorable. It also means dependence on expensive and
sophisticated DCI solutions is largely reduced. As a fast file
delivery and cache system, content delivery network (CDN) provides
not only high performance data transfer channel but also easy-to-go
control plane. Thus, DCI solution should take CDN based scheme into
consideration for VM migration.
This document only focus on overall framework, scenarios and
implementation consideration for CDN based VM migration. Completed
comparisons between VM migration approaches and related networking
issues are outside the scope of this document. Motivation behind
this document can be concluded as follows:.
1.Provide a simple transition scheme before evolving to future's cost
efficient DCI solution in large scale, and give a simple option to
those who don't want to take risk of complex DCI in current phrase.
2.Protect investment on legacy infrastructure. Typically, for some
Asia telecom carries, who not only own broadband network and data
center infrastructure but also a national wide CDN. Comparing CDN
system upgrade, it is a big burden to construct a new DCI specified
data plane over underlying broadband network in the near future.
Ou, et al. Expires April 16, 2013 [Page 3]
Internet-Draft CDN based VM Migration scheme October 2012
2. Terminology
Virtual Resource Node (VRN): Data centers in which VM migration
happens are abstracted as VRN. VRN consists of host, storage and
distributed file system in a data center. Before migrating from one
to another, virtual resource corresponding to VM is stored in the
format of VM image files in VRN, and then is delivered from source
VRN to destination VRN across network.
Cloud Virtual Machine Manager (CVMM): Different from typical virtual
machine management (VMM) located in data center internally, CVMM is
an infrastructure manager that deploys virtualized services on both
local pool of resources and external IaaS cloud to support virtual
resource life-cycle management.
CDN Management Node (CMN): CMN provides content management for cache
node in CDN network, including content adding, content routing,
content delivery, resource discovery function. Only one CMN in a CDN
system.
CDN Cache Node (CCN): CCN carries VM images for migration received
from source VRN, and deliver those images to destination VRN via its
high speed data channel provided by CDN interconnected network which
is typically constructed over legacy L3 network. Any cache node in
legacy CDN system could be a CCN which could receive/deliver image
files from/to VRN.Thus, the CCN is also partitioned into source CCN
and destination CCN.
Resource Migration Gateway (RMG): RMG is a control system that
supports virtual resources in a cloud to migrate across wide area
network via CDN system. Its interface with CVMM parses migration
instruction and reports related migration status. While the
interface with CMN is used to apply for data channel and cache space
from CDN for arriving VM images. RMG also finishes some conversion
or mapping operation, including content /image format, content/image
ID, content location and content routing, between CDN system and VM
file system.
3. CDN based Migration Architecture
3.1. Functional Architecture
Ou, et al. Expires April 16, 2013 [Page 4]
Internet-Draft CDN based VM Migration scheme October 2012
+-----------+ +------+ +------------+
Cloud | VRN (SRC) |<------>| CVMM |<------>| VRN (DEST) |
Layer +-----------+ +------+ +------------+
| | |
| +------+ |
.............|..............| RMG |..............|..........
| +------+ |
| | |
CDN +-----------+ +------+ +-----------+
Layer | CCN (SRC) |<------>| CMN |<------>| CCN (DEST)|
+-----------+ +------+ +-----------+
.............|....................................|.........
| |
Network +-----------------------------------------------+
Layer | Physical Network |
+-----------------------------------------------+
RMG - Resource Migration Gateway CCN - CDN Cache Node
VRN - Virtual Resource Node CMN - CDN Management Node
CVMM - Cloud Virtual Machine Manager
Figure 1 CDN Based VM Migration Architecture
The functional architecture (as depicted in the Figure 1) describes
how RMG, cloud manager and CDN management element collaborate on
fulfilling VM migration via CDN system which set up over legacy
physical L2/L3 network. Here, one key role of the RMG is to decouple
the signaling dependency between cloud system and CDN system. In
terms of mechanism outlined below, the VM images are able to be
transferred from upper cloud layer (SRC VRN) to CDN and network
layer, and then back to the cloud layer (DEST VRN). The typical
workflow is summarized as following steps:
1. CVMM receives VM migration instruction in two ways: manually
requested by cloud administrator or automatically triggered by user
applications. In either case, CVMM records current virtual resource
status and clones necessary disk image from the physical machine that
the VM is hosted.
2. CVMM identifies this migration operation belong to local or
remote. If it is remote migration, CVMM sends a migration request
which also contains source and destination information about VRNs.
3. RMG checks its internal VRN-CCN mapping table to get information
about the CCN directly connected with SRC or DEST VRN. Then, RMG
sends request to CMN with dedicated protocol which includes SRC and
DEST nodes information about CCN and size or identification of VM
Ou, et al. Expires April 16, 2013 [Page 5]
Internet-Draft CDN based VM Migration scheme October 2012
image.
4. Receiving message from RMG, CMN first requires whether the DEST
CCN or network is busy or not. If the DEST is congested, CMN will
suggest RMG to modify the destination for VRN. Otherwise, it will
check its internal database to judge if CDN system has cached such
image file before. If there is an existing image copy, CMN will push
the image file to DEST CCN, and then informs RMG to receive the image
from the VRN connected.
5. If there is no VM image in CDN system at current phrase, CMN
thereafter responds the RMG's requesting for image transport. And
RMG send permitting message for VM migration to CVMM as well.
Finally, CVMM informs VRNs (SRC and DEST) to start up its file
transfer process and keep status for both. CVMM, RMG and CMN
maintain necessary dialog during images transfer phrase to refresh
status messages.
6. Once the DEST VRN successfully receives image file from DEST CCN,
new VM is created in the DEST VRN's internal physical host.
3.2. Protocols
+--------+ C-R +-------+ R-C +-------+
| CVMM |<--------->| RMG |<--------->| CMN |
+--------+ Protocol +-------+ Protocol +-------+
Negotiation Protocol between CVMM, RMG and CMN
+--------+ V-C +-------+
| VRN |<--------->| CCN |
+--------+ Protocol +-------+
Transport Protocol between VRN and CCN
Figure 2 Reference Protocols
There are two type protocols in suggest CDN based VM migration scheme
(see Figure 2): control protocol between CVMM,RMG and CMN, and data
transfer protocol between VRN and CCN.
1. C-R Protocol
TBD
2. R-C Protocol
TBD
Ou, et al. Expires April 16, 2013 [Page 6]
Internet-Draft CDN based VM Migration scheme October 2012
3. V-C Protocol
TBD
3.3. Mapping Tables in RMG
+-------+--------+--------+--------+--------+-------+--------+-------+
|CCN ID | CCN | VRN ID | VRN |HOST ID | VM ID |IMAGE ID| Status|
| |IP addr.| |IP addr.| | | | |
+-------+--------+--------+--------+--------+-------+--------+-------+
| | | | | | | | |
+-------+--------+--------+--------+--------+-------+--------+-------+
| SH 1# |10.3.1.2| DC 2# |1.1.3.2 | west1 | Xen-10|Linux-2 | SRC |
+-------+--------+--------+--------+--------+-------+--------+-------+
| BJ 2# |3.3.1.2 | DC 1# |1.1.3.2 | west1 | kvm-07|Win2k-1 | DEST |
+-------+--------+--------+--------+--------+-------+--------+-------+
Tabel 1: CDN-VRN Content Mapping Table
+-------+--------+-------+--------+--------+--------+-------+--------+
| SRC |SRC CCN | SRC |SRC VRN| DEST |DEST CCN| DEST |DEST VRN|
|CCN ID |IP addr.| VRN ID|IP addr.| CCN ID |IP addr.|VRN ID |IP addr.|
+-------+--------+-------+--------+--------+--------+-------+--------+
| | | | | | | | |
+-------+--------+-------+--------+--------+--------+-------+--------+
| | | | | | | | |
+-------+--------+-------+--------+--------+--------+-------+--------+
Tabel 2: CDN-VRN Location Mapping Table
Figure 3 Mapping Tables in RMG
Figure 3 illustrates mapping tables stored in RMG to reflect how to
assign VRN's VM images to CCN cache system
1. CDN-VRN Content Mapping Table
CCN ID: the identification of CCN supporting VMs migration, directly
connecting to a VRN with VRN ID
CCN IP addr.: the IP address of CCN with CCN ID
VRN ID: the identification of VRN initiating VMs migration,
connecting to a CCN with CCN ID
VRN IP addr.: the IP address of a concrete CCN with VRN ID
Host ID: the identification of host initiating VM migration in a VRN
Ou, et al. Expires April 16, 2013 [Page 7]
Internet-Draft CDN based VM Migration scheme October 2012
with VRN ID
VM ID: the identification of VM to be migrated in a host with HOST ID
Image ID: the identification of image file describing a VM with VM ID
Status: source or destination node
Others: TBD
2. CDN-VRN Location Mapping Table
This table shows how to record the source and destination binding
relation between VRN-CCN pairs in VM migration duration.
4. Use Cases
4.1. Batch Migration
City A City B
+-----------+ +------+ +------------+
Cloud | Src VRN |<------>| CVMM |<------>| Dest VRN |
| (*v1,*v2) | | | |(*v1',*v2') |
Layer +-----------+ +------+ +------------+
| | ^
|(*v1',*v2') +------+ |(*v1',*v2')
.............|..............| RMG |..............|..........
| +------+ |
V | |
CDN +-----------+ +------+ +-----------+
Layer | Src CCN |<------>| CMN |<------>| Dest CCN |
|(*v1',*v2')| | | |(*v1',*v2')|
+-----------+ +------+ +-----------+
| ^
.............|....................................|.........
V |
Network +-----+-----------------------------------------+
| | | |
| +------------------>-----------------+ |
Layer | Physical Network |
+-----------------------------------------------+
(*v1,*v2): Two Original VM image "v1" and "v2" in a host
(*v1',*v2''): VM image copied from images "v1"and "v2"
Figure 4 Batch migrating VM Images from City A to City B
Ou, et al. Expires April 16, 2013 [Page 8]
Internet-Draft CDN based VM Migration scheme October 2012
Typically, a service deployed in a data center is hosted in different
servers or VMs which might belong to different administrative domain.
And these virtual or physical resources are orchestrated by cloud
manager. Hence all requirements shown in this scenario for service
migration are able to be abstracted at the granularity of VMs.
Specifically speaking, it is a batch process for multiple VM images
transfer via high speed network.
This scenario demonstrates how to implement batch migration for VMs
images when dedicated service moves from one data center to another
geographically separated in two cities. As depicted in Figure 4,
image data "v1"and "v2"are cloned in the Src VRN as "v1'"and "v2'".
Image copies and then are fast send to Dest VRN over CDN system in
which the high speed links for cache nodes are pre-constructed and
carefully optimized over physical network. Obviously, VM image
transfer over CDN system is better than VRN to VRN connected network
in quality and bandwidth.
Most steps for negotiation protocol and data transfer are the same as
workflows mentioned in Section 3.1.
6. Once the DEST VRN successfully receives image file from DEST CCN,
new VM is created in the DEST VRN's internal physical host.
4.2. Multi-nodes Backup
Ou, et al. Expires April 16, 2013 [Page 9]
Internet-Draft CDN based VM Migration scheme October 2012
+-------------+ City B City A City C
| +-------+ | +------------+ +-------------+ +------------+
| | CVMM | | | Dest VRN 1 | | Src VRN | | Dest VRN 2 |
| +-------+ |--->| (*v') | | (*v) | | (*v') |
| | | +------------+ +-------------+ +------------+
| +-------+ | ^ | ^
| | RMG | | | (*v') | (*v') |(*v')
| +-------+ | | V |
| | | +------------+ +-------------+ +------------+
| +-------+ | | Dest CCN 1 | | Src CCN | | Dest CCN 2 |
| | CMN | |--->| (*v') | | (*v') | | (*v') |
| +-------+ | +------------+ +-------------+ +------------+
| Management | ^ | | ^
| Platform | |(*v') (*v')| |(*v') |(*v')
+-------------+ | | | |
+-----+----------------+-----+---------------+------+
| | | | | |
| +-------<--------+ +--------->-----+ |
| Physical IP Network |
+---------------------------------------------------+
(*v): Original VM image to be migrated in a host
(*v'):VM image copied from (*v)
Figure 5 VM Image Delivered to Multiple VRNs
(Off-site Backup Service)
This case illustrates an off-site disaster backup service - that is,
how to simultaneously deliver VM image to multiple data centers in
different cities.
All major steps for control protocol are mentioned in Section 3.1.
According to steps in Section 3.1, the management platform will
negotiate mutually to discover VRN-CCN pairs matched in terms of
mapping table in the RMG before a real migration operation occurs.
Once the VRN-CCN pairs are determined, as shown in Figure 5, Src VRN
which locates in city A will receive backup instruction from CVMM.
It clones the coresponding VM image as "v'" and transfer it to Src
CCN connected directly.
Even if the image "v'" should be sent to two destination nodes, Src
VRN just send "v'" to Src CCN one time only. Because Src CCN
"understands" this migration operation with a fork destination.
Therefore, the "v' " image will be duplicated inside Src CCN and
delivered to corresponding Dest CCNs along with the routes pre-
configured by CDN system. Finally, Dest VRN fetches the same image
"v'" respectively from Dest CCNs connected. The more backup nodes in
the backup services, the higher transfer efficiency will be gained
from CDN based migration scheme mentioned in this document.
Ou, et al. Expires April 16, 2013 [Page 10]
Internet-Draft CDN based VM Migration scheme October 2012
4.3. Virtual Desktop Migration
This section illustrates a experience enhancement application for
virtual desktop user, which migrates a remote virtual desktop (VM
instance) to a local host existing in local CCN.
TBD
5. Implementation Consideration
5.1. Discussion
1. Status Synchronization
The effort to implement this scheme mainly exists in control
protocols between CVMM and RMG, RMG-CMN, and necessary modification
on CVMM and CMN system. Such modification may include revising
status synchronization mechanism in VMM and re-defining content
discovery flow in CMN. Since the scheme we presented is an off-line
oriented solution, modification work for migration operations in a
CVMM are kept in application level rather than revising hypervisor
level.
2. Live-migration or offline-migration
One may argue that the scheme mentioned in this document is only
suitable for offline-migration cases in which active VMs must be shut
down by cloud manager before migration initiates. In view of poor
performance in WAN environment and complexity for DCI technologies,
offline-migration across WAN is acceptable choice. Considering
existence of diverse migration levels, such as progress-level or
memory-level, a revised approach for this CDN based scheme is to use
live-migration technologies for real status synchronization as
supplement. While those static big image files are still transferred
in term of our scheme.
3. Routing, Addressing and Content Discovery
This scheme just maintains a simple table to configure static mapping
between content and routing manually. It cannot deal with dynamic
adding/removing either CDN nodes or VRNs. Moreover, maintaining two
resource systems information and related location/routing
information, RMG is easy to become the bottleneck in the whole
system.
4. Engineering issues
Ou, et al. Expires April 16, 2013 [Page 11]
Internet-Draft CDN based VM Migration scheme October 2012
This scheme is only suitable for those hosting service providers or
carriers who retain their own CDN to improve utilization of CDN
system and to avoid large scale investment on DCI devices at current
phrase.
Another issue is that not all data centers are close enough to CDN
cache nodes. Thus, connections between data centers and CDN node
will become a burden for operators
5. Others
TBD
5.2. Experimental Work
TBD (Part of initial experiment work have been done to validate
mechanism of the scheme, which is based on open source software for
CDN and cloud management )
6. IANA Considerations
This document makes no request of IANA.
Note to RFC Editor: this section may be removed on publication as an
RFC.
7. Security Considerations
TBD
8. Acknowledgements
9. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Ou, et al. Expires April 16, 2013 [Page 12]
Internet-Draft CDN based VM Migration scheme October 2012
Authors' Addresses
Liang Ou
China Telecom
109, Zhongshan Ave. West
Guangzhou,
China
Email: oul@gsta.com
Huamin Jin
China Telecom
109, Zhongshan Ave. West
Guangzhou,
China
Email: jhm@gsta.com
Mach(Guoyi) Chen
Huawei Technologies Co., Ltd
No. 3 Xinxi Road, Shang-di, Hai-dian District
Beijing 100085
China
Email: mach@huawei.com
Ou, et al. Expires April 16, 2013 [Page 13]
