Internet DRAFT - draft-gu-statemigration-arch
draft-gu-statemigration-arch
Network Working Group Y. Gu
Internet-Draft Huawei
Intended status: Standards Track M. Shore
Expires: August 23, 2013 No Mountain Software
S. Sivakumar
Cisco Systems
D. Zhang
Huawei Technologies
February 19, 2013
An Architecture for Middlebox State Migration
draft-gu-statemigration-arch-01
Abstract
This draft use several motivation use cases to indicate the
importance of the state migration work. An architecture and
components of a solution is given conceiving the use cases, together
with the interfaces and data models that are required in the
architecture.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 23, 2013.
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
Gu, et al. Expires August 23, 2013 [Page 1]
Internet-Draft State Migration Architecture February 2013
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology and concepts . . . . . . . . . . . . . . . . . . . 3
3. VM migration in a virtual data center network . . . . . . . . 5
3.1. Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1.1. Description . . . . . . . . . . . . . . . . . . . . . 5
3.2. Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2.1. Description . . . . . . . . . . . . . . . . . . . . . 6
4. Architecture and Components . . . . . . . . . . . . . . . . . 8
5. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 9
6. Message Flow . . . . . . . . . . . . . . . . . . . . . . . . . 10
7. Security Considerations . . . . . . . . . . . . . . . . . . . 11
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.1. Normative Reference . . . . . . . . . . . . . . . . . . . 12
8.2. Informative Reference . . . . . . . . . . . . . . . . . . 12
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Gu, et al. Expires August 23, 2013 [Page 2]
Internet-Draft State Migration Architecture February 2013
1. Introduction
It has been introduced in [framework] that an end-to-end network flow
typically traverses one or more stateful "middlebox," such as
firewalls, NATs, and TCP or traffic optimizers. In order to process
the packets in the flow correctly, the middleboxes need to establish
and maintain associated state for the flow. In some cases (for
example, VM migration or load balancing), the path of a flow though
the network may change, and packets of that flow may be processed by
new middleboxes. The new middlebox may not be able to process the
packets properly if they do not have the associated state information
as instantiated in the original middlebox.
This draft first introduces several network scenarios demonstrating
state migration problems and then describes interfaces, components,
and messages to address the issues raised in those scenarios.
2. Terminology and concepts
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
The document uses terms defined in [framework].
VN: A virtual network, or "VN," is a network that consists of
virtualized links, based on virtualized interfaces. It may exist in
network layer 2 (L2) or network layer 3 (L3).
VNI: Virtual Network Instance. This is one instance of a virtual
overlay network. We assume that two VNs are isolated from one
another and may use overlapping addresses.
NVE: Network Virtualization Edge. It is a network entity that sits
on the edge of an NVO3 network. It implements network virtualization
functions that allow for L2 and/or L3 tenant separation and for
hiding tenant addressing information (MAC and IP addresses). An NVE
could be implemented as part of a virtual switch within a hypervisor,
a physical switch or router, a Network Service Appliance or even be
embedded within an End Station.
Virtual Network Context or VN Context: Field that is part of the
overlay encapsulation header which allows the encapsulated frame to
be delivered to the appropriate virtual network endpoint by the
egress NVE. The egress NVE uses this field to determine the
appropriate virtual network context in which to process the packet.
This field MAY be an explicit, unique (to the administrative domain)
Gu, et al. Expires August 23, 2013 [Page 3]
Internet-Draft State Migration Architecture February 2013
virtual network identifier (VNID) or MAY express the necessary
context information in other ways (e.g. a locally significant
identifier).
VNID: Virtual Network Identifier. In the case where the VN context
has global significance, this is the ID value that is carried in each
data packet in the overlay encapsulation that identifies the Virtual
Network the packet belongs to.
Underlay or Underlying Network: This is the network that provides the
connectivity between NVEs. The Underlying Network can be completely
unaware of the overlay packets. Addresses within the Underlying
Network are also referred to as "outer addresses" because they exist
in the outer encapsulation. The Underlying Network can use a
completely different protocol (and address family) from that of the
overlay.
Data Center (DC): A physical complex housing physical servers,
network switches and routers, Network Service Appliances and
networked storage. The purpose of a Data Center is to provide
application and/or compute and/or storage services. One such service
is virtualized data center services, also known as Infrastructure as
a Service.
VM: Virtual Machine. Several Virtual Machines can share the
resources of a single physical computer server using the services of
a Hypervisor (see below definition).
Hypervisor: Server virtualization software running on a physical
compute server that hosts Virtual Machines. The hypervisor provides
shared compute/memory/storage and network connectivity to the VMs
that it hosts. Hypervisors often embed a Virtual Switch (see below).
Virtual Switch: A function within a Hypervisor (typically implemented
in software) that provides similar services to a physical Ethernet
switch. It switches Ethernet frames between VMs' virtual NICs within
the same physical server, or between a VM and a physical NIC card
connecting the server to a physical Ethernet switch. It also
enforces network isolation between VMs that should not communicate
with each other.
Tenant: A customer who consumes virtualized data center services
offered by a cloud service provider. A single tenant may consume one
or more Virtual Data Centers hosted by the same cloud service
provider.
Tenant End System: It defines an end system of a particular tenant,
which can be, for example, a virtual machine (VM), a non-virtualized
Gu, et al. Expires August 23, 2013 [Page 4]
Internet-Draft State Migration Architecture February 2013
server, or a physical appliance.
Virtual Access Points (VAPs): Tenant End Systems are connected to the
Tenant Instance through Virtual Access Points (VAPs). The VAPs can
be in reality physical ports on a ToR or virtual ports identified
through logical interface identifiers (VLANs, internal VSwitch
Interface ID leading to a VM).
VN Name: A globally unique name for a VN. The VN Name is not carried
in data packets originating from End Stations, but must be mapped
into an appropriate VN-ID for a particular encapsulating technology.
Using VN Names rather than VN-IDs to identify VNs in configuration
files and control protocols increases the portability of a VDC and
its associated VNs when moving among different administrative domains
(e.g. switching to a different cloud service provider).
VSI: Virtual Station Interface. Typically, a VSI is a virtual NIC
connected directly with a VM. [bridging]
3. VM migration in a virtual data center network
In this section we look at middlebox flow-associated state migration
during VM migration, using two different but related network
scenarios for demonstration purposes.
3.1. Scenario 1
3.1.1. Description
As illustrated in Figure 1, there is a virtual data center network
supporting multiple tenants. Three servers, Server1, Server2 and
Server3 are connected to the network with three virtual network edge
devices, VNE1, NVE2, and NVE3 respectively. The packets transported
between NVE1 and NVE3 are processed by a stateful firewall FW1 while
the packets transported between NVE2 and NVE3 are processed by a
stateful firewall FW2. The security policies deployed on FW1 and FW2
are identical. However, they are deployed far away from each other,
and there is no state synchronization between them at run time. This
condition is typical when NVE1 and NVE2 are located in different data
centers which have firewalls deployed at the core layer or the
convergence layer.
The virtual machines, VM1 and VM3, belongs to the virtual network
instance VNI1, and are located within Server 1 and Server 3
respectively. VM1 and VM3 communicate using TCP.
Pre-conditions:
Gu, et al. Expires August 23, 2013 [Page 5]
Internet-Draft State Migration Architecture February 2013
VM1 moves from Server1 to Server2.
Post-conditions:
TCP flows between servers VM1 and VM3 must not be disrupted.
Requirements:
The state information associated with VM1 on the hypervisor running
on Server1 MUST be forwarded to the hypervisor on Server2.
NVE2 MUST distribute the mapping information regarding flows
associated with VM1 to other NVEs.
The state associated with the TCP sessions between VM1 and VM3 MUST
be conveyed to FW2 so that it can process traffic between VM1 and VM3
using the same rules instantiated on FW1. Otherwise, FW2 will treat
packets based on local policy, which may include discarding the
packets.
3.2. Scenario 2
3.2.1. Description
The network topology in this case is identical to scenario 1.
Pre-conditions:
The path between NVE1 and NVE3, which traverses FW1, is broken.
Post-conditions:
The communication between VM1 and VM3 are transported through NVE2.
The communication between VM1 and VM3 using TCP transport must not be
disrupted.
Requirements:
Flow-coupled state associated with TCP sessions between VM1 and VM3
MUST be conveyed to FW2
+-------------------------+
| +-------+ |
| | VM3 | Server 3 |
| +---+---+ |
Gu, et al. Expires August 23, 2013 [Page 6]
Internet-Draft State Migration Architecture February 2013
| | |
| +-----+---------------+ |
| | Hypervisor | |
| +---------------------+ |
+-------------+-----------+
|
+--------+---------+
+-----+ NVE3 +-----------+
| +------------------+ |
| |
| |
| |
| |
| |
| ----------------- |
|/ ----- ----- \\ |
////| \\\|\
/// | | \\\
/// | | \\\
| +------|-------+ +--------+------+|
|| | FW1 | | | ||
| | | | FW2 | |
| +-----+--------+ +----+----------+ |
| | | |
| | | |
|| | Network | ||
| | | |
\\\ | | ///
\\\ | | ///
\\\|\ |/////
| \\\----- -----//|
| ----------------- |
| |
+---------------+--+ +--+---------------+
| NVE1 +---------------------------+ NVE2 |
+--------+---------+ +--------+---------+
| |
+-----------+-------------+ +-----------+-------------+
| +---------------------+ | | +---------------------+ |
| | Hypervisor | | | | Hypervisor | |
| +----+----------------+ | | +----+----------------+ |
| | Server 1 | | | Server 2 |
| +---+---+ | VM Migration | +---+---+ |
| | VM1 |--------------+--------------------+->| VM1 | |
| +-------+ | | +-------+ |
+-------------------------+ +-------------------------+
Gu, et al. Expires August 23, 2013 [Page 7]
Internet-Draft State Migration Architecture February 2013
Figure 1. A simple example of a virtual data center
4. Architecture and Components
This section defines the architecture of the proposed solution which
can fulfill the state migration requirements described in the
scenarios in Section 3.
State Host: The entity where the flow-coupled state information is
generated and restored. A state Host can be a physical Firewall, a
Virtual Firewall, a NAT device, a IPS/IDS device, etc.
State Orchestration Agent (SOA): an application runs on a State Host
and collaborates with the State Orchestration Master to perform state
migration.
State Orchestration Master (SOM): A centralized entity which
coordinates the state migration between State Hosts. In order to
accomplish its work, a SOM
o needs to receive notification of a VM migration,
o identify the destination and source State Hosts, and then
o trigger the state migration under the assistance of the associated
SOAs.
The origin of these notification is not designated in this draft.
Topology Discovery Entity (TDE): The entity which is used to collect
the topology information of network, including the location of State
Hosts and VMs. In this solution, it is the job of a TDE to monitor
the change of the network topology caused by e.g., VM Migration and
then find out the destination State Host and source State Host
according to its knowledge of network topology. How a TDE creates
the topology information of a network and how it notices any change
of the topology is out of the scope of this draft. There are many
ways to achieve it in current network deployment.
Gu, et al. Expires August 23, 2013 [Page 8]
Internet-Draft State Migration Architecture February 2013
-----------
| SOM |
----------- ========
************/\************ | TDE |
* __________/ \__________ * =====^==
* / Could be indirect link \ * |
**************************** ~~~~~~~~~~
/ \ |Topology|
--------------- -------------- ~~~~~~~~~~
| ------- | | ------- |
| | SOA | | |? SOA | |
| ------- | | ------- |
| State Host | | State Host|
--------------- --------------
| |
*****************************************
* *
* *
* Could be any kind of network topology *
* *
* *
*****************************************
| |
| |
-------- --------
| VM1 | | VM1' |
-------- --------
Figure 1: State Migration Archittecture
Note that the connections between a State Host and a VM can vary.
The use cases introduced above can fit into this architecture. This
architecture maps well to a NVO3 Data Center network or a traditional
Data Center network. In a virtual firewall case, the connection
betwen a VM and a State Host is usually within a Hypervisor, and the
TDE can be the Hypervisor itself, which obviously has the view of the
topology of virtualized network attached to it.
5. Interfaces
Based on the architecture, there are several interfaces to define.
The interfaces and the functionality of each interface are defined
below.
o IF(M,T):
Gu, et al. Expires August 23, 2013 [Page 9]
Internet-Draft State Migration Architecture February 2013
* F1: State Migration Notification. If a TDE detects any network
topology changes caused by a VM Migration, it notifies the SOM
about the changes. The messages for state migration
notification should includes the identities of the destination
and source State Hosts, and the identity of the migrated VM.
The identity of a State Host can be an IP Address, a system
name or anything that can uniquely reprsent the State Host
within the network. Similarly, the identity of a VM can be
anything that State Host can uniquely identify the VM wthin the
network (e.g., an IP address or an VNID defined in NVO3
documents).
o IF(M,A) :
* F1: SOA authentication and authorization. when a SOA starts
running on State Host, the SOA first sends an authentication
and authorization request to a SOM. In the request, SMA should
inform the SOM of its identity, its certification, the protocol
versions it suports and network addresses being used. If the
authentication succeeds and policy permits it, the SOA is
authorized to participate in state migration.
* F2: State Host Registration. Once a SOA is authenticated by
the SOM, the SOA collects the profile of the State Host on
which it is running and registers the State Host on the SOM.
The profile of the State Host includes the identity of the
State Host, the representation of state on this specific State
Host, functions allowed on this State Host, etc.
* F3: State Migration Indication. While state migration is
required, SOM will send a request to the corresponding SOA to
upload or download the state which needs to be migrated.
------- ----------- =======
| SOA |<---IF(M,A)--->| SOM |<---IF(M,T)--->| TDE |
------- ----------- =======
Figure 2: Interfaces
6. Message Flow
Using scenario 1 as an example, the following message flow shows a
possible message flow.
Gu, et al. Expires August 23, 2013 [Page 10]
Internet-Draft State Migration Architecture February 2013
SOA-Src SOM TDM SOA-Dst
|<---------A&A------>|<--Authorization&----->|
Authentication (A&A)
|--State Host Reg--->|<--State Host Reg------|
==================================================================
VM is
Migrated~~ |<-Notify---|
|<--Upload Request---|
|---Upload State---->|
|--Get ready to-------->|
Receive State
|<-----Ready and -------|
Download Request
|---Download State----->|
Figure 3: Message Flow
7. Security Considerations
Any network technology which changes state in network devices may be
subverted and abused by bad actors.
Because we are modifying state in particular types of network devices
- middleboxes such as NATs, firewalls, and load balancers - we are
concerned with specific types of attacks.
These attacks include:
o Using the middlebox signaling mechanism to open firewall pinholes
in contravention of local policy
o Using the middlebox signaling mechanism to close firewall
pinholes, creating a denial of service for the network flows
associated with those pinholes
o Using the topology discovery mechanism to map network topology and
locate devices of interest
o Using the signaling protocol to consume middlebox resources,
creating a denial of service attack against the entire middlebox
o Using the signaling mechanism to install or alter NAT table
mappings to route traffic to an attacker
Gu, et al. Expires August 23, 2013 [Page 11]
Internet-Draft State Migration Architecture February 2013
o Eavesdropping on signaling or discovery mechanism traffic to watch
for changes in network topology (state migrates when a VM
migrates)
The primary mechanisms against attacks on a middlebox state migration
technology are origin authentication and integrity protection. Every
message MUST be authenticated as having been originated by the
ostensible sender - it must be possible to detect forgeries. Every
message MUST be authenticated as having been unaltered in transit.
It may also be desirable to encrypt the traffic to provide
protections against eavesdropping.
Additionally, there may be operational protections against using a
state migration mechanism as an attack vector against data center
networks. For example, implementing policies against permitting
state migration signaling traffic from outside a range of permitted
addresses or transiting any but a limited list of network filtering
devices.
8. References
8.1. Normative Reference
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", March 1997.
[bridging]
"IEEE P802.1Qbg Edge Virtual Bridging".
8.2. Informative Reference
[framework]
Gu, Y., Shore, M., and S. Sivakumar, "A Framework and
Problem Statement for Flow-associated Middlebox State
Migration", October 2012.
Authors' Addresses
Yingjie Gu
Huawei
Phone: +86-25-56624760
Fax: +86-25-56624702
Email: guyingjie@huawei.com
Gu, et al. Expires August 23, 2013 [Page 12]
Internet-Draft State Migration Architecture February 2013
Melinda Shore
No Mountain Software
PO Box 16271
Two Rivers, AK 99716
US
Phone: +1 907 322 9522
Email: melinda.shore@nomountain.net
Senthil Sivakumar
Cisco Systems
7100-8 Kit Creek Road
Research Triangle Park, NC
US
Email: ssenthil@cisco.com
Dacheng Zhang
Huawei Technologies
Email: zhangdacheng@huawei.com
Gu, et al. Expires August 23, 2013 [Page 13]
