+--------+                        +--------+   
                | VNF-B2 |                       #| VNF-B2 |###
   +--------+   +--------+           +--------+ # +--------+   
###| VNF-A1 |       _|_           ###| VNF-A1 |#      _|_      
   +--------+      (___)             +--------+      (___)     
  ___/    #       /     \      \    ___/            /     \    
 (___)+---#------+       +   ===}  (___)+----------+       +   
          #       \ ___ /      /                    \ ___ /    
          #        (___)                             (___)     
          #          |                                 |
          #     +--------+                        +--------+   
          ######| VNF-B1 |###        (failure)--> | VNF-B1 |   
                +--------+                        +--------+   

### NFP

Figure 1: A fail-over use case

The above is in the case where there is a 1+1 or 1:N redundancy scheme. In event that VNF instance overloads before NFVO has time to scale out, or when resources do not permit a scale-out then we can route the service chain deterministically to a remote VNF instance. This adaptation may be revertive or non-revertive dependent on service provider policy and resource availability.

4.2. Load balancing

A single VNF instance may be a bottleneck point of a service chain due to its overload. It may affect the performance of the whole service chain consequently so that an NFP instance needs to be built to avoid bottleneck points or maintained to distribute workloads of overloaded VNF instances.

With NFVI-PoP Interconnect, service chains can be balanced between NFVI-PoPs in a way that best utilises NFV infrastructure and ensures service integrity. The wide area conditions can be monitored in real-time to provide KPIs, such as BW, delay, delay variation and packet loss per QoS class to the service chaining application which may enable use of external VNF instances when there is an overload or failure condition in the local NFVI-PoP. In this way the service chaining application can make a service chain reroute decision (in the event of failure and/or overload) that is network and platform-aware. The service chaining application understands the state of external VNFs and WAN conditions per QoS class between the local NFVI-PoP and remote NFVI-PoP in real-time.

4.3. Path optimization

Traffic for a network service traverses all of the VNF instances and the connecting VL instances given by a NFP instance to reach a target end point. Thus, quality of the network service depends on the resource constraints (e.g., processing power, bandwidth, topological locations, latency) of VNF instances, VL instances including NFVI-PoP interconnects. In order to optimize the path of the network service, the resource constraints of VNF instances and VLs need to be considered at constructing NFPs. Since the resource state may vary in time during the service, NFP instances also need to adapt to the changes of resource constraints of the VNF instances and VL instances by monitoring and replacing them at run-time.

4.4. Traffic optimization

A network operator may provide multiple network services with different VNF-FGs and different flows of traffic traverse between source and destination end-points along the VNF-FGs. For efficiency of resource usage, the NFP instances need to be built by default to localize the traffic flows and to avoid processing and network bottlenecks. It is only in the case of local failure or overload (whereby the NFVO is unable or has not completed a scale-out of on-site resources) that NFP instances would be constructed between NFVI-PoPs. In this case, multiple VNF instances of different NFVI-PoPs need to be considered together at constructing a new NFP instance or adapting one.

4.5. Energy efficiency

Energy efficiency in the network is getting important to reduce impact on the environment so that energy consumption of VNF instances using NFVI resources (e.g., compute, storage, I/O) needs to be considered at NFP instantiation or adaptation. For example, a NFP can be instantiated as to make traffic flows aggregated into a limited number of VNF instances as much as its performance is preserved in a certain level. Policy may vary between centralized or distributed NFV applications, and could include policies for even energy distribution between sites, time-of-day etc.

5. Evaluation Model

To derive specific algorithms for use cases discussed in Section 4, an evaluation model needs to be developed, which can address two problems for a given network topology and input parameters (e.g., VL/VNF capacity, incoming traffic flows, etc.) : 1) how much traffic flows pass on each VL instance and 2) how much processing capacity is needed for the installed VNF instance. This section first describes the system model and then presents main objectives for the evaluation model.

5.1. System Model

The system model considers the follwing network topoloy. The network topology under consideration is composed of start/end points and multiple NFVI-PoPs where multiple VNF instances locate. On the other hand, VL instances inter-connect VNF instances in NFVI-PoPs.

Start and end points are incoming and outgoing points of traffic flow for a given network service, respectively. Specifically, the amount of incoming traffic flows for a network service (i.e., a VNF-FG) at the start point is given as an input parameter in the model.

Since VNF instances process their incoming traffic flows, their processing capacities should be specified. Therefore, the VNF processing capacity is defined as the maximum amount of traffic flows that a VNF instance can process. Each VNF instance can be with different processing capacities depending on the class of the running virtual machine (VM). In the model, it is assumed that there are multiple VM classes and each VM class has a pre-defined processing capacity. Also, VL instances have different capacities for supporting traffic flows, which is defined as the maximum amount of traffic flows that can pass on a VL instance.

Under the network topology, multiple NFs (e.g., firewall and NAT) can be implemented as VNF instances. Therefore, a network service (i.e., a VNF-FG) refers to an order set in which several NFs are deployed depending on the requirements of traffic flows. That is, traffic flows for a VNF-FG should be processed according to the NF order described in the given VNF-FG. Accoridngly, traffic flows at the start point should not be processed by any NF. Meanchile, traffic flows at the end point should be processed by all NFs specified in the given VNF-FG.

In a given VNF-FG, VNFs should be individually placed on multiple NFVI-PoPs. Therefore, a decision variable, VNF placement indicator function (VPIF), is defined as:

• VNF placement indicator function (VPIF): indicator function (i.e., 0 or 1) to represent the location (i.e., a NFVI-PoP) where the VNF instance with a VM class is placed.

Intuitively, the amount of traffic flows that pass a VL instance should not exceed the VL capacity to avoid any overload at the VL instance. Likewise, the amount of incoming traffic flows to a VNF instance should not exceed the VNF processing capacity. Therefore, traffic flows for a network service (i.e., a VNF-FG) can be distributed to multiple NFPs depending on resource and capacity constraints for VNF and VL instances. Moreover, multiple network services can be supported by distributing traffic flows for each network service. Therefore, another decision variable, traffic flow ratio (TFR), is defined as:

• Traffic flow ratio (TFR): the ratio of the traffic flows distributed to each NFP. Therefore, the amount of traffic flows that passes on each NFP is the product of TFR and the amount of incoming traffic flows for a network service. Note that TFR and the amount of incoming traffic floews can be computed by measuring the amount of traffic flows that passes on each VL.

In addition, traffic flows should be conserved during the network service. Therefore, the amount of incoming traffic flows to a VNF or a NFVI-PoP should be equal to the amount of outgoing traffic flows from the VNF or the NFVI-PoP.

Consequently, in this evaluation model, the path/traffic optimization and VNF instance placement are jointly considered by determining two decision variables, i.e., TFR and VPIF. For the path/traffic optimization, how much traffic flows pass on each VL instance is determined by obtaining TFR. Meanwhile, for the given VNF instance placement, which VNF instances with different processing capacities (i.e., VM class) are installed in each NFVI-PoP is addressed, i.e., VPIF is obtained.

5.2. Main objectives

In the evaluation model, three objectives are considered including, but not limited to, 1) load balancing, 2) flow throughput maximization, and 3) energy efficiency.

5.2.1. Load balancing

For load balancing for a network service, the remaining capacity for VNF instances and VL instances should be balanced to avoid any bottlenecks. To this end, the minimum remaining processing capacity for VNF instances and the minimum remaining link capacity for VL instances should be maximized.

5.2.2. Throuhgput optimization

On the other hand, the flow throughput considers both throughputs for VNF processing and for VL instance. Then, the throughput of an NFP can be calculated as the product of TFR and the sum of capacities, and the total throughput is the sum of computed throughputs for all NFPs. By maximizing the total flow throughput, it is possible to reduce the network service time.

5.2.3. Energy efficiency

Since each VNF instance consumes an amount of energy for processing its function and tranmitting/receiving traffic flows across VL instances, the energy consumption for each VNF instance should be minimized for energy efficiency of network services. Detailed model is under construction.

6. Framework

To support the aforementioned use cases, it is required to support resource management capability which provides service chain (or NFP) construction and adaptation by considering resource state or constraints of VNF instances and VL instances which connect them. The resource management operations for service chain construction and adaptation can be divided into several sub-actions:

• Locate VNF instances
• Evaluate the performance of VNF instances and VL instances
• Relocate (or scale) VNF instances to update a NFP instance
• Monitor state or resource constraints of a VNF instance, VL instances including NFVI-PoP interconnects

As listed above, VNF instances are relocated according to monitoring or evaluation results of performance metrics of the VNF instances and VL instances. Studies about evaluation methodologies and performance metrics for VNF instances and NFVI resources can be found at [ETSI-NFV-PER001] [I-D.liu-bmwg-virtual-network-benchmark] [I-D.ietf-bmwg-virtual-net]. The performance metrics of VNF instances and VL instances specific to service chain construction and adaptation can be defined as follows:

• availability (or failure) of a VNF instance and a VL instance
• a topological location of a VNF instance
• CPU and memory utilization rate of a VNF instance
• a throughput of a VNF instance
• energy consumption of a VNF instance
• bandwidth of a VL instance
• packet loss of a VL instance
• latency of a VL instance
• delay variation of a VL instance

The resource management functionality for dynamic service chain adaptation takes role of NFV orchestration with support of VNF manager (VNFM) and Virtualised Infrastructure Manager (VIM) in the NFV framework [ETSI-NFV-ARCH]. Detailed functional building block and interfaces are still under study.

7. Applicability to SFC

7.1. Related works in IETF SFC WG

IETF SFC WG provides a new service deployment model that delivers the traffic along the predefined logical paths of service functions (SFs), called service function chains (SFCs) with no regard of network topologies or transport mechanisms. Basic concept of the service function chaining is similar to VNF-FG where a network service is composed of SFs and deployed by making traffic flows traversed instances of the SFs in a pre-defined order.

There are several works in progress in IETF SFC WG for resource management of service chaining. [I-D.ietf-sfc-architecture] defines SFC control plane that selects specific SFs for a requested SFC, either statically or dynamically but details are currently outside the scope of the document. There are other works [I-D.ietf-sfc-control-plane] [I-D.lee-sfc-dynamic-instantiation] [I-D.krishnan-sfc-oam-req-framework] [I-D.ietf-sfc-oam-framework] which define the control plane functionality for service function chain construction and adaptation but details are still under study. While [I-D.dunbar-sfc-fun-instances-restoration] and [I-D.meng-sfc-chain-redundancy] provide detailed mechanisms of service chain adaptation, they focus only on resilience or fail-over of service function chains.

7.2. Integration in SFC control-plane architecture

In SFC WG, [I-D.ietf-sfc-control-plane] describes requirements for conveying information between Service Function Chaining (SFC) control elements (including management components) and SFC functional elements. It also identifies a set of control interfaces to interact with SFC-aware elements to establish, maintain or recover service function chains.

                +----------------------------------------------+ 
                |                                              |
                |       SFC  Control & Management Planes       |
        +-------|                                              |
        |       |                                              |
        C1      +------^-----------^-------------^-------------+
 +---------------------|C3---------|-------------|-------------+
 |      |            +----+        |             |             |
 |      |            | SF |        |C2           |C2           |
 |      |            +----+        |             |             |
 | +----V--- --+       |           |             |             |
 | |   SFC     |     +----+      +-|--+        +----+          |
 | |Classifier |---->|SFF |----->|SFF |------->|SFF |          |
 | |   Node    |<----|    |<-----|    |<-------|    |          |
 | +-----------+     +----+      +----+        +----+          |
 |                     |           |              |            |
 |                     |C2      -------           |            |
 |                     |       |       |     +-----------+ C4  |
 |                     V     +----+ +----+   | SFC Proxy |-->  |
 |                           | SF | |SF  |   +-----------+     |
 |                           +----+ +----+                     |
 |                             |C3    |C3                      |
 |  SFC Data Plane Components  V      V                        |
 +-------------------------------------------------------------+