Internet DRAFT - draft-lachosrothenberg-alto-md-e2e-ns

draft-lachosrothenberg-alto-md-e2e-ns







ALTO WG                                                        D. Lachos
Internet-Draft                                             C. Rothenberg
Intended status: Informational                                   Unicamp
Expires: January 14, 2021                                  July 13, 2020


                   Multi-domain E2E Network Services
                draft-lachosrothenberg-alto-md-e2e-ns-02

Abstract

   Evolving networking scenarios (e.g., 5G) are considering the
   provision of value-added and on-demand end-to-end (E2E) network
   services in multi-domain (multi-operator/multi-technology)
   environments.  This document presents different initiatives, mainly
   within standardization efforts and research projects, working on E2E
   network services across multiple domains.  Problem statement and a
   layered network model are also described.  In addition, this document
   raises an initial proposal towards a new ALTO service in support of
   E2E network service requirements.  Finally, another important
   objective of this document is to begin a discussion about motivating
   use cases in scope of the ALTO WG after the re-chartering process.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 14, 2021.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of



Lachos & Rothenberg     Expires January 14, 2021                [Page 1]

Internet-Draft             Multi-domain E2E NS                 July 2020


   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Context and Motivation  . . . . . . . . . . . . . . . . . . .   3
     2.1.  Standardization Activities  . . . . . . . . . . . . . . .   3
       2.1.1.  IETF  . . . . . . . . . . . . . . . . . . . . . . . .   3
       2.1.2.  ETSI  . . . . . . . . . . . . . . . . . . . . . . . .   4
       2.1.3.  MEF . . . . . . . . . . . . . . . . . . . . . . . . .   5
     2.2.  Research projects . . . . . . . . . . . . . . . . . . . .   5
   3.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   6
     3.1.  Network Function Placement Decisions  . . . . . . . . . .   6
     3.2.  Network Inventory . . . . . . . . . . . . . . . . . . . .   6
     3.3.  Publishing Information  . . . . . . . . . . . . . . . . .   6
   4.  Network Function Virtualization Architectures and
       Infrastructures . . . . . . . . . . . . . . . . . . . . . . .   7
     4.1.  Layered Network Model . . . . . . . . . . . . . . . . . .   8
   5.  ALTO Extension: E2E Network Service Requirements
       Representation  . . . . . . . . . . . . . . . . . . . . . . .  10
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  12
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  12
   8.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  12
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  12
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  12
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  12
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  14

1.  Introduction

   The fifth generation (5G) of cellular networks is not only considered
   an evolution but a revolution in the field of information and
   communication technologies [WHITE-PAPER-5G].  5G will support the
   creation of new and novel End-to-End (E2E) services, applications and
   complex use case scenarios, such as massive Internet of Things,
   extreme real-time communications, broadband access everywhere, higher
   user mobility.  All these scenarios and services are triggering a
   modification in the way telecommunications sector deploy new network
   services, shifting from a commonly manual and long process to a
   flexible and programmable process.

   In this context, cloud computing , Software Defined Networking (SDN),
   and Network Function Virtualization (NFV) arise as technological



Lachos & Rothenberg     Expires January 14, 2021                [Page 2]

Internet-Draft             Multi-domain E2E NS                 July 2020


   pillars to achieve the necessary function programmability, network
   programmability, and resource virtualization during the provision of
   E2E network services.

   The delivery of an E2E network service, or simply E2E service, often
   requires VNFs and their specific order [RFC7665].  Network operators
   start offering to their customers the possibility of configuring
   network services with specific requirements in terms of resources
   (e.g., cpu, memory, hard-disk) and performance objectives (e.g.,
   bandwidth, latency) [VNF-PLAC].  Such demands are usually composed by
   distributed resources which are expected to available across multiple
   domains with different technology and/or administration.

   This document offers an overview of standardization activities and
   research projects, including problem statement, behind building E2E
   services traversing different domains (technological and/or
   administrative).  Moreover, from a layered network model, it is
   proposed a potential ALTO extension related to E2E Network Service
   requirements representation based on the ETSI NFV MANO data model.

   The overall rationale of this document is to arouse discussions into
   the ALTO WG concerning potential new items to be considered for the
   re-charter.

2.  Context and Motivation

   Different standardization efforts (e.g., IETF, MEF, ETSI) and
   research projects activities (e.g., 5GEx [H2020.5GEX], 5G-Transformer
   [H2020-5G-TRANSFORMER], T-NOVA [T-NOVA]) have been focused on multi-
   domain network service chaining.  Standardization is essential to
   provide recommendations to create interoperable architectures with
   standardized protocols, and solutions (being developed by different
   projects) are addressing a diverse range of requirements to provide
   network services provided using multiple domains.

   This section briefly describes, on the one hand, main standardization
   efforts delivering collections of norms and recommendations, while on
   the other hand it also provides an overview of several projects
   formed to develop network services across multiple domains.

2.1.  Standardization Activities

2.1.1.  IETF

   SFC that span domains owned by single or multiple administrative
   entities are being proposed.  The Hierarchical Service Function
   Chaining (hSFC) [RFC8459], for example, defines an architecture to
   deploy SFC in large networks.  This RFC proposes to decompose the



Lachos & Rothenberg     Expires January 14, 2021                [Page 3]

Internet-Draft             Multi-domain E2E NS                 July 2020


   network into smaller domains (domains under the control of a single
   organization).  Another proposed initiative is [DRAFT-HH-MDSFC] that
   describes SFC crossing different domains owned by various
   organizations (e.g., ISPs) or by a single organization with
   administration partitions.  The proposed architecture uses a SFC
   eXchange Platform (SXP) to collect and exchange information
   (topology, service states, policies, etc.) between different
   organizations and it works both in centralized (Multiple SFC domains
   connected by a logical SXP) and distributed (SXP server as a broker)
   environments.

   More recently, the IETF ALTO WG started to discuss the uses of ALTO
   as an information model for representing network resource and
   services in multi-domain scenarios:

   o  [DRAFT-ALTO-BROKER-MDO] proposes an ALTO-based Broker-assisted
      architecture where a broker plane works as a coordinator between a
      set of top-level control planes, i.e., Domain Orchestrators (DOs)
      and Multi-Domain Orchestrators (MdOs).  The ALTO services (with
      the proposed extensions) provides abstract maps with a simplified,
      yet enough information view about MdOs involved in the federation.
      This information includes the abstract network topology, resource
      availability (e.g., CPUs, Memory, and Storage) and capabilities
      (e.g., supported network functions).

   o  [DRAFT-ALTO-UNICORN] presents Unicorn, a resource orchestration
      framework for multi-domain, geo-distributed data analytics.  This
      work resorts in ALTO as the information model to support the
      accurate, yet privacy-preserving resource discovery across
      different domains.  The key information to be provided by the use
      of ALTO including different types of resources, e.g., the
      computing, storage, and networking resources.

   o  [DRAFT-MD-SFC-ALTO] describes different standardization activities
      and research projects addressing the challenges posed by Service
      Function Chaining (SFC) across multiple domains (specifically,
      multiple administrative domains).  In addition, this document
      presents an initial approach to realize inter-domain service
      chaining leveraging the ALTO protocol.  Finally, another important
      concern of this document is to initiate a discussion (ALTO, SFC as
      well as 9other WGs) regarding if, how, and under what conditions
      ALTO can be useful to improve the multi-domain SFC process.

2.1.2.  ETSI

   The ETSI NFV ISG is paving the way toward viable architectural
   options supporting the efficient placement of functions in different
   administrative domains.  More specifically, the document



Lachos & Rothenberg     Expires January 14, 2021                [Page 4]

Internet-Draft             Multi-domain E2E NS                 July 2020


   [ETSI-NFV-IFA028] reports different NFV MANO architectural approaches
   with use cases related to network services provided using multiple
   administrative domains.  Besides, it gives a non-exhaustive list of
   key information to be exchanged between administrative domains
   (monitoring parameters, topology view, resource capabilities, etc.)
   and recommendations related to security to permit the correct and
   proper operation of the final service.

2.1.3.  MEF

   With its work on the Service Operations Specification MEF
   55 [MEF-SOE-MEF55], MEF has defined a reference architecture and
   framework for describing functional management entities (and
   interfaces between them) needed to support Lifecycle Service
   Orchestration (LSO).  This LSO architecture enables automated
   management and control of E2E connectivity services across multiple
   operator networks.  The automated service management includes
   fulfillment, control, performance, assurance, usage, security,
   analytics, and policy capabilities that make it possible, for
   example, expanding the footprint of service providers to interact
   with potentially several operators to manage and control the access
   portions of E2E services.

2.2.  Research projects

   Several projects include an architectural model integrating NFV
   management with SDN control capabilities to address the challenges
   towards flexible, dynamic, cost-effective, and on-demand service
   chaining.

   [H2020.5GEX] aims to integrate multiple administrations and
   technologies through the collaboration between operators in the
   context of emerging 5G networking. [VITAL][T-NOVA] follow a
   centralized approach where each domain advertises its capabilities to
   a federation layer which will act as a broker.  In order to avoid one
   network operator per country or regions, [H2020-5G-NORMA] proposes
   the use of management and control into a single virtual domain.
   Also, the 5G-Transformer project [H2020-5G-TRANSFORMER] is defining
   flexible slicing and federation of transport networking and computing
   resources across multiple domains.  The NECOS project [H2020-NECOS],
   focused on the realization of E2E multi-domain cloud network slicing,
   proposes an architectural approach with slice information interfaces
   for resource exposure and resource discovery during slice
   provisioning.  In addition , the architecture includes a slice
   marketplace interface between domain orchestrators and a marketplace
   broker.





Lachos & Rothenberg     Expires January 14, 2021                [Page 5]

Internet-Draft             Multi-domain E2E NS                 July 2020


3.  Problem Statement

3.1.  Network Function Placement Decisions

   An E2E service request specify virtual nodes (a set of required VNFs)
   as well as virtual links (the order in which they must be executed).
   Virtual nodes are deployed in virtual machines hosted by different
   physical servers, and virtual links correspond to physical paths that
   connect those servers hosting VNFs.  Both virtual nodes and virtual
   links are limited resources and both may also be located on different
   technological domains in a single administration and even crossing
   multiple administrations [VNF-MOB][SFC-ORC].  So that the placement
   decision problem involves to discover "best" candidate resources and
   "best" feasible paths between such resources.

3.2.  Network Inventory

   Placement decisions are a fundamental step for the management and
   orchestration of network services.  Management systems (e.g., DOs,
   MdOs) need to maintain an inventory of the network providing a real-
   time representation or view of available infrastructure resources,
   software resources, and their relationships.  However, The size of a
   network inventory can be very large in scenarios, such as distributed
   cloud and edge computing.  As a result, management systems experiment
   scalability problems processing large amounts of data to decide where
   to instantiate a service or part of the service.  Therefore, building
   a network inventory, under these circumstances, needs aggregation
   mechanisms to reduce time for discovery of resources and to simplify
   and optimize management of them.

3.3.  Publishing Information

   Once a network inventory is built, a mechanism for publishing
   information is also necessary so that the network inventory can
   provide a simplified, yet enough network information view to
   management systems.  In order to retrieve such information to perform
   placement decisions, a communication protocol between management
   systems and network inventory is also necessary.

   Therefore, on the one hand, network information (e.g., network
   locations, costs between them, endhost properties) needs to be
   advertised to the network applications and, on the other hand,
   network applications (e.g., DOs, MdOs) needs to describe their
   requirements and obtain information about resources that suit such
   requirements.






Lachos & Rothenberg     Expires January 14, 2021                [Page 6]

Internet-Draft             Multi-domain E2E NS                 July 2020


4.  Network Function Virtualization Architectures and Infrastructures

   With the introduction of NFV, network functions (e.g., switches,
   routers, firewalls), and also complex network functions (e.g., EPC)
   are able to be virtualized and implemented as a collection of virtual
   machines (VMs) deployed over the virtualized infrastructure.  In
   turn, the virtualized infrastructure is instantiated on a substrate
   network.

   In this context, one of the most accepted NFV architectural
   frameworks is the proposed by the ETSI ISG NFV working
   group [ETSI-NFV-WHITEPAPER].  Figure 1 [DRAFT-MD-VIRT] shows this NFV
   reference architecture.  On the left, we can see the data plane:
   NFVIs hardware/software, VNFs, and optional element management
   systems.  On the right, we see the control plane: VIM which is
   something like Openstack or Kubernetes, virtual network function
   managers, and the NFV Orchestrator on the top.


































Lachos & Rothenberg     Expires January 14, 2021                [Page 7]

Internet-Draft             Multi-domain E2E NS                 July 2020


                                                  +--------------------+
   +-------------------------------------------+  | ----------------   |
   |                 OSS/BSS                   |  | | NFV          |   |
   +-------------------------------------------+  | | Orchestrator +-- |
                                                  | ---+------------ | |
   +-------------------------------------------+  |    |             | |
   |  ---------     ---------     ---------    |  |    |             | |
   |  | EM 1  |     | EM 2  |     | EM 3  |    |  |    |             | |
   |  ----+----     ----+----     ----+----    |  | ---+----------   | |
   |      |             |             |        |--|-|    VNF     |   | |
   |  ----+----     ----+----     ----+----    |  | | manager(s) |   | |
   |  | VNF 1 |     | VNF 2 |     | VNF 3 |    |  | ---+----------   | |
   |  ----+----     ----+----     ----+----    |  |    |             | |
   +------|-------------|-------------|--------+  |    |             | |
          |             |             |           |    |             | |
   +------+-------------+-------------+--------+  |    |             | |
   |         NFV Infrastructure (NFVI)         |  |    |             | |
   | -----------    -----------    ----------- |  |    |             | |
   | | Virtual |    | Virtual |    | Virtual | |  |    |             | |
   | | Compute |    | Storage |    | Network | |  |    |             | |
   | -----------    -----------    ----------- |  | ---+------       | |
   | +---------------------------------------+ |  | |        |       | |
   | |         Virtualization Layer          | |--|-| VIM(s) +-------- |
   | +---------------------------------------+ |  | |        |         |
   | +---------------------------------------+ |  | ----------         |
   | | -----------  -----------  ----------- | |  |                    |
   | | | Compute |  | Storage |  | Network | | |  |                    |
   | | | hardware|  | hardware|  | hardware| | |  |                    |
   | | -----------  -----------  ----------- | |  |                    |
   | |          Hardware resources           | |  |  NFV Management    |
   | +---------------------------------------+ |  | and Orchestration  |
   +-------------------------------------------+  +--------------------+


                Figure 1: ETSI MANO Reference Architecture

4.1.  Layered Network Model

   Based on the ETSI NFV reference architecture, a layered network model
   is identified: Network Service layer, VNF layer, and Resource Layer.
   This model allows a separation of network relationships in different
   levels of abstraction.  For example, network services can be queried
   at different levels of abstraction or we can map service paths in
   different layers (from an abstract to a more concrete layer).

   In Figure 2, we have a network service with a set of interconnected
   VNFs.  This network service topology is represented by the Network
   Service layer.



Lachos & Rothenberg     Expires January 14, 2021                [Page 8]

Internet-Draft             Multi-domain E2E NS                 July 2020


   A VNF is typically divided into a set of virtual function components
   (VFCs) which comprise the VNF Layer.  Each VFC is an application
   running within a single VM or container.

   In case of the resource layer, we have virtual layer and physical
   layer.  The virtual layer represents the virtual overlay network and
   the physical layer represents the substrate network.  Virtualized
   infrastructures (e.g., VMs, virtual routers) are instantiated on a
   physical infrastructure.










































Lachos & Rothenberg     Expires January 14, 2021                [Page 9]

Internet-Draft             Multi-domain E2E NS                 July 2020


          +----------+
          |          |
          | +------+ |
  Network | | NFVO | |    +-----+    +------+     +------+     +------+
  Service | +---+--+ |    | SRC +--->-|VNF1 +---->+ VNF2 +---->+ DST  |
  Layer   |     |    |    +-----+    +--+---+     +------+     +------+
          |     |    |                  |
  +--------------------------------------------------------------------+
          |     |    |                  |
          |     |    |     +------------v------------------+
          |     |    |     |           VNF1                |
          |     |    |     +-------+-----+--------------+--+
          | +---+--+ |             |     |              |     Repeat for
  VNF     | | VNFM | |             |     |              |     each VNF
  Layer   | +---+--+ |   +----+    |     |    +----+  +-+--+
          |     |    |   |VFC +----+     +----+VFC |  |VFC |
          |     |    |   +-+--+               +-+--+  +-+--+
          |     |    |     |                    |       |
  +-------+----------+-------------------------------------------------+
                |          |                    |       |             V
          |     |    |     |                    |       |             i
          |     |    |   +-+-+     +---+      +-+-+   +-+-+    +---+  r
          |     |    |   |VM |     |VM |      |VM |   |VM |    |VM |  t
          |     |    |   +-+-+     ++--+      +-+-+   +-+-+    ++--+  u
          |     |    |     |        |           |       |       |     a
          |     |    |     |  +---+ |           |       |       |     l
          | +---+-+  |     |  |VM | |           |       |       |
  Resource| | VIM |  |     |  ++--+ |           |       |       |
  Layer   | +-----+  |     |   |    |     +-----v-------v--+    |     P
          |          |     |   |    | XXXX| Host/Hypervisor|XXX |     h
          |          |     |   |    | X   +----------------+  X |     y
          |          |     |   |    | X                       X |     s
          |          | +---v--->  +-v-X---+              +----X-v+    i
          |          | | Host/ +  | Host/ XXXXXXXXXXXXXXX| Host/ |    c
          |  NFV     | | Hyp   XXX| Hyp   +              | Hyp   |    a
          |  MANO    | +-------+  +-------+              +-------+    l
          +----------+


                Figure 2: ETSI MANO Reference Architecture

5.  ALTO Extension: E2E Network Service Requirements Representation

   From the layered network model described in the previous section, we
   are considering an ALTO extension related to E2E Network Service
   requirements representation.  An initial proposal has been presented
   in the ALTO-based Broker-assisted MdO draft [DRAFT-ALTO-BROKER-MDO]
   where network applications (as ALTO clients) can specify a set of



Lachos & Rothenberg     Expires January 14, 2021               [Page 10]

Internet-Draft             Multi-domain E2E NS                 July 2020


   basic E2E service requirements to an ALTO server in order to obtain
   candidate resources (domains) and candidate paths.

   This initial E2E service requirement representation is inspired on
   the ETSI NFV MANO data model [ETSI-NFV-MAN001].  This model defines
   network services as a composition of network functions including the
   specification of deployment and operational requirements.  Such
   specifications are captured in templates called Network Service
   Descriptor (NSD) and Virtual Network Function Descriptor (VNFD) that
   contain (relatively) static information used in the process of on-
   boarding network services and VNFs, respectively.

   o  High level objects in a NSD include (among others)
      [ETSI-NFV-MAN001][OSM-DM]:

      *  Constituent VNFs: List of VNFDs that are part of the network
         service.

      *  VNF Dependencies: This describes dependencies between VNFs.
         For example, the order in which the VNFs inside a network
         service should be started.

      *  Network service Connection Points: Each network service has one
         or more external connection points (which act as endpoints)
         used to link two network services or to link external networks.

      *  Virtual Links: List of Virtual Link Descriptors (VLDs) that
         describe how VNFs (in the NSD) are connected.

   o  High level objects in a VNFD include (among others)
      [ETSI-NFV-MAN001][OSM-DM]:

      *  Constituent VDUs: List of virtual deployment units (VDUs) in a
         specific VNF.  Each VDU (also referred to VFC) describes the
         VM/Container capabilities (e.g., CPU, RAM, disks).

      *  VDU Dependencies: List of VDU dependencies used for determining
         the order of startup for VDUs.

      *  VNF Connection Points: List of external connection points used
         for connecting a VNF to other VNFs or to external networks.

      *  Internal VLDs: List of internal virtual links to connect
         various VDUs/VFCs.







Lachos & Rothenberg     Expires January 14, 2021               [Page 11]

Internet-Draft             Multi-domain E2E NS                 July 2020


6.  IANA Considerations

   This document includes no request to IANA.

7.  Security Considerations

   TBD.

8.  Acknowledgments

   This work is supported by the Innovation Center of Ericsson S.A.,
   Brazil (grant agreement UNI.64).  This document however does not
   necessary represent Ericsson's official viewpoint.

9.  References

9.1.  Normative References

   [RFC7665]  Halpern, J., Ed. and C. Pignataro, Ed., "Service Function
              Chaining (SFC) Architecture", RFC 7665,
              DOI 10.17487/RFC7665, October 2015,
              <https://www.rfc-editor.org/info/rfc7665>.

   [RFC8459]  Dolson, D., Homma, S., Lopez, D., and M. Boucadair,
              "Hierarchical Service Function Chaining (hSFC)", RFC 8459,
              DOI 10.17487/RFC8459, September 2018,
              <https://www.rfc-editor.org/info/rfc8459>.

9.2.  Informative References

   [DRAFT-ALTO-BROKER-MDO]
              Perez, D. and C. Rothenberg, "ALTO-based Broker-assisted
              Multi-domain Orchestration", draft-lachosrothenberg-alto-
              brokermdo-03 (work in progress), March 2020.

   [DRAFT-ALTO-UNICORN]
              Xiang, Q., Zhang, J., Le, F., Yang, Y., and H. Newman,
              "Resource Orchestration for Multi-Domain, Exascale, Geo-
              Distributed Data Analytics", draft-xiang-alto-multidomain-
              analytics-03 (work in progress), March 2020.

   [DRAFT-HH-MDSFC]
              Li, G., Li, G., Xu, Q., Zhou, H., and B. Feng, "Hybrid
              Hierarchical Multi-Domain Service Function chaining",
              draft-li-sfc-hhsfc-08 (work in progress), March 2020.






Lachos & Rothenberg     Expires January 14, 2021               [Page 12]

Internet-Draft             Multi-domain E2E NS                 July 2020


   [DRAFT-MD-SFC-ALTO]
              Perez, D., Xiang, Q., Rothenberg, C., and Y. Yang, "Multi-
              domain Service Function Chaining with ALTO", draft-lachos-
              multi-domain-sfc-alto-01 (work in progress), March 2020.

   [DRAFT-MD-VIRT]
              Bernardos, C., Contreras, L., Vaishnavi, I., Szabo, R.,
              Li, X., Paolucci, F., Sgambelluri, A., Martini, B.,
              Valcarenghi, L., Landi, G., Andrushko, D., and A. Mourad,
              "Multi-domain Network Virtualization", draft-bernardos-
              nfvrg-multidomain-05 (work in progress), September 2018.

   [ETSI-NFV-IFA028]
              ETSI, "Report on architecture options to support multiple
              administrative domains V3.1.1", Jan 2018,
              <http://www.etsi.org/deliver/etsi_gr/NFV-
              IFA/001_099/028/03.01.01_60/gr_NFV-IFA028v030101p.pdf>.

   [ETSI-NFV-MAN001]
              ETSI, "Network Functions Virtualisation (NFV); Management
              and Orchestration", Dec 2014,
              <https://www.etsi.org/deliver/etsi_gs/NFV-
              MAN/001_099/001/01.01.01_60/gs_nfv-man001v010101p.pdf>.

   [ETSI-NFV-WHITEPAPER]
              ETSI, "Network Functions Virtualisation - White Paper 2",
              Oct 2013,
              <https://portal.etsi.org/NFV/NFV_White_Paper2.pdf>.

   [H2020-5G-NORMA]
              H2020, "5G-NORMA -- 5G Novel Radio Multiservice adaptive
              network Architecture", 2015, <https://5gnorma.5g-ppp.eu/>.

   [H2020-5G-TRANSFORMER]
              H2020, "5G-Transformer -- 5G Mobile Transport Platform for
              Vertical", 2017, <http://5g-transformer.eu/>.

   [H2020-NECOS]
              H2020 EU-Brazil, "NECOS -- Novel Enablers for Cloud
              Slicing", 2018, <http://www.h2020-necos.eu/>.

   [H2020.5GEX]
              Bernardos, C., Dugeon, O., Galis, A., Morris, D., Simon,
              C., and R. Szabo, "5G Exchange (5GEx)-- Multi-domain
              Orchestration for Software Defined Infrastructures",
              focus vol. 4, no.5, p.2, 2015.





Lachos & Rothenberg     Expires January 14, 2021               [Page 13]

Internet-Draft             Multi-domain E2E NS                 July 2020


   [MEF-SOE-MEF55]
              Metro Ethernet Forum, "Lifecycle Service Orchestration
              (LSO): Reference Architecture and Framework", Mar 2016,
              <https://www.mef.net/Assets/Technical_Specifications/PDF/
              MEF_55.pdf>.

   [OSM-DM]   Open Source MANO, "OSM - Data Model", 2016,
              <https://osm.etsi.org/wikipub/index.php/
              Release_0_Data_Model_Details>.

   [SFC-ORC]  Sun, G., Li, Y., Liao, D., and V. Chang, "Service Function
              Chain Orchestration across Multiple domains: A Full Mesh
              Aggregation Approach", IEEE Transactions on Network and
              Service Management 1175--1191, 2018.

   [T-NOVA]   FP7 project T-NOVA, "T-NOVA Project, Network Functions as
              a Service over Virtualised Infrastructures", 2014,
              <http://www.t-nova.eu/>.

   [VITAL]    VITAL PROJECT H2020, "VITAL -- VIrtualized hybrid
              satellite-TerrestriAl systems for resilient and fLexible
              future networks", 2015, <http://www.ict-vital.eu/>.

   [VNF-MOB]  Patel, A., Vutukuru, M., and D. Krishnaswamy, "Mobility-
              aware VNF placement in the LTE EPC", IEEE Conference on
              Network Function Virtualization and Software Defined
              Networks (NFV-SDN) 1--7, 2017.

   [VNF-PLAC]
              Slim, F., Guillemin, F., Gravey, A., and Y. Hadjadj-Aoul,
              "Towards a dynamic adaptive placement of virtual network
              functions under ONAP", IEEE Conference on Network Function
              Virtualization and Software Defined Networks (NFV-SDN)
              210--215, 2017.

   [WHITE-PAPER-5G]
              NetWorld2020, ETP, "5g: Challenges, research priorities,
              and recommendations", Journal: Joint White Paper
              September, 2014.

Authors' Addresses










Lachos & Rothenberg     Expires January 14, 2021               [Page 14]

Internet-Draft             Multi-domain E2E NS                 July 2020


   Danny Alex Lachos Perez
   University of Campinas
   Av. Albert Einstein 400
   Campinas, Sao Paulo  13083-970
   Brazil

   Email: dlachosp@dca.fee.unicamp.br
   URI:   https://intrig.dca.fee.unicamp.br/danny-lachos/


   Christian Esteve Rothenberg
   University of Campinas
   Av. Albert Einstein 400
   Campinas, Sao Paulo  13083-970
   Brazil

   Email: chesteve@dca.fee.unicamp.br
   URI:   https://intrig.dca.fee.unicamp.br/christian/

































Lachos & Rothenberg     Expires January 14, 2021               [Page 15]