Internet DRAFT - draft-lachosrothenberg-alto-brokermdo
draft-lachosrothenberg-alto-brokermdo
ALTO WG D. Lachos
Internet-Draft C. Rothenberg
Intended status: Informational Unicamp
Expires: January 14, 2021 July 13, 2020
ALTO-based Broker-assisted Multi-domain Orchestration
draft-lachosrothenberg-alto-brokermdo-04
Abstract
Evolving networking scenarios (e.g., 5G) demand new multiple
administrative domain (aka multi-domain) orchestration models. This
document proposes a new broker-plane approach working on top of per-
domain management and orchestration functions to coordinate the
delivery of a multi-operator End-to-End Network Service (E2ENS).
This proposed design resorts to the Application-Layer Traffic
Optimization (ALTO) protocol to offer topology and resource
information from different administrative domains. The ALTO services
with the proposed protocol extension offer aggregated views on
various types of resources contributing to a more simple and scalable
solution.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 14, 2021.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
Lachos & Rothenberg Expires January 14, 2021 [Page 1]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Problem Statement and Challenges . . . . . . . . . . . . . . 5
5. Proposed Approach . . . . . . . . . . . . . . . . . . . . . . 6
5.1. Inter-domain Resource (IdR) Component . . . . . . . . . . 7
5.2. Inter-domain Topology (IdT) Component . . . . . . . . . . 7
5.3. ALTO Server Functionalities . . . . . . . . . . . . . . . 7
5.4. Filtered Cost Map Extension . . . . . . . . . . . . . . . 8
5.4.1. Accept Input Parameters . . . . . . . . . . . . . . . 8
5.4.2. Response . . . . . . . . . . . . . . . . . . . . . . 9
5.5. Examples of Message Exchange . . . . . . . . . . . . . . 9
5.5.1. Property Map Service . . . . . . . . . . . . . . . . 9
5.5.2. Filtered Cost Map Service . . . . . . . . . . . . . . 10
6. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.1. Benefits . . . . . . . . . . . . . . . . . . . . . . . . 14
6.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 14
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 15
10.1. Normative References . . . . . . . . . . . . . . . . . . 15
10.2. Informative References . . . . . . . . . . . . . . . . . 16
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Appendix A. Proof of Concept Use Case Implementation . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction
Envisioned 5G network architectures and related service models
consider broader cooperation between stakeholders in order to provide
flexible multi-operator multi-domain services. These multi-provider
orchestration operations will require the information exchange across
Multi-domain Orchestrators (MdOs). The key information to be
exchanged between MdOs includes the abstract network topology,
resource availability (e.g., CPUs, Memory, and Storage) and
capability (e.g., supported network functions).
Lachos & Rothenberg Expires January 14, 2021 [Page 2]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
This document presents a federation networking paradigm where a
broker-plane works on top of the management and orchestration plane
to assist and coordinate the creation of an End-to-End Network
Service (E2ENS) spanning over multi-operator multi-domain networks.
Our design resorts to the Application-Layer Traffic Optimization
(ALTO) protocol [RFC7285] to address the lack of abstractions to
discover and adequately represent in confidentiality-preserving
fashion the resource and topology information from different
administrative domains. Moreover, this draft introduces an extension
to the ALTO base protocol for inter-domain connectivity information
discovery.
2. Terminology
We use the following definitions, as established in [ETSI-NFV-DEF]:
Administrative Domain: Collection of systems and networks operated
by a single organization or administrative authority.
Network Function (NF): Functional block within a network
infrastructure that has well-defined external interfaces and well-
defined functional behaviour.
Network Functions Virtualisation (NFV): The principle of separating
network functions from the hardware they run on by using virtual
hardware abstraction.
NF Forwarding Graph: (NFFG): Graph of logical links connecting NF
nodes for the purpose of describing traffic flow between these
network functions.
Network Service Orchestration (NSO): Function responsible for
network service lifecycle management.
Resource Orchestration (RO): Function responsible for global
resource management governance.
Our proof of concept implementation follows the architectural
proposal of the 5GEx project [H2020.5GEX]. Some additional 5GEx
terms commonly used in this document are defined below:
Domain Orchestrator (DO): Performs Resource Orchestration and/or
Service Orchestration within the same administrative domain.
Multi-domain Orchestrator (MdO): Coordinates resource and/or service
orchestration at multi-domain level, where multi-domain may refer
to multiple DOs or multiple administrative domains.
Lachos & Rothenberg Expires January 14, 2021 [Page 3]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
Resource Topology (RT): Functional module that is responsible for
keeping an updated global view of the underlying infrastructure
topology exposed by DOs.
Service Graph (SG): A high-level data model for defining flexible
network services (including traffic steering primitives).
Service Access Point (SAP): A named/tagged port supporting stitching
(service to service, domain to domain, etc.)
3. Scope
Envisioned 5G scenarios are expected to work not only with
heterogeneous technologies but also across different network
operators. Many ongoing standardization activities and research
projects are addressing the multi-provider multi-domain orchestration
challenges under different approaches.
For example, within the IETF, [RFC8459] proposes a hierarchical
Service Function Chaining (SFC) for multiple domains under the same
administrative entity, and the document "Hybrid Hierarchical Multi-
Domain SFC [DRAFT-HHSFC] describes SFC crossing different domains
owned by various organizations or by a single organization with
administration partitions. In the NFVRG, the draft "Multi-domain
Network Virtualization" [DRAFT-MD-VIRT] envisions a complete E2E
logical network as stitching services offered by multiple domains
from multiple providers. Another initiative is the ETSI Industry
Specification Group for Network Functions Virtualization (ETSI NFV
ISG), the document [ETSI-NFV-IFA028] reports different NFV MANO
architectural approaches with use cases related to network services
provided using multiple administrative domains.
In case of research projects, [H2020.5GEX] [H2020-5G-TRANSFORMER]
seek to integrate multiple administrations and technologies through
the collaboration between operations. Other studies are envisioned
to use a centralized approach, where each domain advertises its
capabilities to a federation layer which will act as a
broker [VITAL][T-NOVA]. The proposed architecture in [ICAF] allows
the creation of cloud services from different administrative domains,
however, it is not related to the provisioning of NFV-based cross-
domain network services.
All such proposals described above envision the potential
introduction of new business model approaches, including federation
models [PPP-5:2013] among administrative domains. In this context,
this document considers each network operator involved in the
community advertises its abstracted capabilities (e.g., software/
hardware resources, physical/virtual network functions, etc.) to a
Lachos & Rothenberg Expires January 14, 2021 [Page 4]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
broker (i.e., 3rd party). This broker, in its turn, provides or
assists coordinate E2E network services spanning multi-domain
networks.
4. Problem Statement and Challenges
The provision of a complete E2E network service requires chaining
services provided by multiple network operators with multiple
technologies. In this multi-domain environment, the orchestration
process will require an advertisement mechanism through which
individual domains can describe their capabilities, resources, and
VNFs in an interoperable manner. Moreover, a discovery mechanism is
also necessary so that source domains can obtain candidate domains
(with the corresponding connectivity information) which can provide a
part of the service and/or slice in an E2ENS requirement.
In order that the advertising and discovery process works in a proper
way, a number of challenges can be identified:
Lack of Abstractions: Multiple vendors with heterogeneous
technologies need an information model to adequately represent
in confidentiality-preserving fashion the resource and topology
information.
Scalability: Involves the distribution of topology and resource
information in a peer-to-peer fashion (MdO-to-MdO). Multi-
operator multi-domain environments where the information
distribution is advertised in a peer-to-peer model scales
linearly. It means more MdO interconnections one has, the more
it "costs" to distribute.
Flexibility: Considers that a distributed approach does not allow
domains without physical infrastructure (e.g., without BGP or
BGP-LS) to advertise resource capabilities and networking
resources. Such procedures consist in deploying and configuring
physical peering points for these domains.
Complexity: Refers to the discovery mechanism to pre-select
candidate domains, accounting for resources and capabilities,
necessary for an E2E network service deployment. An intrinsic
complexity exists in the process of assembling, logically
organizing, and enabling abstraction views of different
resources and capabilities in multi-domain scenarios.
Lachos & Rothenberg Expires January 14, 2021 [Page 5]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
5. Proposed Approach
The primary design goal for ALTO-based Broker-assisted Multi-domain
Orchestration is to discover resource and topology information from
different administrative domains involved in the federation, while
also safeguarding the privacy and autonomy of every domain.
In the architectural proposal shown in Figure 1, a broker component
is conceived to be working as coordinator of a set of MdOs. In
particular, the broker-assisted design consists of the following key
components: (i) Inter-domain Resource (IdR), (ii) Inter-domain
Topology (IdT), and (iii) ALTO Server.
BROKER COMPONENT
+--------------------------------------------+
| |
| +-----------------+ |
| | | |
XXXXXXXXXXXXXXXXXXX ALTO SERVER(s) | |
X | | + |
X | +----------------+\ |
X | / \ |
X | / \ |
X | +------+/+-------+ +---------------+ |
X | | Inter-domain | | Inter-domain | |
X | | Topology (IdT) | | Resource (IdR)| |
X | +-^------^-------+ +---^-------^---+ |
X | . . * * |
X +----.------.-----------------*-------*------+
X . . * *
X . . * *
+--X--------.---+********************* *
| | . *
| | .............+------------*---+
| MdO1 | | |
| +<------------->+ +---+
+---------------+ | MdO2 | |
| | |
Legend: +-+--------------+ |
XXX ALTO Protocol | |
... BGP/BGP-LS/REST | MdO(n) |
*** UNIFY/TOSCA/ETSI-NFV +------------------+
Figure 1: Broker-assisted Multi-operator Network Architecture
Lachos & Rothenberg Expires January 14, 2021 [Page 6]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
5.1. Inter-domain Resource (IdR) Component
It creates a hierarchical database that contains inter-domain
resource information such as resource availability (i.e., CPU,
memory, and storage), Virtual Network Functions (VNFs) and Physical
Network Functions (PNFs) supported and Service Access Points (SAPs)
to access those resources. UNIFY [UNIFY.NFFG], TOSCA [TOSCA], ETSI-
NFV [ETSI-NFV-MANO], among other data models can be used to create
the interface between IdR and MdOs.
5.2. Inter-domain Topology (IdT) Component
A hierarchical TED (Traffic Engineering Database) that contains
inter-domain network topology information including additional key
parameters (e.g., throughput and latency of links). This information
can be retrieved from each MdO through BGP-LS or REST interfaces.
5.3. ALTO Server Functionalities
The ALTO server component is the core of the broker layer. Multiple
logically centralized ALTO servers use the information collected from
IdR and IdT components to create and provide abstract maps with a
simplified view, yet enough information about MdOs involved in the
federation. This information includes domain-level topology,
resource availability (i.e., CPU, memory, and storage), PNF/VNF
capabilities, and SAPs.
As an ALTO client, each MdO sends ALTO service queries to the ALTO
server. This server provides aggregated inter-domain information
exposed as set ALTO base services defined in [RFC7285], e.g., Network
Map, Cost Map and ALTO extension services, e.g., Property
Map [DRAFT-PM], Multi-Cost Map [RFC8189], Path Vector [DRAFT-PV].
For example, when a source MdO receives a customer service request,
it checks whether or not it can deliver the full service. If it is
unable to do so, the MdO consumes from the ALTO Server the Property
Map service to have a clear global view of the resource information
offered by other MdOs. This information allows discovering which
candidate MdOs may be contacted to deliver the remaining requirements
of a requested end-to-end service deployment. The connectivity
information among discovered MdOs can be retrieved by a Cost Map
service, responding, for instance, a path vector with the AS-level
topology distance between the source MdO and candidate MdOs.
Lachos & Rothenberg Expires January 14, 2021 [Page 7]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
5.4. Filtered Cost Map Extension
The ALTO server MUST provide connectivity information for every SG
link in the SG path for an E2E requirement. This information is the
AS-level topology distance in the form of path vector, and it
includes all possible ways for each (source node, destination node)
pair in the SG link.
In this section, we introduce a non-normative overview of the
Filtered Cost Map defined in Section 6.1 of [DRAFT-PV] [1].
The specifications for the "Media Types", "HTTP method",
"Capabilities" and "Uses" (described in Section 6.1 of [DRAFT-PV]
[2]) are unchanged.
5.4.1. Accept Input Parameters
The ReqFilteredCostMap object in Section 6.1.2 of [DRAFT-PV] [3] is
extended as follow:
object {
NFFG sg;
} ReqFilteredCostMap;
object {
JSONString nfs<1..*>;
JSONString saps<1..*>;
NextHops sg_links<1..*>;
REQs reqs<1..*>;
} NFFG;
object {
JSONNumber id;
JSONString src-node;
JSONString dst-node;
} NextHops;
object {
JSONString id;
JSONString src-node;
JSONString dst-node;
JSONNumber sg-path<1..*>;
} REQs;
Lachos & Rothenberg Expires January 14, 2021 [Page 8]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
sg: If present, the ALTO Server MUST allow the request input to
include an SG with a formatted body as an NFFG object. An NFFG
object contains NFs, SAPs, SG links representing logical
connections between NFs, SAPs or both and E2E requirements as a
list of ids of SG links.
It is worth noting that further versions of this draft will define a
more elaborated NFFG object to support extended parameters such as
monitoring parameters, resource requirements, etc.
5.4.2. Response
If the ALTO client includes the path vector cost mode in the "cost-
type" or "multi-cost-types" field of the input parameter, the
response for each SG link in each E2E requirement MUST be encoded as
a JSONArray of JSONArrays of JSONStrings. Anyone of the sub-arrays
indicates a potential candidate path calculated as the per-domain
topological distance corresponding to the amount of traversing
domains.
Moreover, as defined in Section 6.3.6 of [DRAFT-PV] [4], If an ALTO
client sends a request of the media type "application/alto-
costmapfilter+json" and accepts "multipart/related", the ALTO server
MUST provide path vector information along with the associated
Property Map information (e.g., entry points of the corresponding
foreign domains), in the same body of the response.
Section 5.5.2 gives an example of the Filtered Cost Map query and the
corresponding responses.
5.5. Examples of Message Exchange
This section list a couple of examples of the Property Map and
Filtered Cost Map queries and the corresponding responses. These
responses are based on the information in Table 1 and Table 2 of a
use case implementation described in Appendix A.
5.5.1. Property Map Service
In this example, the ALTO client wants to retrieve the entire
Property Map for PID entities with the "entry-point", "cpu", "mem",
"storage", "port" and "nf" properties.
o HTTP Request:
Lachos & Rothenberg Expires January 14, 2021 [Page 9]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
GET /propmap/full/inet-ucmspn HTTP/1.1
Host: alto.example.com
Accept: application/alto-propmap+json,application/alto-error+json
o HTTP Response:
HTTP/1.1 200 OK
Content-Length: ###
Content-Type: application/alto-propmap+json
{
"property-map": {
"pid:AS1": {
"entry-point": [ "http://172.25.0.10:8888/escape" ],
"cpu": [ "50.0" ],
"mem": [ "60.0" ],
"storage": [ "70.0" ],
"port": [ "SAP1" ],
"nf": [ "NF1", "NF3" ]
},
"pid:AS2": {
"entry-point": [ "http://172.26.0.10:8888/escape" ],
"cpu": [ "10.0" ],
"mem": [ "20.0" ],
"storage": [ "30.0" ],
"nf": [ "NF2" ]
},
"pid:AS3": {
"entry-point": [ "http://172.27.0.10:8888/escape" ],
"cpu": [ "80.0" ],
"mem": [ "90.0" ],
"storage": [ "100.0" ],
"port": [ "SAP2" ],
"nf": [ "NF1", "NF3" ]
}
}
}
5.5.2. Filtered Cost Map Service
The following example uses the Filtered Cost Map service to request
the path vector for a given E2E requirement. The SG request
information in Table 2 is used to describe the service, and it is
composed of three NFs (NF1, NF2, and NF3) and two SAPs (SAP1 and
SAP2). Links connecting the NFs and SAPs ("sg_links" tag) are also
Lachos & Rothenberg Expires January 14, 2021 [Page 10]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
included, followed by an E2E requirement ("reqs" tag) with
information about the order in which NFs are traversed from SAP1 to
SAP2.
Note that the request accepts "multipart/related" media type. This
means the ALTO server will include associated property information in
the same response.
o HTTP Request:
POST /costmap/pv HTTP/1.1
Host: alto.example.com
Accept: multipart/related, application/alto-costmap+json,
application/alto-propmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-costmapfilter+json
{
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
"sg": {
"nfs": [ "NF1", "NF2", "NF3" ],
"saps": [ "SAP1", "SAP2" ],
"sg_links":[
{
"id": 2,
"src-node": "SAP1",
"dst-node": "NF1",
},
{
"id": 2,
"src-node": "NF1",
"dst-node": "NF2",
},
{
"id": 3,
"src-node": "NF2",
"dst-node": "NF3",
},
{
"id": 4,
"src-node": "NF3",
"dst-node": "SAP2",
}
Lachos & Rothenberg Expires January 14, 2021 [Page 11]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
],
"reqs": [
{
"id": 1,
"src-node": "SAP1",
"dst-node": "SAP2",
"sg-path": [ 1, 2, 3, 4 ]
}
]
}
}
o HTTP Response: The ALTO server returns connectivity information
for the E2E requirement provided by the ALTO Client request of the
above example.
For each SG link in the E2E requirement (SAP1->NF1, NF1->NF2,
NF2->NF3, NF3->SAP2), the ALTO server returns sub-arrays
indicating potential candidate paths calculated as the AS-level
topological distance corresponding to the amount of traversing
domains.
Also, the response includes Property Map information for each
element in the path vector. In this case, it is retrieved a
Property Map with the "entry-point" property, i.e., the URL of the
MdO entry point for the corresponding network.
HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: multipart/related; boundary=example
--example
Content-Type: application/alto-endpointcost+json
{
"meta": {
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
},
"cost-map": {
"SAP1": {
"SAP2": {
Lachos & Rothenberg Expires January 14, 2021 [Page 12]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
"SAP1": {
"NF1": [
[ "AS1" ], [ "AS1", "AS2", "AS3" ]
]
},
"NF1": {
"NF2": [
[ "AS1", "AS2" ], [ "AS3", "AS2" ]
]
},
"NF2": {
"NF3": [
[ "AS2", "AS1" ], [ "AS2", "AS3" ]
]
},
"NF3": {
"SAP2": [
[ "AS1", "AS2", "AS3" ], [ "AS3" ]
]
}
}
}
}
}
--example
Content-Type: application/alto-propmap+json
{
"property-map": {
"pid:AS1": { "entry-point": "http://172.25.0.10:8888/escape" },
"pid:AS2": { "entry-point": "http://172.26.0.10:8888/escape" },
"pid:AS3": { "entry-point": "http://172.27.0.10:8888/escape" }
}
}
--example--
6. Discussion
In this section, we analyze the benefits and open issues in our
broker-assisted architecture.
Lachos & Rothenberg Expires January 14, 2021 [Page 13]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
6.1. Benefits
The broker-assisted orchestration has numerous benefits, such as:
o Avoid the distribution of topology and resource information in a
peer-to-peer fashion (MdO-to-MdO)
o The (abstracted) information and offered resources, services are
maintained in each local MdO.
o Allow domains without physical infrastructure (hence without BGP
or BGP-LS) to advertise their capabilities.
o An ALTO-based privacy-preserving information model to provide
computing, storage, and networking resource info.
o An MdO discovery method to determine the underlying network graph
and a potential set of paths before bilateral negotiation between
MdOs is started.
6.2. Open Issues
Although the broker-assisted information exchange has several
advantages, it also raises some questions which we try to answer from
our lessons learned.
o What kind of organization will manage and support the operation of
a broker entity? If a broker is used to exchange information,
then how does one ensure that the data delivered amongst the
operators by this 3rd party has not been changed?
* The broker entity must be trusted by each operator since it
stores and handles sensitive information. For example, future
deployment of SDN at IXPs can be used as a trusted third-party
platform to support rich business models between different
operators [DRAFT-HHSFC].
o In the case of peer-to-peer information exchange model, an MdO
failure concerns only the domain where the failure occurs, other
peers can perform the information exchange without any limitation.
However, If any error occurs in the broker entity the information
exchange among all involved ASes will be impacted. How avoid this
single point of failure?
* The broker entity maintains a centralized database. Local
restoration/replication options may be applied.
Lachos & Rothenberg Expires January 14, 2021 [Page 14]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
o The MdO information exchange depends on the policies. Operators
have a preference to share a different view about its compute and
network resources towards different operators. For example, a
detailed view for the operators that are belonging to same
operator group and a high-level information towards the other
operators. How is the fine-grained/coarse-grained information
exchange handled?.
* It requires much more complex database handling and information
exchange with the MdOs depending on the policies.
7. IANA Considerations
This document includes no request to IANA.
8. Security Considerations
TBD.
9. Acknowledgments
This work is supported by the Innovation Center of Ericsson S.A.,
Brazil (grant agreement UNI.64).
Thank you to Robert Szabo (Ericsson Research, Hungary) for the
contribution and substantial feedback and suggestions in this
document.
Many thanks to Richard Yang, Dawn Chan, Jensen Zhang, Shawn Lin, Qiao
Xiang, Sabine Randriamasy for their feedback on this draft.
10. References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997,
<http://xml.resource.org/public/rfc/html/rfc2119.html>.
[RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
"Application-Layer Traffic Optimization (ALTO) Protocol",
RFC 7285, DOI 10.17487/RFC7285, September 2014,
<https://www.rfc-editor.org/info/rfc7285>.
Lachos & Rothenberg Expires January 14, 2021 [Page 15]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
[RFC8189] Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
Application-Layer Traffic Optimization (ALTO)", RFC 8189,
DOI 10.17487/RFC8189, October 2017,
<https://www.rfc-editor.org/info/rfc8189>.
[RFC8459] Dolson, D., Homma, S., Lopez, D., and M. Boucadair,
"Hierarchical Service Function Chaining (hSFC)", RFC 8459,
DOI 10.17487/RFC8459, September 2018,
<https://www.rfc-editor.org/info/rfc8459>.
10.2. Informative References
[DRAFT-HHSFC]
Li, G., Li, G., Xu, Q., Zhou, H., and B. Feng, "Hybrid
Hierarchical Multi-Domain Service Function chaining",
draft-li-sfc-hhsfc-08 (work in progress), March 2020.
[DRAFT-MD-VIRT]
Bernardos, C., Contreras, L., Vaishnavi, I., Szabo, R.,
Li, X., Paolucci, F., Sgambelluri, A., Martini, B.,
Valcarenghi, L., Landi, G., Andrushko, D., and A. Mourad,
"Multi-domain Network Virtualization", draft-bernardos-
nfvrg-multidomain-05 (work in progress), September 2018.
[DRAFT-PM]
Roome, W., Randriamasy, S., Yang, Y., Zhang, J., and K.
Gao, "Unified Properties for the ALTO Protocol", draft-
ietf-alto-unified-props-new-11 (work in progress), March
2020.
[DRAFT-PV]
Gao, K., Randriamasy, S., Yang, Y., and J. Zhang, "ALTO
Extension: Path Vector", draft-ietf-alto-path-vector-10
(work in progress), March 2020.
[ETSI-NFV-DEF]
ETSI, "Network Functions Virtualisation (NFV); Terminology
for Main Concepts in NFV V1.3.1", Jan 2018,
<https://docbox.etsi.org/isg/nfv/open/Publications_pdf/
Specs-Reports/NFV%20003v1.3.1%20-%20GR%20-%20Terminology%2
0for%20Main%20Concepts%20in%20NFV.pdf>.
[ETSI-NFV-IFA028]
ETSI, "Report on architecture options to support multiple
administrative domains V3.1.1", Jan 2018,
<http://www.etsi.org/deliver/etsi_gr/NFV-
IFA/001_099/028/03.01.01_60/gr_NFV-IFA028v030101p.pdf>.
Lachos & Rothenberg Expires January 14, 2021 [Page 16]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
[ETSI-NFV-MANO]
ETSI, "Network Functions Virtualisation (NFV) Management
and Orchestration V1.1.1", Dec 2014,
<http://www.etsi.org/deliver/etsi_gs/NFV-
MAN/001_099/001/01.01.01_60/gs_NFV-MAN001v010101p.pdf>.
[H2020-5G-TRANSFORMER]
H2020, "5G-Transformer -- 5G Mobile Transport Platform for
Vertical", 2017, <http://5g-transformer.eu/>.
[H2020.5GEX]
Bernardos, C., Dugeon, O., Galis, A., Morris, D., Simon,
C., and R. Szabo, "5G Exchange (5GEx)--Multi-domain
Orchestration for Software Defined Infrastructures",
focus vol. 4, no.5, p.2, 2015.
[H2020.5GEX.ESCAPE]
5GEx Project, "ESCAPE: Extensible Service ChAin
Prototyping Environment", 2015,
<https://github.com/5GExchange/escape>.
[ICAF] Demchenko, Y., Makkes, M., Strijkers, R., Ngo, C., and C.
Laat, "Intercloud Architecture Framework for Heterogeneous
Multi-Provider Cloud based Infrastructure Services
Provisioning", International Journal of Next-Generation
Computing vol. 4, no.2, 2013.
[PPP-5:2013]
5G-PPP, "Advanced 5G Network Infrastructure for the Future
Internet", 2013, <https://5g-ppp.eu/wp-
content/uploads/2014/02/Advanced-5G-Network-
Infrastructure-PPP-in-H2020_Final_November-2013.pdf>.
[T-NOVA] FP7 project T-NOVA, "T-NOVA Project, Network Functions as
a Service over Virtualised Infrastructures", 2014,
<http://www.t-nova.eu/>.
[TELEFONICA.NET.TOPO]
Telefonica I+D, "Netphony-Topology", 2016,
<https://github.com/telefonicaid/netphony-topology>.
[TOSCA] OASIS, "TOSCA: Topology and Orchestration Specification
for Cloud Applications V1.0", 2013, <http://docs.oasis-
open.org/tosca/TOSCA/v1.0/os/TOSCA-v1.0-os.pdf>.
Lachos & Rothenberg Expires January 14, 2021 [Page 17]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
[UNIFY.NFFG]
UNIFY Deliverable D3.2a, "Network Function Forwarding
Graph specification", 2015, <http://www.fp7-
unify.eu/files/fp7-unify-eu-docs/Results/Deliverables/
UNIGY_D3.2a_NFFG%20Specification.pdf>.
[VITAL] VITAL PROJECT H2020, "VITAL -- VIrtualized hybrid
satellite-TerrestriAl systems for resilient and fLexible
future networks", 2015, <http://www.ict-vital.eu/>.
10.3. URIs
[1] https://tools.ietf.org/html/draft-ietf-alto-path-vector-
02#section-6.1
[2] https://tools.ietf.org/html/draft-ietf-alto-path-vector-
02#section-6.1
[3] https://tools.ietf.org/html/draft-ietf-alto-path-vector-
02#section-6.1.2
[4] https://tools.ietf.org/html/draft-ietf-alto-path-vector-
02#section-6.3.6
Appendix A. Proof of Concept Use Case Implementation
A strawman use case scenario has been implemented following the
architectural proposal of the 5GEx project [H2020.5GEX]. It refers
to an E2ENS orchestration involving three administrative domains.
For reproducibility purposes, all supporting codes are publicly
available in our research group repository:
https://intrig.dca.fee.unicamp.br:8865/intrig-unicamp/alto-based-
broker-assisted-mdo
As shown in Figure 2, each administrative domain has an MdO (MdO-AS1,
MdO-AS2, and MdO-AS3) to coordinate resource and/or service
orchestration at multi-operator level via interface I2 APIs. For the
orchestration within the same administrative domain, each MdO uses
emulated DOs with emulated I3 interfaces, since no data-plane is
present. DOs use static configuration files to load local
information about resources (I3-RC) and topology (I3-RT). The
different MdO components are based on existing open source tools such
as ESCAPE [H2020.5GEX.ESCAPE] (Service/Resource Orchestrator) and
Netphony-topology [TELEFONICA.NET.TOPO] (Resource Topology) and run
in Docker containers on a single computer. Besides, MdOs expose I1
interfaces to the tenants who request services and/or slices which
should follow a Network Function Forwarding Graph (NFFG) [UNIFY.NFFG]
format.
Lachos & Rothenberg Expires January 14, 2021 [Page 18]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
In case of the broker layer, the IdR and IdT components use a UNIFY
Virtualizer API [UNIFY.NFFG] (broker-based I2-RC API) and a REST API
(broker-based I2-RT API) respectively, in order to create the
hierarchical databases. Regarding the IdT, the administrative domain
2 is a transit provider so that the domain-level topology computed
is: AS1-AS2-AS3. From the inter-domain information are created the
two different ALTO Map Services: (i) Property Map and (ii) Cost Map.
Lachos & Rothenberg Expires January 14, 2021 [Page 19]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
+----------------------------------------+
| +---------------+ BROKER LAYER|
XXXXXXXXXXXXXX ALTO Server | |
X | +--------+----+-+ |
X | / \ |
X | +-----------+/+--+ +-\-------------+ |
X | | Inter-domain | | Inter-domain | |
+-----------+ X | | Topology (IdT) | | Resource (IdR)| |
| Service | X | +----------------+ +---------------+ |
| Graph (SG)| X +---------^-^----------------^--^--------+
| Request | X * * ............. .
+-----+-----+ X * * . ..............
| X * * . . MdO-AS3
I1| X * * . . +--------------+
| X MdO-AS1 * * . . | |
+-----|---------X-----------+ * * . . | MdO-AS2 |
| | | * * . . +---------------+ |
| +---v-------------------+ | * * . . | +-----------+ | |
| | | | * * . . | | | | |
| | Network Service Orch.| | * * . . | | NSO | | |
| | (NSO) | | * * . . | | | | |
| +-----------------------+ | * * . . | +-----------+ | |
| | * * . . | | |
| +---------+ | * * . . | +---+ | |
| | Resource........................... . | | | | |
| | Topoloy | | * * .......RT | | |
| | (RT) | +-----------+ | * * | | | | |
| +---------+ |Resource | | * * | +---+ +---+ | |
| |Orch. | | * ********************** | | |
| |(RO) ****** | |RO | +-+
| +-----------+ | | | | |
| |<------------->| +---+ |
+---------------------------+ I2 +-----+---------+
/ \ |
I3/ \ |I3
+-------+---+ +-----------+ +-----------+
| Domain | | Domain | | Domain |
| Orch (DO) | | Orch (DO) | | Orch (DO) |
+-----------+ +-----------+ +-----------+
Legend:
XXX ALTO Protocol
... broker-based I2-RT API
*** broker-based I2-RC API
Figure 2: Broker-assisted 5GEx Info Exchange
Lachos & Rothenberg Expires January 14, 2021 [Page 20]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
The Property Map includes property values grouped by Autonomous
System (AS). Such values are SAPs, NFs and the 5GEx Entry Point
(e.g., the URL of the ESCAPE orchestrator). An example of the
Property Map in our prototype is:
+-----+------------+-------+--------------+-----+-----+-------+-----+
| | Entry | Port | Capabilities | CPU | MEM | Stor | ... |
| | Point | SAP | | | | age | |
+-----+------------+-------+--------------+-----+-----+-------+-----+
| AS1 | http://... | SAP1 | {NF1, NF3} | 50 | 60 | 70 | ... |
| AS2 | http://... | - | {NF2} | 10 | 20 | 30 | ... |
| AS3 | http://... | SAP2 | {NF1, NF3} | 80 | 90 | 100 | ... |
+-----+------------+-------+--------------+-----+-----+-------+-----+
Table 1: ALTO Property Map
The Cost Map defines a path vector as an array of ASes, representing
the AS-level topological distance for a given E2ENS request. Path
vector constraints (as described in the Multi-Cost Map [RFC8189]) can
be applied to restricts the response to costs that satisfy a list of
simple predicates.
Table 2 below shows a brief example of an SG request and its path
vector response containing a list of potential providers to be
traversed to deliver such service. Every AS path is computed from
the inter-domain topology information in the IdT module. In our
scenario, MdO-AS2 is a transit provider, so that the domain-level
topology map is AS1<->AS2<->AS3.
+--------------------+----------------------------------------------+
| Service Graph (SG) | Path(s) Vector |
| Request | |
+--------------------+----------------------------------------------+
| SAP1->NF1->NF2->NF | 1:{AS1:SAP1->AS1:NF1->AS2:NF2->AS3:NF3->AS3: |
| 3->SAP2 | SAP2} |
| | 2:{AS1:SAP1->AS1:NF1->AS2:NF2->AS1:NF3->AS2- |
| | >AS3:SAP2} |
| | 3:{AS1:SAP1->AS2->AS3:NF1->AS2:NF2->AS3:NF3- |
| | AS3:SAP2} |
| | 4:{AS1:SAP1->AS2->AS3:NF1->AS2:NF2->AS1:NF3- |
| | >AS2->AS3:SAP2} |
+--------------------+----------------------------------------------+
Table 2: ALTO Cost Map
Lachos & Rothenberg Expires January 14, 2021 [Page 21]
Internet-Draft ALTO-based Multi-domain Orchestration July 2020
Authors' Addresses
Danny Alex Lachos Perez
University of Campinas
Av. Albert Einstein 400
Campinas, Sao Paulo 13083-970
Brazil
Email: dlachosp@dca.fee.unicamp.br
URI: https://intrig.dca.fee.unicamp.br/danny-lachos/
Christian Esteve Rothenberg
University of Campinas
Av. Albert Einstein 400
Campinas, Sao Paulo 13083-970
Brazil
Email: chesteve@dca.fee.unicamp.br
URI: https://intrig.dca.fee.unicamp.br/christian/
Lachos & Rothenberg Expires January 14, 2021 [Page 22]
