Internet DRAFT - draft-ietf-detnet-controller-plane-framework

draft-ietf-detnet-controller-plane-framework







Network Working Group                                           A. Malis
Internet-Draft                                               Independent
Intended status: Informational                              X. Geng, Ed.
Expires: 25 March 2024                                           M. Chen
                                                                  Huawei
                                                                  F. Qin
                                                            China Mobile
                                                                B. Varga
                                                                Ericsson
                                                            C. Bernardos
                                        Universidad Carlos III de Madrid
                                                       22 September 2023


      Deterministic Networking (DetNet) Controller Plane Framework
            draft-ietf-detnet-controller-plane-framework-05

Abstract

   This document provides a framework overview for the Deterministic
   Networking (DetNet) controller plane.  It discusses concepts and
   requirements for DetNet controller plane which could be basis for
   future solution specification.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 25 March 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.






Malis, et al.             Expires 25 March 2024                 [Page 1]

Internet-Draft           DetNet Controller Plane          September 2023


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  DetNet Controller Plane Requirements  . . . . . . . . . . . .   4
     2.1.  DetNet Control Plane Requirements . . . . . . . . . . . .   4
     2.2.  DetNet Management Plane Requirements  . . . . . . . . . .   5
     2.3.  Requirements For Both Planes  . . . . . . . . . . . . . .   5
   3.  DetNet Control Plane Architecture . . . . . . . . . . . . . .   6
     3.1.  Distributed Control Plane and Signaling Protocols . . . .   6
     3.2.  SDN/Fully Centralized Control Plane . . . . . . . . . . .   7
     3.3.  Hybrid Control Plane (partly centralized, partly
           distributed)  . . . . . . . . . . . . . . . . . . . . . .   7
   4.  DetNet Control Plane for DetNet Mechanisms  . . . . . . . . .   8
     4.1.  Explicit Paths  . . . . . . . . . . . . . . . . . . . . .   8
     4.2.  Resource Reservation  . . . . . . . . . . . . . . . . . .   8
     4.3.  PREOF Support . . . . . . . . . . . . . . . . . . . . . .   9
     4.4.  Data Plane specific considerations  . . . . . . . . . . .   9
       4.4.1.  DetNet in an MPLS Domain  . . . . . . . . . . . . . .   9
       4.4.2.  DetNet in an IP Domain  . . . . . . . . . . . . . . .  10
       4.4.3.  DetNet in a Segment Routing Domain  . . . . . . . . .  11
   5.  Management Plane Overview . . . . . . . . . . . . . . . . . .  11
     5.1.  Provisioning  . . . . . . . . . . . . . . . . . . . . . .  11
     5.2.  DetNet Operations, Administration and Maintenance
           (OAM) . . . . . . . . . . . . . . . . . . . . . . . . . .  12
       5.2.1.  OAM for Performance Monitoring (PM) . . . . . . . . .  12
       5.2.2.  OAM for Connectivity and Fault/Defect Management
               (CFM) . . . . . . . . . . . . . . . . . . . . . . . .  12
   6.  Multidomain Aspects . . . . . . . . . . . . . . . . . . . . .  12
   7.  Gap Analysis  . . . . . . . . . . . . . . . . . . . . . . . .  13
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
   10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  14
   11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  14
     11.2.  Informative References . . . . . . . . . . . . . . . . .  16
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  18





Malis, et al.             Expires 25 March 2024                 [Page 2]

Internet-Draft           DetNet Controller Plane          September 2023


1.  Introduction

   Deterministic Networking (DetNet) provides the capability to carry
   specified unicast and/or multicast data flows for real-time
   applications with extremely low data loss rates and bounded latency
   within a network domain.  As defined in [RFC8655], techniques used to
   provide DetNet capability include reserving data plane resources for
   individual (or aggregated) DetNet flows in some or all of the
   intermediate nodes along the path of the flow, providing explicit
   routes for DetNet flows that do not immediately change with the
   network topology, and distributing data from DetNet flow packets over
   time and/or space to ensure delivery of each packet’s data in spite
   of the loss of a path.


   DetNet data plane is defined in a set of documents that are anchored
   by the DetNet Data Plane Framework[RFC8938] (and the associated
   DetNet MPLS defined in [RFC8964] and DetNet IP defined in [RFC8939]
   and other data plane specifications defined in [RFC9023], [RFC9024],
   [RFC9025], [RFC9037] and [RFC9056])

   While the Detnet Architecture and Data Plane documents are primarily
   concerned with data plane operations, they do contain some
   requirements for functions that would be required in order to
   automate DetNet service provisioning and monitoring via a DetNet
   controller plane.  The purpose of this document is to gather these
   requirements into a single document and discuss how various possible
   DetNet controller plane architectures could be used to satisfy these
   requirements, while not providing the protocol details for a DetNet
   controller plane solution.  Such controller plane protocol solutions
   will be the subject of subsequent documents.

   Note that in the DetNet overall architecture, the controller plane
   includes what are more traditionally considered separate control and
   management planes.  Traditionally, the management plane is primarily
   involved with fault management, configuration management and
   performance management(sometimes accounting management and security
   management is also considered in the management plane, but not in the
   scope of this document). , while the control plane is primarily
   responsible for the instantiation and maintenance of flows, MPLS
   label allocation and distribution, and active in-band or out-of-band
   signaling to support DetNet functions.  In the DetNet architecture,
   all of this functionality is combined into a single Controller Plane.
   See Section 4.4.2 of [RFC8655] and the aggregation of Control and
   Management planes in [RFC7426] for further details.






Malis, et al.             Expires 25 March 2024                 [Page 3]

Internet-Draft           DetNet Controller Plane          September 2023


1.1.  Terminology

   This document uses the terminology established in the DetNet
   Architecture [RFC8655], and the reader is assumed to be familiar with
   that document and its terminology.

   The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
   “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and
   “OPTIONAL” in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174]when, and only when, they appear in all
   capitals, as shown here.

2.  DetNet Controller Plane Requirements

   Other DetNet documents, including [RFC8655] and [RFC8938], contain
   requirements for the Controller Plane.  For convenience, these
   requirements have been compiled here.  These requirements have been
   organized into 3 groups, including: requirements primarily applicable
   to control plane, requirements primarily applicable to management
   plane and requirements applicable to both planes.

2.1.  DetNet Control Plane Requirements

   The primary requirements for the DetNet Control Plane include:

   *  Support the dynamic creation, modification, and deletion of DetNet
      flows.  This may include some or all of explicit path
      determination, link bandwidth reservations, restricting flows to
      specific links (e.g., IEEE 802.1 Time-Sensitive Networking (TSN)
      links), node buffer and other resource reservations, specification
      of required queuing disciplines along the path, ability to manage
      bidirectional flows, etc., as needed for a flow.

   *  Support DetNet flow aggregation and de-aggregation via the ability
      to dynamically create and delete flow aggregates (FAs), and be
      able to modify existing FAs by adding or deleting participating
      flows.

   *  Allow flow instantiation requests to originate in an end
      application (via an Application Programming Interface (API), via
      static provisioning, or via a dynamic control plane, such as a
      centralized SDN controller or distributed signaling protocols.
      See Section 3 for further discussion of these options.

   *  In the case of the DetNet MPLS data plane, manage DetNet Service
      Label (S-Label), Forwarding Label (F-Label), and Aggregation Label
      (A-Label) [RFC8964] allocation and distribution.




Malis, et al.             Expires 25 March 2024                 [Page 4]

Internet-Draft           DetNet Controller Plane          September 2023


   *  Also in the case of the DetNet MPLS data plane, support the DetNet
      service sub-layer, which provides DetNet service functions such as
      protection and reordering through the use of packet replication,
      duplicate elimination, and packet ordering functions (PREOF).

   *  Support queue control techniques defined in Section 4.5 of
      [RFC8655] and [I-D.ietf-detnet-bounded-latency] that require time
      synchronization among network nodes.

   *  Advertise static and dynamic node and link resources such as
      capabilities and adjacencies to other network nodes (for dynamic
      signaling approaches) or to network controllers (for centralized
      approaches).

   *  Scale to handle the number of DetNet flows expected in a domain
      (which may require per-flow signaling or provisioning).

   *  Provision flow identification information at each of the nodes
      along the path.  Flow identification may differ depending on the
      location in the network and the DetNet functionality (e.g. transit
      node vs. relay node).

2.2.  DetNet Management Plane Requirements

   The primary requirements of the DetNet Management Plane are that it
   must be able to:

   *  Monitor the performance of DetNet flows and nodes to ensure that
      they are meeting required objectives, both proactively and on-
      demand.

   *  Support DetNet flow continuity check and connectivity verification
      functions.

   *  Support testing and monitoring of packet replication, duplicate
      elimination, and packet ordering functionality in the DetNet
      domain.

2.3.  Requirements For Both Planes

   The following requirements apply to both the DetNet Controller and
   Management Planes:

   *  Operate in a converged network domain that contains both DetNet
      and non-DetNet flows.

   *  Adapt to DetNet domain topology changes such as links or nodes
      failures (fault recovery/restoration), additions and removals.



Malis, et al.             Expires 25 March 2024                 [Page 5]

Internet-Draft           DetNet Controller Plane          September 2023


3.  DetNet Control Plane Architecture

   As noted in the Introduction, the DetNet control plane is responsible
   for the instantiation and maintenance of flows, allocation and
   distribution of flow related information (e.g., MPLS label), and
   active in-band or out-of-band information distribution to support
   these functions.

   The following sections define three types of DetNet control plane
   architectures: a fully distributed control plane utilizing dynamic
   signaling protocols, a fully centralized SDN-like control plane, and
   a hybrid control plane containing both distributed protocols and
   centralized controlling . This document describes the various
   information exchanges between entities in the network for Each type
   of these architectures and the corresponding advantages and
   disadvantages.

   In each of the following sections, there are examples to illustrate
   possible mechanisms that could be used in each type of the
   architectures.  They are not meant to be exhaustive or to preclude
   any other possible mechanism that could be used in place of those
   used in the examples.

3.1.  Distributed Control Plane and Signaling Protocols

   In a fully distributed configuration model, User-to-Network Interface
   (UNI) information is transmitted over a DetNet UNI protocol from the
   user side to the network side.Then UNI and network configuration
   information propagate in the network via distributed control plane
   signaling protocols.  Such a DetNet UNI protocol is not necessary in
   case that the End-systems are DetNet capable.

   Taking an RSVP-TE MPLS network as an example, where end systems are
   not part of the DetNet domain:

   1.  Network nodes collects topology information and DetNet
       capabilities of the network nodes through IGP;

   2.  Ingress edge node receives a flow establishment request from the
       UNI and calculates one or more valid path(s);

   3.  The ingress node sends a PATH message with an explicit route
       through RSVP-TE [RFC3209].  After receiving the PATH message, the
       egress edge node sends a RESV message with the distributed label
       and resource reservation request.

   In this example, both IGP and RSVT-TE may request extensions for
   DetNet.



Malis, et al.             Expires 25 March 2024                 [Page 6]

Internet-Draft           DetNet Controller Plane          September 2023


3.2.  SDN/Fully Centralized Control Plane

   In the fully SDN/centralized configuration model, flow/UNI
   information is transmitted from a Centralized User Controller or from
   applications via an API/ northbound interface to a Centralized
   Controlle.  Network node configurations for DetNet flows are
   performed by the controller using a protocol such as NETCONF
   [RFC6241]/YANG [RFC6020] or PCE-CC [RFC8283].

   Take the following case as an example::

   1.  A Centralized Controller collects topology information and DetNet
       capabilities of the network nodes via NETCONF/YANG;

   2.  The Controller receives a flow establishment request from a UNI
       and calculates one or more valid path(s) through the network;

   3.  The Controller chooses the optimal path and configures the
       devices along that path for DetNet flow transmission via PCE-CC.

   Protocols in the above example may require extensions for DetNet.

3.3.  Hybrid Control Plane (partly centralized, partly distributed)

   In the hybrid model, controller and control plane protocols work
   together to provide DetNet services, and there are a number of
   possible combinations.

   In the following case, RSVP-TE and controller are used together:

   1.  Controller collects topology information and DetNet capabilities
       of the network nodes via an IGP and/or BGP-LS [RFC7752];

   2.  Controller receives a flow establishment request through API and
       calculates one or more valid path(s) through the network ;

   3.  Based on the calculation result, the Controller distributes flow
       path information to the ingress edge node and configures network
       nodes along the path with necessary DetNet information (e.g. for
       replication/duplicate elimination)

   4.  Using RSVP-TE, the ingress edge node sends a PATH message with an
       explicit route.  After receiving the PATH message, the egress
       edge node sends a RESV message with the distributed label and
       resource reservation request.






Malis, et al.             Expires 25 March 2024                 [Page 7]

Internet-Draft           DetNet Controller Plane          September 2023


   There are many other variations that could be included in a hybrid
   control plane.  The requested DetNet extensions for protocol in each
   possible case is for future work.

4.  DetNet Control Plane for DetNet Mechanisms

   This section discusses requested control plane features for DetNet
   mechanisms as defined in [RFC8655], including explicit path, resource
   reservation, service protection(PREOF).  Different DetNet service may
   implement part/all of them based on the requirements.

4.1.  Explicit Paths

   Explicit paths are required in DetNet to provide a stable forwarding
   service and guarantee that DetNet service is not impacted when the
   network topology changes.  The following features are necessary in
   control plane to implement explicit paths in DetNet:

   *  Path computation: DetNet explicit paths need to meet the SLA
      (Service Level Agreement) requirements of the application, which
      include bandwidth, maximum end- to-end delay, maximum end-to-end
      delay variation, maximum loss ratio, etc.  In a distributed
      network system, IGP with CSPF (Constrained Shortest Path First)
      may be used to compute a set of feasible paths for a DetNet
      service.  In a centralized network system, controller can compute
      paths satisfying the requirements of DetNet based on the network
      information collected from the DetNet domain.

   *  Path establishment: The computed path for the DetNet service has
      to be sent/configured/signaled to the network device, so the
      corresponding DetNet flow could pass through the network domain
      following the specified path.

4.2.  Resource Reservation

   DetNet flows are supposed to be protected from congestion, so
   sufficient resource reservation for DetNet service could protect
   service from congestion.  There are multiple types of resources in
   the network that could be allocated to DetNet flows, e.g., packet
   processing resource, buffer resource, and bandwidth of the output
   port.  The network resource requested by a specified DetNet service
   is determined by the SLA requirements and network capability.

   *  Resource Allocation: Port bandwidth is one of the basic attributes
      of a network device which is easy to obtain or calculate.  In
      current traffic engineering implementations, network resource
      allocation is synonymous with bandwidth allocation.  A DetNet flow
      is characterized with a traffic specification as defined in



Malis, et al.             Expires 25 March 2024                 [Page 8]

Internet-Draft           DetNet Controller Plane          September 2023


      [RFC9016], including attributes such as Interval, Maximum Packets
      Per Interval, and Maximum Payload Size.  The traffic specification
      describes the worst case, rather than the average case, for the
      traffic, to ensure that sufficient bandwidth and buffering
      resources are reserved to satisfy the traffic specification.
      However, in case of DetNet, resource allocation is more than
      simple bandwidth reservation.  For example, allocation of buffers
      and required queuing disciplines during forwarding may be required
      as well.  Furthermore, resources must be ensured to execute DetNet
      service sub-layer functions on the node, such as protection and
      reordering through the use of packet replication, duplicate
      elimination, and packet ordering functions (PREOF).

   *  Device configuration with or without flow discrimination: The
      resource allocation can be guaranteed by device configuration.
      For example, an output port bandwidth reservation can be
      configured as a parameter of queue management and the port
      scheduling algorithm.  When DetNet flows are aggregated, a group
      of DetNet flows share the allocated resource in the network
      device.  When the DetNet flows are treated independently, the
      device should maintains a mapping relationship between a DetNet
      flow and its corresponding resources.

4.3.  PREOF Support

   DetNet path redundancy is supported via packet replication, duplicate
   elimination, and packet ordering functions (PREOF).  A DetNet flow is
   replicated and goes through multiple networks paths to avoid packet
   loss caused by device or link failures.  In general, current control
   plane mechanisms that can be used to establish an explicit path,
   whether distributed or centralized, support point-to-point (P2P) and
   point-to-multipoint (P2MP) path establishment.  PREOF requires the
   ability to compute and establish a set of multiple paths (e.g.,
   multiple LSP segments in an MPLS network) from the point(s) of packet
   replication to the point(s) of packet merging and ordering.  Mapping
   of DetNet (member) flows to explicit path segments has to be ensured
   as well.  Protocol extensions will be required to support these new
   features.  Terminology will also be required to refer to this
   coordinated set of path segments (such as an “LSP graph” in case of
   DetNet MPLS data plane).

4.4.  Data Plane specific considerations

4.4.1.  DetNet in an MPLS Domain

   For the purposes of this document, “traditional MPLS” is defined as
   MPLS without the use of segment routing (see Section 4.4.3 for a
   discussion of MPLS with segment routing) or MPLS-TP [RFC5960].



Malis, et al.             Expires 25 March 2024                 [Page 9]

Internet-Draft           DetNet Controller Plane          September 2023


   In traditional MPLS domains, a dynamic control plane using
   distributed signaling protocols is typically used for the
   distribution of MPLS labels used for forwarding MPLS packets.  The
   dynamic signaling protocols most commonly used for label distribution
   are LDP [RFC5036], RSVP-TE, and BGP [RFC8277] (which enables BGP/
   MPLS-based Layer 3 VPNs [RFC4384] and Layer 2 VPNs [RFC7432]).

   Any of these protocols could be used to distribute DetNet Service
   Labels (S-Labels) and Aggregation Labels (A-Labels) [RFC8964].  As
   discussed in [RFC8938], S-Labels are similar to other MPLS service
   labels, such as pseudowire, L3 VPN, and L2 VPN labels, and could be
   distributed in a similar manner, such as through the use of targeted
   LDP or BGP.  If these were to be used for DetNet, they would require
   extensions to support DetNet-specific features such as PREOF,
   aggregation (A-Labels), node resource allocation, and queue
   placement.

   However, as discussed in Section 3.1, distributed signaling protocols
   may have difficulty meeting DetNet’s scalability requirements.  MPLS
   also allows SDN-like centralized label management and distribution as
   an alternative to distributed signaling protocols, using protocols
   such as PCEP and OpenFlow [OPENFLOW].

   PCEP, particularly when used as a part of PCE-CC, is a possible
   candidate protocol to use for centralized management of traditional
   MPLS-based DetNet domains.  However, PCE path calculation algorithms
   would need to be extended to include the location determination for
   PREOF nodes in a path, and the means to signal the necessary resource
   reservation and PREOF function placement information to network
   nodes.  See ((?I-D.ietf-pce-pcep-extension-for-pce-controller)) for
   further discussion of PCE-CC and PCEP for centralized control of an
   MPLS domain.

4.4.2.  DetNet in an IP Domain

   For the purposes of this document, “traditional IP” is defined as IP
   without the use of segment routing (see Section 4.4.3 for a
   discussion of IP with segment routing).  In a later revision of this
   document, this section will discuss possible protocol extensions to
   existing IP routing protocols such as OSPF, IS-IS, and BGP.  It
   should be noted that a DetNet IP data plane [RFC8939] is simpler than
   a DetNet MPLS data plane [RFC8964], and doesn’t support PREOF, so
   only one path per flow or flow aggregate is required.








Malis, et al.             Expires 25 March 2024                [Page 10]

Internet-Draft           DetNet Controller Plane          September 2023


4.4.3.  DetNet in a Segment Routing Domain

   Segment Routing [RFC8402] is a scalable approach to building network
   domains that provides explicit routing via source routing encoded in
   packet headers and it is combined with centralized network control to
   compute paths through the network.  Forwarding paths are distributed
   with associated policy to network edge nodes for use in packet
   headers.  As such, segment routing can be considered as a new data
   plane for both MPLS and IP.  It reduces the amount of network
   signaling associated with distributed signaling protocols such as
   RSVP-TE, and also reduces the amount of state in core nodes compared
   with that required for traditional MPLS and IP routing, as the state
   is now in the packets rather than in the routers.  This could be
   useful for DetNet, where a very large number of flows through a
   network domain are expected, which would otherwise require the
   instantiation of state for each flow traversing each node in the
   network.  However, further analysis is needed on the expected gain,
   as DetNet flows may require various type of DetNet specific resources
   as well.

   In a later revision of this document, this section will discuss the
   impact of DetNet on the Segment Routing Control and Management
   planes.  Note that the DetNet MPLS and IP data planes described in
   [RFC8964] and [RFC8939] were constructed to be compatible with both
   types of segment routing, SR-MPLS [RFC8660] and SRv6
   [I-D.ietf-6man-segment-routing-header].  However, as of this writing,
   traffic engineering and resource reservation for segment routing are
   currently unsolved problems.

   Editor’s note: this section may be collapsed to previous sections and
   listing MPLS segment routing in the MPLS section as one of the
   possible explicit routing techniques for MPLS, and do the same for
   IP.

5.  Management Plane Overview

   The Management Plane includes the ability to statically provision
   network nodes and to use OAM to monitor DetNet performance and detect
   outages or other issues at the DetNet layer.

5.1.  Provisioning

   Static provisioning in a Detnet network nodes will be performed via
   the use of appropriate YANG models, including [I-D.ietf-detnet-yang]
   and [I-D.ietf-detnet-topology-yang].






Malis, et al.             Expires 25 March 2024                [Page 11]

Internet-Draft           DetNet Controller Plane          September 2023


5.2.  DetNet Operations, Administration and Maintenance (OAM)

   This document covers the general considerations for OAM.


5.2.1.  OAM for Performance Monitoring (PM)

5.2.1.1.  Active PM

   Active PM is performed by injecting OAM packets into the network to
   estimate the performance of the network by measuring the performance
   of the OAM packets.  Adding extra traffic can affect the delay and
   throughput performance of the network, and for this reason active PM
   is not recommended for use in operational DetNet domains.  However,
   it is a useful test tool when commissioning a new network or during
   troubleshooting.

5.2.1.2.  Passive PM

   Passive PM monitors the actual service traffic in a network domain in
   order to measure its performance without having a detrimental affect
   on the network.  As compared to Active PM, Passive PM is much
   preferred for use in DetNet domains.

5.2.2.  OAM for Connectivity and Fault/Defect Management (CFM)

   The detailed requirements for connectivity and fault/defect detection
   and management in DetNet IP domain and DetNet MPLS domain are defined
   in respectively in [I-D.ietf-detnet-ip-oam] and
   [I-D.ietf-detnet-mpls-oam].

6.  Multidomain Aspects

   When there are multiple domains involved, one or multiple controller
   plane function (CFP) would have to collaborate to implement the
   requests received from the flow management entity (FME, as defined in
   þRFC8655]) as per-flow, per-hop behaviors installed in the DetNet
   nodes for each individual flow.  Adding multi-domain support might
   require some support at the CPF.  For example, CPFs sitting at
   different domains need to discover themselves, authenticate and
   negotiate per-hop behaviors.  Depending on the multi-domain support
   provided by the application plane, the controller plane might
   berelieved from some responsibilities (e.g., if the application plane
   is taking care of splitting what needs to be provided by each
   domain).






Malis, et al.             Expires 25 March 2024                [Page 12]

Internet-Draft           DetNet Controller Plane          September 2023


   If the case of RAW is considered, additional considerations for
   communications among per-domain PCEs and/or PSEs would be required,
   as these entities might not have the full visibility nor capability
   to act on the other domains (e.g., there are no multi-domain OAM
   solutions in place yet), limiting its capability to guarantee any
   given SLA.

7.  Gap Analysis

   In a later revision of this document, this section will contain a gap
   analysis of existing IETF control and management plane protocols not
   already discussed elsewhere in this document for their ability (or
   inability) to satisfy the requirements in Section 2, and discuss
   possible protocol extensions to existing protocols to fill the gaps,
   if any.

8.  IANA Considerations

   This document has no actions for IANA.

   Note to RFC Editor: this section may be removed on publication as an
   RFC.

9.  Security Considerations

   Editor's note: This section needs more details.

   The overall security considerations of DetNet are discussed in
   [RFC8655] and [I-D.ietf-detnet-security].  For DetNet networks that
   make use of Segment Routing (whether SR-MPLS or SRv6), the security
   considerations in [RFC8402] also apply.

   DetNet networks that make use of a centralized controller plane may
   be threatened by the loss of connectivity (whether accidental or
   malicious) between the central controller and the network nodes, and/
   or the spoofing of control messages from the controller to the
   network nodes.  This is important since such networks depend on
   centralized controllers to calculate flow paths and instantiate flow
   state in the network nodes.  For networks that use both DetNet and
   Segment Routing with a centralized controller, this would also
   include the calculation of SID lists and their installation in edge/
   border routers.









Malis, et al.             Expires 25 March 2024                [Page 13]

Internet-Draft           DetNet Controller Plane          September 2023


   In both cases, such threats may be mitigated through redundant
   controllers, the use of authentication between the controller(s) and
   the network nodes, and other mechanisms for protection against DOS
   attacks.  A mechanism for supporting one or more alternative central
   controllers and the ability to fail over to such an alternative
   controller will be required.

10.  Acknowledgments

   Thanks to Jim Guichard, Donald Eastlake, and Stewart Bryant for their
   review comments.

11.  References

11.1.  Normative References

   [I-D.ietf-detnet-bounded-latency]
              Finn, N., Le Boudec, J., Mohammadpour, E., Zhang, J., and
              B. Varga, "Deterministic Networking (DetNet) Bounded
              Latency", Work in Progress, Internet-Draft, draft-ietf-
              detnet-bounded-latency-10, 8 April 2022,
              <https://datatracker.ietf.org/doc/html/draft-ietf-detnet-
              bounded-latency-10>.

   [I-D.ietf-detnet-flow-information-model]
              Varga, B., Farkas, J., Cummings, R., Jiang, Y., and D.
              Fedyk, "DetNet Flow Information Model", Work in Progress,
              Internet-Draft, draft-ietf-detnet-flow-information-model-
              10, 15 May 2020, <http://www.ietf.org/internet-drafts/
              draft-ietf-detnet-flow-information-model-10.txt>.

   [I-D.ietf-detnet-ip-oam]
              Mirsky, G., Chen, M., and D. L. Black, "Operations,
              Administration and Maintenance (OAM) for Deterministic
              Networks (DetNet) with IP Data Plane", Work in Progress,
              Internet-Draft, draft-ietf-detnet-ip-oam-09, 1 August
              2023, <https://datatracker.ietf.org/doc/html/draft-ietf-
              detnet-ip-oam-09>.

   [I-D.ietf-detnet-mpls-oam]
              Mirsky, G., Chen, M., and B. Varga, "Operations,
              Administration and Maintenance (OAM) for Deterministic
              Networks (DetNet) with MPLS Data Plane", Work in Progress,
              Internet-Draft, draft-ietf-detnet-mpls-oam-13, 6 July
              2023, <https://datatracker.ietf.org/doc/html/draft-ietf-
              detnet-mpls-oam-13>.





Malis, et al.             Expires 25 March 2024                [Page 14]

Internet-Draft           DetNet Controller Plane          September 2023


   [I-D.ietf-detnet-security]
              Mizrahi, T. and E. Grossman, "Deterministic Networking
              (DetNet) Security Considerations", Work in Progress,
              Internet-Draft, draft-ietf-detnet-security-10, 30 May
              2020, <http://www.ietf.org/internet-drafts/draft-ietf-
              detnet-security-10.txt>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC7426]  Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
              Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
              Defined Networking (SDN): Layers and Architecture
              Terminology", RFC 7426, DOI 10.17487/RFC7426, January
              2015, <https://www.rfc-editor.org/info/rfc7426>.

   [RFC8174]  "".

   [RFC8402]  Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
              Decraene, B., Litkowski, S., and R. Shakir, "Segment
              Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
              July 2018, <https://www.rfc-editor.org/info/rfc8402>.

   [RFC8655]  Finn, N., Thubert, P., Varga, B., and J. Farkas,
              "Deterministic Networking Architecture", RFC 8655,
              DOI 10.17487/RFC8655, October 2019,
              <https://www.rfc-editor.org/info/rfc8655>.

   [RFC8938]  Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
              Bryant, "Deterministic Networking (DetNet) Data Plane
              Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020,
              <https://www.rfc-editor.org/info/rfc8938>.

   [RFC8939]  Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S.
              Bryant, "Deterministic Networking (DetNet) Data Plane:
              IP", RFC 8939, DOI 10.17487/RFC8939, November 2020,
              <https://www.rfc-editor.org/info/rfc8939>.

   [RFC8964]  Varga, B., Ed., Farkas, J., Berger, L., Malis, A., Bryant,
              S., and J. Korhonen, "Deterministic Networking (DetNet)
              Data Plane: MPLS", RFC 8964, DOI 10.17487/RFC8964, January
              2021, <https://www.rfc-editor.org/info/rfc8964>.







Malis, et al.             Expires 25 March 2024                [Page 15]

Internet-Draft           DetNet Controller Plane          September 2023


   [RFC9016]  Varga, B., Farkas, J., Cummings, R., Jiang, Y., and D.
              Fedyk, "Flow and Service Information Model for
              Deterministic Networking (DetNet)", RFC 9016,
              DOI 10.17487/RFC9016, March 2021,
              <https://www.rfc-editor.org/info/rfc9016>.

   [RFC9023]  Varga, B., Ed., Farkas, J., Malis, A., and S. Bryant,
              "Deterministic Networking (DetNet) Data Plane: IP over
              IEEE 802.1 Time-Sensitive Networking (TSN)", RFC 9023,
              DOI 10.17487/RFC9023, June 2021,
              <https://www.rfc-editor.org/info/rfc9023>.

   [RFC9024]  Varga, B., Ed., Farkas, J., Malis, A., Bryant, S., and D.
              Fedyk, "Deterministic Networking (DetNet) Data Plane: IEEE
              802.1 Time-Sensitive Networking over MPLS", RFC 9024,
              DOI 10.17487/RFC9024, June 2021,
              <https://www.rfc-editor.org/info/rfc9024>.

   [RFC9025]  Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
              Bryant, "Deterministic Networking (DetNet) Data Plane:
              MPLS over UDP/IP", RFC 9025, DOI 10.17487/RFC9025, April
              2021, <https://www.rfc-editor.org/info/rfc9025>.

   [RFC9037]  Varga, B., Ed., Farkas, J., Malis, A., and S. Bryant,
              "Deterministic Networking (DetNet) Data Plane: MPLS over
              IEEE 802.1 Time-Sensitive Networking (TSN)", RFC 9037,
              DOI 10.17487/RFC9037, June 2021,
              <https://www.rfc-editor.org/info/rfc9037>.

   [RFC9056]  Varga, B., Ed., Berger, L., Fedyk, D., Bryant, S., and J.
              Korhonen, "Deterministic Networking (DetNet) Data Plane:
              IP over MPLS", RFC 9056, DOI 10.17487/RFC9056, October
              2021, <https://www.rfc-editor.org/info/rfc9056>.

11.2.  Informative References

   [I-D.ietf-6man-segment-routing-header]
              Filsfils, C., Dukes, D., Previdi, S., Leddy, J.,
              Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header
              (SRH)", Work in Progress, Internet-Draft, draft-ietf-6man-
              segment-routing-header-26, 22 October 2019,
              <http://www.ietf.org/internet-drafts/draft-ietf-6man-
              segment-routing-header-26.txt>.








Malis, et al.             Expires 25 March 2024                [Page 16]

Internet-Draft           DetNet Controller Plane          September 2023


   [I-D.ietf-detnet-topology-yang]
              Geng, X., Chen, M., Li, Z., and R. Rahman, "Deterministic
              Networking (DetNet) Topology YANG Model", Work in
              Progress, Internet-Draft, draft-ietf-detnet-topology-yang-
              00, 25 January 2019, <http://www.ietf.org/internet-drafts/
              draft-ietf-detnet-topology-yang-00.txt>.

   [I-D.ietf-detnet-yang]
              Geng, X., Chen, M., Ryoo, Y., Li, Z., Rahman, R., and D.
              Fedyk, "Deterministic Networking (DetNet) Configuration
              YANG Model", Work in Progress, Internet-Draft, draft-ietf-
              detnet-yang-06, 11 June 2020, <http://www.ietf.org/
              internet-drafts/draft-ietf-detnet-yang-06.txt>.

   [IEEE.802.1QBV_2015]
              IEEE, "IEEE Standard for Local and metropolitan area
              networks -- Bridges and Bridged Networks - Amendment 25:
              Enhancements for Scheduled Traffic", IEEE 802.1Qbv-2015,
              DOI 10.1109/IEEESTD.2016.7572858, 18 March 2016,
              <http://ieeexplore.ieee.org/servlet/
              opac?punumber=7572858>.

   [OPENFLOW] Open Networking Foundation, "OpenFlow Switch
              Specification, Version 1.5.1 (Protocol version 0x06)",
              ONF TS-025, March 2015, <https://www.opennetworking.org/
              wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf>.

   [RFC3209]  Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
              and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
              Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
              <https://www.rfc-editor.org/info/rfc3209>.

   [RFC4384]  Meyer, D., "BGP Communities for Data Collection", BCP 114,
              RFC 4384, DOI 10.17487/RFC4384, February 2006,
              <https://www.rfc-editor.org/info/rfc4384>.

   [RFC5036]  Andersson, L., Ed., Minei, I., Ed., and B. Thomas, Ed.,
              "LDP Specification", RFC 5036, DOI 10.17487/RFC5036,
              October 2007, <https://www.rfc-editor.org/info/rfc5036>.

   [RFC5439]  Yasukawa, S., Farrel, A., and O. Komolafe, "An Analysis of
              Scaling Issues in MPLS-TE Core Networks", RFC 5439,
              DOI 10.17487/RFC5439, February 2009,
              <https://www.rfc-editor.org/info/rfc5439>.







Malis, et al.             Expires 25 March 2024                [Page 17]

Internet-Draft           DetNet Controller Plane          September 2023


   [RFC5960]  Frost, D., Ed., Bryant, S., Ed., and M. Bocci, Ed., "MPLS
              Transport Profile Data Plane Architecture", RFC 5960,
              DOI 10.17487/RFC5960, August 2010,
              <https://www.rfc-editor.org/info/rfc5960>.

   [RFC6020]  Bjorklund, M., Ed., "YANG - A Data Modeling Language for
              the Network Configuration Protocol (NETCONF)", RFC 6020,
              DOI 10.17487/RFC6020, October 2010,
              <https://www.rfc-editor.org/info/rfc6020>.

   [RFC6241]  Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed.,
              and A. Bierman, Ed., "Network Configuration Protocol
              (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011,
              <https://www.rfc-editor.org/info/rfc6241>.

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC7752]  Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and
              S. Ray, "North-Bound Distribution of Link-State and
              Traffic Engineering (TE) Information Using BGP", RFC 7752,
              DOI 10.17487/RFC7752, March 2016,
              <https://www.rfc-editor.org/info/rfc7752>.

   [RFC8277]  Rosen, E., "Using BGP to Bind MPLS Labels to Address
              Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017,
              <https://www.rfc-editor.org/info/rfc8277>.

   [RFC8283]  Farrel, A., Ed., Zhao, Q., Ed., Li, Z., and C. Zhou, "An
              Architecture for Use of PCE and the PCE Communication
              Protocol (PCEP) in a Network with Central Control",
              RFC 8283, DOI 10.17487/RFC8283, December 2017,
              <https://www.rfc-editor.org/info/rfc8283>.

   [RFC8660]  Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S.,
              Decraene, B., Litkowski, S., and R. Shakir, "Segment
              Routing with the MPLS Data Plane", RFC 8660,
              DOI 10.17487/RFC8660, December 2019,
              <https://www.rfc-editor.org/info/rfc8660>.

Authors' Addresses

   Andrew G. Malis
   Independent
   Email: agmalis@gmail.com




Malis, et al.             Expires 25 March 2024                [Page 18]

Internet-Draft           DetNet Controller Plane          September 2023


   Xuesong Geng
   Huawei
   Email: gengxuesong@huawei.com


   Mach (Guoyi) Chen
   Huawei
   Email: mach.chen@huawei.com


   Fengwei Qin
   China Mobile
   Email: qinfengwei@chinamobile.com


   Balazs Varga
   Ericsson
   Email: balazs.a.varga@ericsson.com


   Carlos J. Bernardos
   Universidad Carlos III de Madrid
   Av. Universidad, 30
   28911 Leganes, Madrid
   Spain
   Phone: +34 91624 6236
   Email: cjbc@it.uc3m.es
   URI:   http://www.it.uc3m.es/cjbc/























Malis, et al.             Expires 25 March 2024                [Page 19]