| Internet Engineering Task Force | I. Hussain, Ed. |
| Internet-Draft | R. Valiveti |
| Intended status: Informational | K. Pithewan |
| Expires: April 23, 2017 | Infinera Corp |
| Q. Wang, Ed. | |
| ZTE | |
| L. Andersson, Ed. | |
| F. Zhang | |
| M. Chen | |
| J. Dong | |
| Z. Du | |
| Z. Haomian | |
| X. Zhang | |
| J. Huang | |
| Q. Zhong | |
| Huawei | |
| October 20, 2016 |
GMPLS Routing and Signaling Framework for Flexible Ethernet (FlexE)
draft-izh-ccamp-flexe-fwk-00
Traditionally, Ethernet MAC rates were constrained to match the rates of the Ethernet PHY(s). OIF's implementation agreement [OIFMLG3] was the first step in allowing MAC rates to be different than the PHY rates. OIF has recently approved another implementation agreement [OIFFLEXE1] which allows complete decoupling of the MAC data rates and the Ethernet PHY(s) that support them. This includes support for (a) MAC rates which are greater than the rate of a single PHY (satisfied by bonding of multiple PHY(s)), (b) MAC rates which are less than the rate of a PHY (sub-rate), (c) support of multiple FlexE client signals carried over a single PHY, or over a collection of bonded PHY(s). The FlexE SHIM functions which bond multipe Ethernet PHY(s) to form a large "pipe" view the connectivity between two FlexE aware devices as a collection of multiple point-to-point links (one link per Ethernet PHY). These logical point-to-point links can either be direct links (without an intervening transport network), or realized via a Optical transport network. This draft catalogs the usecases that capture the FlexE deployment scenarios -- including the cases that include/exclude OTNs.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 23, 2017.
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Traditionally, Ethernet MAC rates were constrained to match the rates of the Ethernet PHY(s). OIF's implementation agreement [OIFMLG3] was the first step in allowing MAC rates to be different than the PHY rates standardized by IEEE. OIF has recently approved another implementation agreement [OIFFLEXE1] which allows complete decoupling of the MAC data rates and the Ethernet PHY(s) that support them. This includes support for (a) MAC rates which are greater than the rate of a single PHY (satisfied by bonding of multiple PHY(s)), (b) MAC rates which are less than the rate of a PHY (sub-rate), (c) support of multiple FlexE client signals carried over a single PHY, or over a collection of bonded PHY(s). The capabilities supported by the OIF FlexE implementation agreement version 1.0 are:
All networks which support the bonding of Ehernet interfaces (as per [OIFFLEXE1]) include a basic building block -- which consists of two FlexE SHIM functions (located at opposite ends of a link) and the (logical) point to point links that carry the Ethernet PHY signals between the two FlexE SHIM Functions. These logical point-to-point PHY links can be realized in a variety of ways:
Internet-draft examines the usescases that arise when the logical links between FlexE capable devices are (a) point-to-point links without any intervening network (b) realized via Optical transport networks. This draft considers the variants in which fhe two peer FlexE devices are both customer-edge devices, or customer-edge/provider edge devices. This list of usecases will help identify the Control Plane (i.e. Routing and Signaling) extensions that may be required).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
The FlexE shim layer in a router maps the FlexE client(s) over the FlexE group. The transport network is unware of the FlexE. Each of the FlexE group PHY is carried independently across the transport network over the same fiber route. The FlexE shim in the router tolerates end-to-end skew across the network. In this usecase, the router makes flexible use of the full capacity of the FlexE group, and depends on legacy transport equipment to realize PCS-codeword-transparent transport of 100GbE. It allows striping of PHYs in the FlexE group over multiple line cards in the transport equipment. It is worth mentioning that in this case, the FlexE SHIM layer is terminated at the routers, and the coordination of operations related to FlexE clients, e.g. creating new FlexE clients, deleting existing FlexE clients, and resizing the bandwidth of existing FlexE clients (if desired) happens between the two routers. Note that the transport network is completely transparent to the FlexE signals, and doesn't participate in any FlexE protocols.
==================================================================
+ FlexE Ethernet Client(s) +
+-----------------------------------------------------------+
+ +
+ FlexE skew tolerance
+----------------------------------------+
+ for end-to-distance +
+-----------+ 2x100GE +---------+ +----------+ +------------+
| | | | | | | |
| Router1 | | | | | | |
|FlexE Shim +---------+ A-end | | Z-end +-----+Router 2 |
| | | (FlexE | | (FlexE | |(FlexE Shim)|
| +---^-----+ unaware)| | unaware)+-----+ |
| | | | | | | | |
| | | | | | | | |
+-----------+ + +---------+ +----------+ +------------+
FlexE Group
\----------Transport----------/
network
+--------------+ +----------------+
| FlexE Clients| | FlexE Client(s)|
+--------------+ +----------------+
| FlexE Shim | | FlexE Shim |
+----+----+----+ +----+------+----+
|PHY | | PHY | | PHY | | PHY |
+---+---+--+---+ +---+--+ +--+--+
| | +-----+ +-----+ | |
| +----------+ PHY | | PHY |-------+ |
| +-----+ +-----+ |
| | ODU4+-----------+ ODU4| |
| +-----+ +-----+ |
| |
| +-----+ +-----+ |
+-----------------+ PHY | | PHY +-----------------+
+-----+ +-----+
| ODU4+-----------+ ODU4|
+-----+ +-----+
==================================================================
Figure 1: FlexE unaware transport
This scenario represents an optimization of the FlexE unaware transport presented in Section 3.1, and illustrated in Figure 1. In this application (see Figure 2), the devices at the edge of the transport network do not terminate the FlexE shim layer, but are aware of the (a) composition of the FlexE grpup (i.e. set of all contained Ethernet PHYs) and (b) format of the FlexE overhead. They "snoop" the FlexE overhead to determine the subset of the set of all calendar slots that are available for use (i.e. these calendar slots may be used, or unused). The transport network edge removes the unavailable calendar slots at the ingress to the network, and adds the same unavailable calendar slots back when exiting the network. The result is that the FlexE Shim layers at both routers see exactly the same input that they saw in the FlexE unware scenario -- with the added benefit that the line (or DWDM) side bandwidth has been optimized to be sufficient to carry only the available calendar slots in all of the Ethernet PHY(s) in the FlexE group. This mode may be used in cases where the bandwidth of the Ethernet PHY is greater than the bit rate supported by a wavelength (and it is known that that all calendar slots in the PHY are not "available").
The transport network edge device could learn of the set of unavailable calendar slots in a variety of ways; a few examples are listed below:
In the basic FlexE aware mode, the transport network edge does not expect the number of unavailable calendar slots to change dynamically.
Note that the process of removing unavailable calendar slots from a FlexE PHY is called "crunching" (see [OIFFLEXE1]). The following additional notes apply to Figure 2:
Note: The figure may need further editing to accurately depict the signal hierarchy.
================================================================
FlexE Ethernet Client(s)
+-----------------------------------------------------+
FlexE skew tolerance
+---------------------------------------------+
for end+to+distance
+--------+ 2 x 100GE +---------+ +---------+ +------+
| R1 | | | | +----+ R2 |
| (FlexE+-----------+ NE A | | NE Z | |(FlexE|
| Shim) | | (FlexE | | (FlexE +----+ Shim |
| +-----^-----+ aware) | | aware) | | |
| | | | | | | | |
+--------+ + +---------+ +---------+ +------+
FlexE Group
\+--------+Transport+--------+/
network
+-------------+ +-------------+
|FlexE clients| |FlexE clients|
+-------------+ +-------------+
| FlexE Shim | | FlexE Shim |
+-------------+ +-------------+
| PHY | PHY | | PHY | PHY |
+-------------+ +-------------+
| | | |
| | +-------------+ +------------+ | |
| | | FlexE-psg | | FlexE-psg | | |
| | +-------------+ +------------+ | |
| +--+ PHY|ODUflex +------- |ODUFlex|PHY +--+ |
| +-------------+ +------------+ |
| |
| +-------------+ +------------+ |
| | FlexE|psg | | FlexE|psg | |
| +-------------+ +------------+ |
+--------+ PHY|ODUflex +------- |ODUFlex|PHY +--------+
+-------------+ +------------+
+ Legend:
| R1, R2 + Routers (supporting the FlexE clients)
| NE A, Z + Transport Network Edge nodes
+ FlexE-psg: FlexE partial rate (sub) group signal
(per G.709:17.12)
===============================================================
Figure 2: FlexE Aware Transport
These usecases build upon the basic router-transport equipment connectivity illustrated in Figure 1. The FlexE shim layer at the router maps to the set of FlexE clients over the FlexE group, as usual. This section considers various usecases in which the equipment located at the edge of the transport network instantiates the FlexE Shim function which peers with the FlexE shim on the customer device. In the router to network direction, the transport edge node terminates the FlexE shim layer, and extracts one or more FlexE client signals, and transports them through the network. That is, these usecases are distinguished from the FlexE unaware cases in that the FlexE group, and the FlexE shim layer end at the transport network edge, and only the extracted FlexE client signals transit the optical network. In the network to router direction, the transport edge node maps a set of FlexE clients to the FlexE group (i.e. performing the same functions as the router which connects to the transport network).The various usecases differ in the combination of service endpoints in the transport network. In the FlexE termination scenarios, the distance between the FlexE Shims is limited the normal Ethernet link distance. The FlexE shims in the router, and the equipment need to support a small amount skew.
In this scenario, service consists of transporting a FlexE client through the transport network, and possibly combining this FlexE client with other FlexE clients into a FlexE group at the endpoints. The FlexE client signal can be transported in two manners within the OTN: (i) directly over one or more wavelengths (ii) mapped into an ODUflex (of the appropriate rate) and then switched across the OTN. Figure 3 illustrates the scenario involving the mapping of a FlexE client to an ODUflex envelope; this figure only shows the signal "stack" at the service endpoints, and doesn't illustrate the switching of the ODUflex entity through the OTN. The ODUflex mapping will be beneficial in scenarios where the rate of the FlexE client is less than the capacity of a single wavelength deployed on the DWDM side of the OTN network, and allows the network operators to packet multiple FlexE client signals into the same wavelength -- thereby improving the network efficiency. Although Figure 3 illustrates the scenario in which one FlexE client is transported within the OTN, the following points should be noted:
==================================================================
+--------+ 2 x 100GE +---------+ +----------+ +--------+
| | | | | | | |
| Router1| | | | | | |
| FlexE +-----------+ A-end | | Z-end +------+Router2 |
| Shim | | (FlexE | | (FlexE | |FlexE |
| +-----^-----+ term) | term) +------+ Shim |
| | | | | | | | |
| | | | | | | | |
+--------+ + +---------+ +----------+ +--------+
FlexE Group
\+--------+Transport+--------+/
network
+-----------+ +--------------+ +-------------+ +-----------+
| Client(s) | | Client | | Client | | Client(s) |
+-----------+ +--------+-----+ +------+------+ +-----------+
| FlexE Shim| | Shim | | | | Shim | | FlexE Shim|
+-----------+ +--------+ ODU | | ODU +------+ +-----------+
| PHY(s) | | PHY(s) | flex| | flex |PHY(s)| | PHY(s) |
+---+-------+ +---+----+--+--+ +---+--+---+--+ +---+-------+
| | | | | |
+---------------+ +-----------+------+----------+
=================================================================
Figure 3: FlexE termination: FlexE clients at both endpoints
The OIF implementation agreement [OIFFLEXE1] currently supports FlexE client signals carried over one or more 100GBASE-R PHY(s). There is a calendar of 5G timeslots associated with each PHY, and each FlexE client can make use of a number of timeslots (possibly distributed across the members of the FlexE group. This implies that the FlexE client rates are multiples of 5Gbps. When the rates of the FlexE client signals matches the MAC rates corresponding to existing Ethernet PHYs, i.e. 10GBASE-R/40GBASE-R/100GBASE-R, there is a need for the FlexE client signal to interwork with the native Ethernet client received from a single (non-FlexE capable) Ethernet PHY. This capability is expected to be extended to any future Ethernet PHY rates that the IEEE may define in future (e.g. 25G, 50G, 200G etc.). In these cases, although the bit rate of the FlexE client matches the MAC rate of other endpoint, the 64B66B PCS codewords for the FlexE client need to be transformed (via ordered set translation) to match the specification for the specific Ethernt PHY. These details are described in Section 7.2.2 of [OIFFLEXE1] and are not eloborated any further in this document.
Figure 4 illustrates a scenario involving the interworking of a 10G FlexE client with a 10GBASE-R native Ethernet signal. In this example, the network wrapper is ODU2e.
==================================================================
+--------+ 2 x 100GE +-------+ +-------+ +--------+
| | | | | | | |
| Router1| | | | | | |
|(FlexE +-----------+ A-end | | Z-end | 10GE |Router 2|
| Shim) | |(FlexE | | +------+ |
| +-----^-----+ term) | | | | |
| | | | | | | | |
| | | | | | | | |
+--------+ + +-------+ +-------+ +--------+
FlexE Group
\+---------Transport---------+/
network
+-----------+ +---------------+
| Client(s) | | Client | +------------+ +---------+
+-----------+ +-------+-------+ | 10GE PCS | | 10GE PCS|
| FlexE Shim| | Shim | | +-------+----+ +---------+
+-----------+ +-------+ ODU | | ODU2e | PHY| | PHY |
| PHY(s) | | PHY(s)| 2e | +---+---+--+-+ +-----+---+
+---+-------+ +---+-------+---+ | | |
| | | | | |
| | | | | |
+---------------+ +-------------+ +------------+
=================================================================
Figure 4: FlexE client interop with Native Ethernet Client
As explained in the Introduction section ( Section 1 OIFMLG3 [OIFMLG3] introduced support for carrying 10GE and 40GE client signals over a group of 100GBASE-R Ethernet PHY(s). While the most recent implementation agreement doesn't call it out explicitly, it is expected that the FlexE clients (as defined in [OIFFLEXE1]), and 10GBASE-R/40GBASE-R clients supported by OIFMLG3 [OIFMLG3]) will interoperate.
Figure 5 illustrates a scenario involving the interworking of a 10G FlexE client with a 10GBASE-R client supported by an OIFMLG3 interface. In this example, the network wrapper is ODU2e.
==================================================================
+--------+ 2 x 100GE +---------+ +---------+ +---------+
| | | | | | | |
| Router1| | | | | | |
| FlexE +-----------+ A-end | | Z-end +------+Router 2 |
| Shim | | (FlexE | | | |(MLG-3.0)|
| +-----^-----+ term) | | +------+ |
| | | | | | | | |
| | | | | | | | |
+--------+ + +---------+ +---------+ +---------+
FlexE Group
\+--------+Transport+--------+/
network
+-----------+ +-------------+ +--------------+ +----------+
| Client(s) | | Client | | 10GE PCS | | 10GE Cl. |
+-----------+ +--------+----+ +------+-------+ +----------+
| FlexE Shim| | Shim | | | | MLG3 | | MLG3 |
+-----------+ +--------+ ODU| | ODU +-------+ +----------+
| PHY(s) | | PHY(s) | 2e | | 2e | PHY(s)| | PHY(s) |
+---+-------+ +---+----+--+-+ +---+--+---+---+ +---+------+
| | | | | |
+---------------+ +------------+ +------------+
=================================================================
Figure 5: FlexE client interop with Ethernet Client supported by MLG3
This section covers a degenerate FlexE termination scenario in which router1, router2, and router3 are interconnected through back-to-back FlexE groups without an intermediate transport network (see Figure 6). In this example, the FlexE SHIM at Router2 extracts one or more FlexE client signals from the FlexE group connected to Router1, and mutliplexes these extracted FlexE signals into the FlexE group towards the appropriate router (e.g. Router3). Note that each of the extracted FlexE client signals can be indepdenently routed towards its respective FlexE group.
==================================================================
+--------+ 2 x 100GE +---------+ 3 x 100GE +---------+
| | | | | |
| Router1| | | | |
| FlexE +-----------+ Router2 +-----------+ Router3 |
| Shim | | FlexE +-----------+ FlexE |
| +-----^-----+ Shim +-----^-----+ Shim |
| | | | | | | |
| | | | | | | |
+--------+ + +---------+ + +---------+
FlexE Group FlexE Group
=================================================================
Figure 6: Back-to-Back FlexE
In the scenario presented in Figure 6, it is possible to support the FlexE client signal resizing on an end-to-end basis. Thus, for example, the resizing of the end-to-end FlexE client circuit with a scope of Router1-Router2-Router3 is accomplished by correctly coordinating the resizing operations across these two segments: Router1-Router2, Router2-Router3. The hop-by-hop FlexE client signal resizing operations across each of these segments (or hops) are accomplished by using the following FlexE overhead (as per [OIFFLEXE1]):
It is expected that the exact sequence of FlexE client resizing operations will be different for the cases involving bandwidth increase/decrease.
This section summarizes solution requirements for the usecases described in this document to help identify the Control Plane (i.e. Routing and Signaling) extensions that may be required.
This memo includes no request to IANA.
None.
| [G709] | ITU, "Optical Transport Network Interfaces (http://www.itu.int/rec/T-REC-G.709-201606-P/en)", July 2016. |
| [G798] | ITU, "Characteristics of optical transport network hierarchy equipment functional blocks (http://www.itu.int/rec/T-REC-G.798-201212-I/en)", February 2014. |
| [OIFFLEXE1] | OIF, "FLex Ethernet Implementation Agreement Version 1.0 (OIF-FLEXE-01.0)", March 2016. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [OIFMLG3] | OIF, "Multi-Lane Gearbox Implementation Agreement Version 3.0 (OIF-MLG-3.0)", April 2016. |
This becomes an Appendix.