| Internet Engineering Task Force | I. Hussain |
| Internet-Draft | R. Valiveti |
| Intended status: Informational | K. Pithewan |
| Expires: January 8, 2017 | Infinera Corp |
| July 7, 2016 |
FlexE Usecases
draft-hussain-ccamp-flexe-usecases-01
Traditionally, Ethernet MAC rates were constrained to match the rates of the Ethernet PHY(s). OIF's implementation agreement [OIFMLG3] was the first step in allowing MAC rates to be different than the PHY rates. OIF has recently approved another implementation agreement [OIFFLEXE1] which allows complete decoupling of the MAC data rates and the Ethernet PHY(s) that support them. This includes support for (a) MAC rates which are greater than the rate of a single PHY (satisfied by bonding of multiple PHY(s)), (b) MAC rates which are less than the rate of a PHY (sub-rate), (c) support of multiple FlexE client signals carried over a single PHY, or over a collection of bonded PHY(s). This draft catalogs the usecases that are encountered when these Flexible rate Ethernet client signals are transported over OTN networks.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 8, 2017.
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Traditionally, Ethernet MAC rates were constrained to match the rates of the Ethernet PHY(s). OIF's implementation agreement [OIFMLG3] was the first step in allowing MAC rates to be different than the PHY rates standardized by IEEE. OIF has recently approved another implementation agreement [OIFFLEXE1] which allows complete decoupling of the MAC data rates and the Ethernet PHY(s) that support them. This includes support for (a) MAC rates which are greater than the rate of a single PHY (satisfied by bonding of multiple PHY(s)), (b) MAC rates which are less than the rate of a PHY (sub-rate), (c) support of multiple FlexE client signals carried over a single PHY, or over a collection of bonded PHY(s). The capabilities supported by the OIF FlexE implementation agreement version 1.0 are:
Optical Transport Networks (defined by [G709] and [G798]) have, until recently, only dealt with bit (or codeword) transparent transport of Ethernet client signals. The introduction of the FlexE capabilities at the OTN client interfaces requires the OTNs to examine the various usecases. This Internet-Draft examines the various usecases that arise when transporting the Flexible Rate Ethernet signals in Optical transport networks. This list of usecases will help identify the Control Plane (i.e. Routing and Signaling) extensions that may be required).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
The FlexE shim layer in a router maps the FlexE client(s) over the FlexE group. The transport network is unware of the FlexE. Each of the FlexE group PHY is carried independently across the transport network over the same fiber route. The FlexE shim in the router tolerates end-to-end skew across the network. This usecase allows to utilize Network Processor Unit (NPU) and router port rate full capacities with legacy transport equipment that provides PCS-codeword transparent transport of 100GbE. It allows striping of PHYs in the FlexE group over multiple transport line cards.
==================================================================
+ FlexE Ethernet Client(s) +
+-----------------------------------------------------------+
+ +
+ FlexE skew tolerance
+----------------------------------------+
+ for end-to-distance +
+-----------+ 2x100GE +---------+ +----------+ +------------+
| | | | | | | |
| Router1 | | | | | | |
|FlexE Shim +---------+ A-end | | Z-end +-----+Router 2 |
| | | (FlexE | | (FlexE | |(FlexE Shim)|
| +---^-----+ unaware)| | unaware)+-----+ |
| | | | | | | | |
| | | | | | | | |
+-----------+ + +---------+ +----------+ +------------+
FlexE Group
\----------Transport----------/
network
+--------------+ +----------------+
| FlexE Clients| | FlexE Client(s)|
+--------------+ +----------------+
| FlexE Shim | | FlexE Shim |
+----+----+----+ +----+------+----+
|PHY | | PHY | | PHY | | PHY |
+---+---+--+---+ +---+--+ +--+--+
| | +-----+ +-----+ | |
| +----------+ PHY | | PHY |-------+ |
| +-----+ +-----+ |
| | ODU4+-----------+ ODU4| |
| +-----+ +-----+ |
| |
| +-----+ +-----+ |
+-----------------+ PHY | | PHY +-----------------+
+-----+ +-----+
| ODU4+-----------+ ODU4|
+-----+ +-----+
==================================================================
Figure 1: FlexE unaware transport
This scenario represents an optimization of the FlexE unaware transport presented in Section 3.1, and illustrated in Figure 1. In this application (see Figure 2), the devices at the edge of the transport network do not terminate the FlexE shim layer, but are aware of the format of the FlexE overhead. They "snoop" the FlexE overhead to determine the subset of the set of all calendar slots that are available for use (i.e. these calendar slots may be used, or unused). The transport network edge removes the unavailable calendar slots at the ingress to the network, and adds the same unavailable calendar slots back when exiting the network. The result is that the FlexE Shim layers at both routers see exactly the same input that they saw in the FlexE unware scenario -- with the added benefit that the line (or DWDM) side bandwidth has been optimized to be sufficient to carry only the available calendar slots in all of the Ethernet PHY(s) in the FlexE group. This mode may be used in cases where the bandwidth of the Ethernet PHY is greater than the bit rate supported by a wavelength (and it is known that that all calendar slots in the PHY are not "available").
The transport network edge device could learn of the set of unavailable calendar slots in a variety of ways; a few examples are listed below:
In the basic FlexE aware mode, the transport network edge does not expect the number of unavailable calendar slots to change dynamically.
Note that the process of removing unavailable calendar slots from a FlexE PHY is called "crunching" (see [OIFFLEXE1]). The following additional notes apply to Figure 2:
================================================================
FlexE Ethernet Client(s)
+-----------------------------------------------------+
FlexE skew tolerance
+---------------------------------------------+
for end+to+distance
+--------+ 2 x 100GE +---------+ +---------+ +------+
| R1 | | | | +----+ R2 |
| (FlexE+-----------+ NE A | | NE Z | |(FlexE|
| Shim) | | (FlexE | | (FlexE +----+ Shim |
| +-----^-----+ aware) | | aware) | | |
| | | | | | | | |
+--------+ + +---------+ +---------+ +------+
FlexE Group
\+--------+Transport+--------+/
network
+-------------+ +-------------+
|FlexE clients| |FlexE clients|
+-------------+ +-------------+
| FlexE Shim | | FlexE Shim |
+------+------+ +------+------+
| PHY | PHY | | PHY | PHY |
+---+--+--+---+ +---+--+---+--+
| | | |
| | +--------+ +--------+ | |
| +-------+PHY-c | |PHY-c +-+ |
| +--------+ +--------+ |
| |ODUflex +------------+ODUflex | |
| +--------+ +--------+ |
| |
| +--------+ +--------+ |
+-------------+PHY-c | |PHY-c +--------+
+--------+ +--------+
|ODUflex +------------+ODUflex |
+--------+ +--------+
| Legend:
| R1, R2 - Routers (supporting the FlexE clients)
| NE A, Z - Transport Network Edge nodes
| PHY-c - Crunched FlexE PHY(s)
===============================================================
Figure 2: FlexE Aware Transport
These usecases build upon the basic router-transport equipment connectivity illustrated in Figure 1. The FlexE shim layer at the router maps to the set of FlexE clients over the FlexE group, as usual. This section considers various usecases in which the equipment located at the edge of the transport network is fully aware of the FlexE OH, and FlexE Shim layers on the transport network edge, and the customer edge are peers. In the router to network direction, the transport edge node terminates the FlexE shim layer, and extracts one or more FlexE client signals, and transports them through the network. That is, these usecases are distinguished from the FlexE unaware cases in that the FlexE group, and the FlexE shim layer end at the transport network edge, and only the extracted FlexE client signals transit the optical network. In the network to router direction, the transport edge node maps a set of FlexE clients to the FlexE group (i.e. performing the same functions as the router which connects to the transport network).The various usecases differ in the combination of service endpoints in the transport network. In the FlexE termination scenarios, the distance between the FlexE Shims is limited the normal Ethernet link distance. The FlexE shims in the router, and the equipment need to support a small amount skew.
In this scenario, service consists of transporting a FlexE client through the transport network, and possibly combining this FlexE client with other FlexE clients into a FlexE group at the endpoints. The FlexE client signal can be transported in two manners within the OTN: (i) directly over one or more wavelengths (ii) mapped into an ODUflex (of the appropriate rate) and then switched across the OTN. Figure 3 illustrates the scenario involving the mapping of a FlexE client to an ODUflex envelope; this figure only shows the signal "stack" at the service endpoints, and doesn't illustrate the switching of the ODUflex entity through the OTN. The ODUflex mapping will be beneficial in scenarios where the rate of the FlexE client is less than the capacity of a single wavelength deployed on the DWDM side of the OTN network, and allows the network operators to packet multiple FlexE client signals into the same wavelength -- thereby improving the network efficiency. Although Figure 3 illustrates the scenario in which one FlexE client is transported within the OTN, the following points should be noted:
==================================================================
+--------+ 2 x 100GE +---------+ +----------+ +--------+
| | | | | | | |
| Router1| | | | | | |
| FlexE +-----------+ A-end | | Z-end +------+Router2 |
| Shim | | (FlexE | | (FlexE | |FlexE |
| +-----^-----+ term) | term) +------+ Shim |
| | | | | | | | |
| | | | | | | | |
+--------+ + +---------+ +----------+ +--------+
FlexE Group
\+--------+Transport+--------+/
network
+-----------+ +--------------+ +-------------+ +-----------+
| Client(s) | | Client | | Client | | Client(s) |
+-----------+ +--------+-----+ +------+------+ +-----------+
| FlexE Shim| | Shim | | | | Shim | | FlexE Shim|
+-----------+ +--------+ ODU | | ODU +------+ +-----------+
| PHY(s) | | PHY(s) | flex| | flex |PHY(s)| | PHY(s) |
+---+-------+ +---+----+--+--+ +---+--+---+--+ +---+-------+
| | | | | |
+---------------+ +-----------+------+----------+
=================================================================
Figure 3: FlexE termination: FlexE clients at both endpoints
The OIF implementation agreement [OIFMLG3] currently supports FlexE client signals carried over one or more 100GBASE-R PHY(s). There is a calendar of 5G timeslots associated with each PHY, and each FlexE client can make use of a number of timeslots (possibly distributed across the members of the FlexE group. This implies that the FlexE client rates are multiples of 5Gbps. When the rates of the FlexE client signals matches the MAC rates corresponding to existing Ethernet PHYs, i.e. 10GBASE-R/40GBASE-R/100GBASE-R, there is a need for the FlexE client signal to interwork with the native Ethernet client received from a single (non-FlexE capable) Ethernet PHY. This capability is expected to be extended to any future Ethernet PHY rates that the IEEE may define in future (e.g. 25G, 50G, 200G etc.). In these cases, although the bit rate of the FlexE client matches the MAC rate of other endpoint, the 64B66B PCS codewords for the FlexE client need to be transformed (via ordered set translation) to match the specification for the specific Ethernt PHY. These details are described in Section 7.2.2 of [OIFMLG3] and are not eloborated any further in this document.
Figure 4 illustrates a scenario involving the interworking of a 10G FlexE client with a 10GBASE-R native Ethernet signal. In this example, the network wrapper is ODU2e.
==================================================================
+--------+ 2 x 100GE +-------+ +-------+ +--------+
| | | | | | | |
| Router1| | | | | | |
|(FlexE +-----------+ A-end | | Z-end | 10GE |Router 2|
| Shim) | |(FlexE | | +------+ |
| +-----^-----+ term) | | | | |
| | | | | | | | |
| | | | | | | | |
+--------+ + +-------+ +-------+ +--------+
FlexE Group
\+---------Transport---------+/
network
+-----------+ +---------------+
| Client(s) | | Client | +------------+ +---------+
+-----------+ +-------+-------+ | 10GE PCS | | 10GE PCS|
| FlexE Shim| | Shim | | +-------+----+ +---------+
+-----------+ +-------+ ODU | | ODU2e | PHY| | PHY |
| PHY(s) | | PHY(s)| 2e | +---+---+--+-+ +-----+---+
+---+-------+ +---+-------+---+ | | |
| | | | | |
| | | | | |
+---------------+ +-------------+ +------------+
=================================================================
Figure 4: FlexE client interop with Native Ethernet Client
As explained in the Introduction section (Section 1 OIFMLG3 [OIFMLG3] introduced support for carrying 10GE and 40GE client signals over a group of 100GBASE-R Ethernet PHY(s). While the most recent implementation agreement doesn't call it out explicitly, it is expected that the FlexE clients (as defined in [OIFFLEXE1]), and 10GBASE-R/40GBASE-R clients supported by OIFMLG3 [OIFMLG3]) will interoperate.
Figure 5 illustrates a scenario involving the interworking of a 10G FlexE client with a 10GBASE-R client supported by an OIFMLG3 interface. In this example, the network wrapper is ODU2e.
==================================================================
+--------+ 2 x 100GE +---------+ +---------+ +---------+
| | | | | | | |
| Router1| | | | | | |
| FlexE +-----------+ A-end | | Z-end +------+Router 2 |
| Shim | | (FlexE | | | |(MLG-3.0)|
| +-----^-----+ term) | | +------+ |
| | | | | | | | |
| | | | | | | | |
+--------+ + +---------+ +---------+ +---------+
FlexE Group
\+--------+Transport+--------+/
network
+-----------+ +-------------+ +--------------+ +----------+
| Client(s) | | Client | | 10GE PCS | | 10GE Cl. |
+-----------+ +--------+----+ +------+-------+ +----------+
| FlexE Shim| | Shim | | | | MLG3 | | MLG3 |
+-----------+ +--------+ ODU| | ODU +-------+ +----------+
| PHY(s) | | PHY(s) | 2e | | 2e | PHY(s)| | PHY(s) |
+---+-------+ +---+----+--+-+ +---+--+---+---+ +---+------+
| | | | | |
+---------------+ +------------+ +------------+
=================================================================
Figure 5: FlexE client interop with Ethernet Client supported by MLG3
This section covers an extension of the scenario presented in Section 3.3.1. Each FlexE client signal defined in [OIFFLEXE1] has a rate which is a multiple of 5G, and occupies the required number of calendar slots (5G granularity) in the FlexE group (possibly distributed among the PHY(s) which have been bonded. The OIF implementation agreement defines two calendars, one currently active, and the future calendar to which the sender wants to transition to. This capability can be used to coordinate a synchronized switchover of calendars between the two FlexE Shim functions -- one located in the customer eddge device (typically a router), and the transport network edge. In this scenario, there are three independent resizing domains which must be coordinated (see Figure 3).
It is possible to coordinate the resize operations in these domains in such a manner that the FlexE clients get the benefit of an end-to-end bandwidth change (increase/decrease), without involving any additional provisioning steps in the provider network. Note for the FlexE unaware use case (Section 3.1), the client BW can be resized by FlexE shim coordination between router 1 and router 2.
This section covers a degenerate FlexE aware scenario where router1, router2, and router3 are interconnected through back-to-back FlexE groups without an intermediate transport network (see Figure 6).
==================================================================
+--------+ 2 x 100GE +---------+ 3 x 100GE +---------+
| | | | | |
| Router1| | | | |
| FlexE +-----------+ Router2 +-----------+ Router3 |
| Shim | | FlexE +-----------+ FlexE |
| +-----^-----+ Shim +-----^-----+ Shim |
| | | | | | | |
| | | | | | | |
+--------+ + +---------+ + +---------+
FlexE Group FlexE Group
=================================================================
Figure 6: Back-to-Back FlexE
The list of aforementioned FlexE usecases can also be supported by mapping FlexE directly over one or more wavelengths. An example for the FlexE unaware transport over wavelength is depicted in Figure 7. Equivalent network diagrams for the other usecases can be obtained by replacing an OTN container with an Optical Channel (OCh).
==================================================================
+ FlexE Ethernet Client(s) +
+-----------------------------------------------------------+
+ +
+ FlexE skew tolerance
+----------------------------------------+
+ for end-to-distance +
+-----------+ 2x100GE +---------+ +----------+ +------------+
| | | | | | | |
| Router1 | | | | | | |
|FlexE Shim +---------+ A-end | | Z-end +-----+Router 2 |
| | | (FlexE | | (FlexE | |(FlexE Shim)|
| +---^-----+ unaware)| | unaware)+-----+ |
| | | | | | | | |
| | | | | | | | |
+-----------+ + +---------+ +----------+ +------------+
FlexE Group
\----------Transport----------/
network
+--------------+ +----------------+
| FlexE Clients| | FlexE Client(s)|
+--------------+ +----------------+
| FlexE Shim | | FlexE Shim |
+----+----+----+ +----+------+----+
|PHY | | PHY | | PHY | | PHY |
+---+---+--+---+ +---+--+ +--+--+
| | +-----+ +-----+ | |
| +----------+ PHY | | PHY |-------+ |
| +-----+ +-----+ |
| | OCh +-----------+ OCh | |
| +-----+ +-----+ |
| |
| +-----+ +-----+ |
+-----------------+ PHY | | PHY +-----------------+
+-----+ +-----+
| OCh +-----------+ OCh |
+-----+ +-----+
==================================================================
Figure 7: FlexE unaware transport over wavelength
This section summarizes solution requirements for the usecases described in this document to help identify the Control Plane (i.e. Routing and Signaling) extensions that may be required.
This memo includes no request to IANA.
None.
| [G709] | ITU, "Optical Transport Network Interfaces", February 2016. |
| [G798] | ITU, "Characteristics of optical transport network hierarchy equipment functional blocks", February 2012. |
| [OIFFLEXE1] | OIF, "FLex Ethernet Implementation Agreement Version 1.0 (OIF-FLEXE-01.0)", March 2016. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [OIFMLG3] | OIF, "Multi-Lane Gearbox Implementation Agreement Version 3.0 (OIF-MLG-3.0)", April 2016. |
This becomes an Appendix.