| RAW | P. Thubert, Ed. |
| Internet-Draft | Cisco Systems |
| Intended status: Informational | D. Cavalcanti |
| Expires: December 8, 2019 | Intel |
| X. Vilajosana | |
| Universitat Oberta de Catalunya | |
| June 6, 2019 |
Reliable and Available Wireless Technologies
draft-thubert-raw-technologies-01
This document presents a series of recent technologies that are capable of time synchronization and scheduling of transmission, making them suitable to carry time-sensitive flows with requirements of both reliable delivery in bounded time, and availability at all times, regardless of packet transmission or individual equipement failures.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 8, 2019.
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
When used in math or philosophy, the term "deterministic" generally refers to a perfection where all aspect are understood and predictable. A perfectly Deterministic Network would ensure that every packet reach its destination following a predetermined path along a predefined schedule to be delivered at the exact due time. In a real and imperfect world, a Deterministic Network must highly predictable, which is a combination of reliability and availability. On the one hand the network must be reliable, meaning that it will perform as expected for all packets and in particular that it will always deliver the packet at the destination in due time. On the other hand, the network must be available, meaning that it is resilient to any single outage, whether the cause is a software, a hardware or a transmission issue.
RAW (Reliable and Available Wireless) is an effort to provide Deterministic Networking on across a path that include a wireless physical layer. Making Wireless Reliable and Available is even more challenging than it is with wires, due to the numerous causes of loss in transmission that add up to the congestion losses and the delays caused by overbooked shared resources. In order to maintain a similar quality of service along a multihop path that is composed of wired and wireless hops, additional methods that are specific to wireless must be leveraged to combat the sources of loss that are also specific to wireless.
Such wireless-specific methods include per-hop retransmissions (HARQ) and P2MP overhearing whereby multiple receivers are scheduled to receive the same transmission, which balances the adverse effects of the transmission losses that are experienced when a radio is used as pure P2P.
This specification uses several terms that are uncommon on protocols that ensure bets effort transmissions for stochastics flows, such as found in the traditional Internet and other statistically multiplexed packet networks.
The operations of a Deterministic Network often rely on precisely applying a tight schedule, in order to avoid collision loss and guarantee the worst-case time of delivery. To achieve this, there must be a shared sense of time throughout the network. The sense of time is usually provided by the lower layer and is not in scope for RAW.
A network is reliable when the statistical effects that affect the packet transmission are eliminated. This involves maintaining at all time the amount of critical packets within the physical capabilities of the hardware and that of the radio medium. This is achieved by controlling the use of time-shared resources such as CPUs and buffers, by shaping the flows and by scheduling the time of transmission of the packets that compose the flow at every hop.
Equipment failure, such as an access point rebooting, a broken radio adapter, or a permanent obstacle to the transmission, is a secondary source of packet loss. When a breakage occurs, multiple packets are lost in a row before the flows are rerouted or the system may recover. This is not acceptable for critical applications such as related to safety. A typical process control loop will tolerate an occasional packet loss, but a loss of several packets in a row will cause an emergency stop (e.g., after 4 packets lost, within a period of 1 second).
Network Availability is obtained by making the transmission resilient against hardware failures and radio transmission losses due to uncontrolled events such as co-channel interferers, multipath fading or moving obstacles. The best results are typically achieved by pseudo randomly cumulating all forms of diversity, in the spatial domain with replication and elimination, in the time domain with ARQ and diverse scheduled transmissions, and in the frequency domain with frequency hopping or channel hopping between frames.
In addition to the benefits listed in Section 3.1, scheduling transmissions provides specific value to the wireless medium.
On the one hand, scheduling avoids collisions between scheduled transmissions and can ensure both time and frequency diversity between retries in order to defeat co-channel interference from un-controlled transmitters as well as multipath fading. Transmissions can be scheduled on multiple channels in parallel, which enables to use the full available spectrum while avoiding the hidden terminal problem, e.g., when the next packet in a same flow interferes on a same channel with the previous one that progressed a few hops farther.
On the other hand, scheduling optimizes the bandwidth usage: compared to classical Collision Avoidance techniques, there is no blank time related to inter-frame space (IFS) and exponential back-off in scheduled operations. A minimal Clear Channel Assessment may be needed to comply with the local regulations such as ETSI 300-328, but that will not detect a collision when the senders are synchronized. And because scheduling allows a time-sharing operation, there is no limit to the ratio of isolated critical traffic.
Finally, scheduling plays a critical role to save energy. In IOT, energy is the foremost concern, and synchronizing sender and listener enables to always maintain them in deep sleep when there is no scheduled transmission. This avoids idle listening and long preambles and enables long sleep periods between traffic and resynchronization, allowing battery-operated nodes to operate in a mesh topology for multiple years.
With an active portfolio of nearly 1,300 standards and projects under development, IEEE is a leading developer of industry standards in a broad range of technologies that drive the functionality, capabilities, and interoperability of products and services, transforming how people live, work, and communicate.
The IEEE 802 LAN/MAN Standards Committee (SC) develops and maintains networking standards and recommended practices for local, metropolitan, and other area networks, using an open and accredited process, and advocates them on a global basis. The most widely used standards are for Ethernet, Bridging and Virtual Bridged LANs Wireless LAN, Wireless PAN, Wireless MAN, Wireless Coexistence, Media Independent Handover Services, and Wireless RAN. An individual Working Group provides the focus for each area. Standards produced by the IEEE 802 SC are freely available from the IEEE GET Program after they have been published in PDF for six months.
The IEEE 802.11 LAN standards define the underlying MAC and PHY layers for the Wi-Fi technology. Wi-Fi/802.11 is one of the most successful wireless technologies, supporting many application domains. While previous 802.11 generations, such as 802.11n and 802.11ac, have focused mainly on improving peak throughput, more recent generations are also considering other performance vectors, such as efficiency enhancements for dense environments in 802.11ax, and latency and support for Time-Sensitive Networking (TSN) capabilities in 802.11be.
IEEE 802.11 already supports some 802.1 TSN standards and it is undergoing efforts to support for other 802.1 TSN capabilities required to address the use cases that require time synchronization and timeliness (bounded latency) guarantees with high reliability and availability. The IEEE 802.11 working group has been working in collaboration with the IEEE 802.1 group for several years extending 802.1 features over 802.11. As with any wireless media, 802.11 imposes new constraints and restrictions to TSN-grade QoS, and tradeoffs between latency and reliability guarantees must be considered as well as managed deployment requirements. An overview of 802.1 TSN capabilities and their extensions to 802.11 are discussed in [Cavalcanti_2019].
Wi-Fi Alliance (WFA) is the worldwide network of companies that drives global Wi-Fi adoption and evolution through thought leadership, spectrum advocacy, and industry-wide collaboration. The WFA work helps ensure that Wi-Fi devices and networks provide users the interoperability, security, and reliability they have come to expect.
The following IEEE 802.11 specifications/certifications are relevant in the context of reliable and available wireless services and support for time-sensitive networking capabilities:
In addition, major amendments being developed by the IEEE802.11 Working Group include capabilities that can be used as the basis for providing more reliable and predictable wireless connectivity and support time-sensitive applications:
The main 802.11ax and 802.11be capabilities and their relevance to RAW are discussed in the remainder of this document.
The next generation Wi-Fi (Wi-Fi 6) is based on the IEEE802.11ax amendment [IEEE80211ax], which includes new capabilities to increase efficiency, control and reduce latency. Some of the new features include higher order 1024-QAM modulation, support for uplink multi-user MIMO, OFDMA, trigger-based access and Target Wake time (TWT) for enhanced power savings. The OFDMA mode and trigger-based access enable scheduled operation, which is a key capability required to support deterministic latency and reliability for time-sensitive flows. 802.11ax can operate in up to 160 MHz channels and it includes support for operation in the new 6 GHz band, which is expected to be open to unlicensed use by the FCC and other regulatory agencies worldwide.
802.11ax introduced a new orthogonal frequency-division multiple access (OFDMA) mode in which multiple users can be scheduled across the frequency domain. In this mode, the Access Point (AP) can initiate multi-user (MU) Uplink (UL) transmissions in the same PHY Protocol Data Unit (PPDU) by sending a trigger frame. This centralized scheduling capability gives the AP much more control of the channel, and it can remove contention between devices for uplink transmissions, therefore reducing the randomness caused by CSMA-based access between stations. The AP can also transmit simultaneously to multiple users in the downlink direction by using a Downlink (DL) MU OFDMA PPDU. In order to initiate a contention free Transmission Opportunity (TXOP) using the OFDMA mode, the AP still follows the typical listen before talk procedure to acquire the medium, which ensures interoperability and compliance with unlicensed band access rules. However, 802.11ax also includes a multi-user Enhanced Distributed Channel Access (MU-EDCA) capability, which allows the AP to get higher channel access priority.
The 802.11ax PHY can operate with 0.8, 1.6 or 3.2 microsecond guard interval (GI). The larger GI options provide better protection against multipath, which is expected to be a challenge in industrial environments. The possibility to operate with smaller resource units (e.g. 2 MHz) enabled by OFDMA also helps reduce noise power and improve SNR, leading to better packet error rate (PER) performance.
802.11ax supports beamforming as in 802.11ac, but introduces UL MU MIMO, which helps improve reliability. The UL MU MIMO capability is also enabled by the trigger based access operation in 802.11ax.
The 802.11ax specification [IEEE80211ax] includes support for operation in the new 6 GHz band. Given the amount of new spectrum available as well as the fact that no legacy 802.11 device (prior 802.11ax) will be able to operate in this new band, 802.11ax operation in this new band can be even more efficient.
TSN capabilities, as defined by the IEEE 802.1 TSN standards, provide the underlying mechanism for supporting deterministic flows in a Local Area Network (LAN). The 802.11 working group has already incorporated support for several TSN capabilities, so that time-sensitive flow can experience precise time synchronization and timeliness when operating over 802.11 links. TSN capabilities supported over 802.11 (which also extends to 802.11ax), include:
The exiting 802.11 TSN capabilities listed above, and the 802.11ax OFDMA and scheduled access provide a new set of tools to better server time-sensitive flows. However, it is important to understand the tradeoffs and constraints associated with such capabilities, as well as redundancy and diversity mechanisms that can be used to provide more predictable and reliable performance.
Time-sensitive applications and TSN standards are expected to operate under a managed network (e.g. industrial/enterprise network). Thus, the Wi-Fi operation must also be carefully managed and integrated with the overall TSN management framework, as defined in the IEEE Std. 802.1Qcc specification [IEEE8021Qcc].
Some of the random-access latency and interference from legacy/unmanaged devices can be minimized under a centralized management mode as defined in IEEE Std. 802.1Qcc, in which admission control procedures are enforced.
Existing traffic stream identification, configuration and admission control procedures defined in IEEE Std. 802.11 QoS mechanism can be re-used. However, given the high degree of determinism required by many time-sensitive applications, additional capabilities to manage interference and legacy devices within tight time-constraints need to be explored.
As discussed earlier, the 802.11ax OFDMA mode introduces the possibility of assigning different RUs (frequency resources) to users within a PPDU. Several RU sizes are defined in the specification (26, 52, 106, 242, 484, 996 subcarriers). In addition, the AP can also decide on MCS and grouping of users within a given OFMDA PPDU. Such flexibility can be leveraged to support time-sensitive applications with bounded latency, especially in a managed network where stations can be configured to operate under the control of the AP.
As shown in [Cavalcanti_2019], it is possible to achieve latencies in the order of 1msec with high reliability in an interference free environment. Obviously, there are latency, reliability and capacity tradeoffs to be considered. For instance, smaller Resource Units (RU)s result in longer transmission durations, which may impact the minimal latency that can be achieved, but the contention latency and randomness elimination due to multi-user transmission is a major benefit of the OFDMA mode.
The flexibility to dynamically assign RUs to each transmission also enables the AP to provide frequency diversity, which can help increase reliability.
The 802.11be is the next major 802.11 amendment (after 802.11ax) for operation in the 2.4, 5 and 6 GHz bands. 802.11be is expected to include new PHY and MAC features and it is targeting extremely high throughput (at least 30 Gbps), as well as enhancements to worst case latency and jitter. It is also expected to improve the integration with 802.1 TSN to support time-sensitive applications over Ethernet and Wireless LANs.
The 802.11be Task Group started its operation in May 2019, therefore, detailed information about specific features is not yet available. Only high level candidate features have been discussed so far, including:
The 802.11 Real-Time Applications (RTA) Topic Interest Group (TIG) provided detailed information on use cases, issues and potential solution directions to improve support for time-sensitive applications in 802.11. The RTA TIG report [IEEE_doc_11-18-2009-06] was used as input to the 802.11be project scope.
Improvements for worst-case latency, jitter and reliability were the main topics identified in the RTA report, which were motivated by applications in gaming, industrial automation, robotics, etc. The RTA report also highlighted the need to support additional TSN capabilities, such as time-aware (802.1Qbv) shaping and packet replication and elimination as defined in 802.1CB.
802.11be is expected to build on and enhance 802.11ax capabilities to improve worst case latency and jitter. Some of the enhancement areas are discussed next.
In addition to the throughput enhancements, 802.11be will leverage the trigger-based scheduled operation enabled by 802.11ax to provide efficient and more predictable medium access. 802.11be is expected to include enhancements to reduce overhead and enable more efficient operation in managed network deployments [IEEE_doc_11-19-0373-00].
Multi-AP coordination is one of the main new candidate features in 802.11be. It can provide benefits in throughput and capacity and has the potential to address some of the issues that impact worst case latency and reliability. Multi-AP coordination is expected to address the contention due to overlapping Basic Service Sets (OBSS), which is one of the main sources of random latency variations. 802.11be can define methods to enable better coordination between APs, for instance, in a managed network scenario, in order to reduce latency due to unmanaged contention.
Several multi-AP coordination approaches have been discussed with different levels of complexities and benefits, but specific coordination methods have not yet been defined.
802.11be will introduce new features to improve operation over multiple bands and channels. By leveraging multiple bands/channels, 802.11be can isolate time-sensitive traffic from network congestion, one of the main causes of large latency variations. In a managed 802.11be network, it should be possible to steer traffic to certain bands/channels to isolate time-sensitive traffic from other traffic and help achieve bounded latency.
The IEEE 802.11ad amendment defines PHY and MAC capabilities to enable multi-Gbps throughput in the 60 GHz millimeter wave (mmWave) band. The standard addresses the adverse mmWave signal propagation characteristics and provides directional communication capabilities that take advantage of beamforming to cope with increased attenuation. An overview of the 802.11ad standard can be found in [Nitsche_2015] .
The IEEE 802.11ay is currently developing enhancements to the 802.11ad standard to enable the next generation mmWave operation targeting 100 Gbps throughput. Some of the main enhancements in 802.11ay include MIMO, channel bonding, improved channel access and beamforming training. An overview of the 802.11ay capabilities can be found in [Ghasempour_2017]
The high data rates achievable with 802.11ad and 802.11ay can significantly reduce latency down to microsecond levels. Limited interference from legacy and other unlicensed devices in 60 GHz is also a benefit. However, directionality and short range typical in mmWave operation impose new challenges such as the overhead required for beam training and blockage issues, which impact both latency and reliability. Therefore, it is important to understand the use case and deployment conditions in order to properly apply and configure 802.11ad/ay networks for time sensitive applications.
The 802.11ad standard include a scheduled access mode in which stations can be allocated contention-free service periods by a central controller. This scheduling capability is also available in 802.11ay, and it is one of the mechanisms that can be used to provide bounded latency to time-sensitive data flows. An analysis of the theoretical latency bounds that can be achieved with 802.11ad service periods is provided in [Cavalcanti_2019].
The IEEE802.15.4 Task Group has been driving the development of low-power low-cost radio technology. The Timeslotted Channel Hopping mode, added to the 2015 revision of the IEEE802.15.4 standard [IEEE802154], is targeted at the embedded and industrial world, where reliability, energy consumption and cost drive the application space.
The IEEE802.15.4 physical layer has been designed to support demanding low-power scenarios targeting the use of unlicensed bands, both the 2.4 GHz and sub GHz Industrial, Scientific and Medical (ISM) bands. This has imposed requirements in terms of frame size, data rate and bandwidth to achieve reduced collision probability, reduced packet error rate, and acceptable range with limited transmission power. The PHY layer supports frames of up to 127 bytes. The Medium Access Control (MAC) sublayer overhead is in the order of 10-20 bytes, leaving about 100 bytes to the upper layers. IEEE802.15.4 uses spread spectrum modulation such as the Direct Sequence Spread Spectrum (DSSS).
IPv6 over TSCH is enabled by the work done at the 6TiSCH WG. 6TiSCH has enabled best effort distributed scheduling to exploit the deterministic access capabilities provided by TSCH. The group designed the essential mechanisms to enable the management plane operation while ensuring IPv6 is supported. Yet the charter did not focus to providing a solution to establish end to end tracks while meeting quality of service requirements. 6TiSCH, through the RFC8480 [RFC8480] defines the 6P protocol which provides a pairwise negotiation mechanism to the control plane operation. The protocol supports agreement on a schedule between neighbors, enabling distributed scheduling. 6P goes hand-in-hand with a Scheduling Function (SF), the policy that decides how to maintain cells and trigger 6P transactions. The Minimal Scheduling Function (MSF) [I-D.ietf-6tisch-msf] is the default SF defined by the 6TiSCH WG; other standardized SFs can be defined in the future. MSF extends the minimal schedule configuration, and is used to add child-parent links according to the traffic load.
Time sensitive networking on low power constrained wireless networks have been addressed by ISA100.11a and WirelessHART. TODO
The 6TiSCH architecture [I-D.ietf-6tisch-architecture] already identified different models to schedule resources along tracks exploiting the TSCH schedule structure however these models have not been standardized. A Track, in the 6TiSCH architecture is considered a directed path from a source 6TiSCH node to one or more destination(s) 6TiSCH node(s) through the 6TiSCH network. A Track in 6TiSCH is the implementation of the deterministic path in the Detnet architecture [I-D.ietf-detnet-architecture] . Along a Track, 6TiSCH nodes reserve the resources to enable the efficient transmission of packets while aiming to optimize certain properties such as reliability and ensure small jitter or bounded latency. The track structure enables Layer-2 forwarding schemes, reducing the overhead of taking routing decisions at the Layer-3. Serial Tracks can be understood as the concatenation of cells or bundles along a routing path from a node towards a destination. The serial track concept is analogous to the circuit concept where resources are chained through the multi-hop topology. For example, A bundle of Tx Cells in a particular node is paired to a bundle of Rx Cells in the next hop node following a routing path. More complex approaches are described in and complemented by extensions to the RPL routing protocol in [I-D.ietf-roll-nsa-extension]. Reliability measures are for example achieved by exploiting concepts such as Replication and Elimination. In them, packets at origin are replicated and transmitted along disjoint tracks. This redundancy measure exploiting track forwarding increases energy consumption of the network nodes but improves significantly the reliability of the network.
Useful References include:
As a core technique in IEEE802.15.4, TSCH splits time in multiple time slots that repeat over time. The structure is referred as a Slotframe. For each timeslot, a set of available frequencies can be used, resulting in a matrix-like schedule (see Fig. Figure 1).
timeslot offset
| 0 1 2 3 4 | 0 1 2 3 4 | Nodes
+------------------------+------------------------+ +-----+
| | | | | | | | | | | | C |
CH-1 | EB | | |C->B| | EB | | |C->B| | | |
| | | | | | | | | | | +-----+
+-------------------------------------------------+ |
| | | | | | | | | | | |
CH-2 | | |B->C| |B->A| | |B->C| |B->A| +-----+
| | | | | | | | | | | | B |
+-------------------------------------------------+ | |
... +-----+
|
+-------------------------------------------------+ |
| | | | | | | | | | | +-----+
CH-15| |A->B| | | | |A->B| | | | | A |
| | | | | | | | | | | | |
+-------------------------------------------------+ +-----+
ch.
offset
Figure 1: Slotframe example with scheduled cells between nodes A, B and C
This schedule represents the possible communications of a node with its neighbors, and is managed by a Scheduling Function such as The Minimal Scheduling Function (MSF) [I-D.ietf-6tisch-msf]. Each cell in the schedule is identified by its slotoffset and channeloffset coordinates. A cell's timeslot offset indicates its position in time, relative to the beginning of the slotframe. A cell's channel offset is an index which maps to a frequency at each iteration of the slotframe. Each packet exchanged between neighbors happens within one cell. An Absolute Slot Number (ASN) indicates the number of slots elapsed since the network started. It increments at every slot. This is a 5 byte counter that can support networks running for more than 300 years without wrapping (assuming a 10 ms timeslot). Channel hopping provides increased reliability to multi-path fading and external interference. It is handled by TSCH through a channel hopping sequence referred as macHopSeq in the IEEE802.15.4 specification.
The Time-Frequency Division Multiple Access provided by TSCH enables the orchestration of traffic flows, spreading them in time and frequency, and hence enabling an efficient management of the bandwidth utilization. Such efficient bandwidth utilization can be combined to OFDM modulations also supported by the IEEE802.15.4 standard [IEEE802154] since the 2015 version.
In the RAW context, low power reliable networks should address non-critical control scenarios such as Class 2 and monitoring scenarios such as Class 4 defined by the RFC5673 [RFC5673]. As a low power technology targeting industrial scenarios radio transducers provide low data rates (typically between 50kbps to 250kbps) and robust modulations to trade-off performance to reliability. TSCH networks are organized in mesh topologies and connected to a backbone. Latency in the mesh network is mainly influenced by propagation aspects such as interference. ARQ methods and redundancy techniques such as replication and elimination should be studied to provide the needed performance to address deterministic scenarios.
Nodes in a TSCH network are tightly synchronized. This enables to build the slotted structure an ensure efficient utilization of resources thranks to proper scheduling policies. Scheduling is a key to orchestrate the resources that different nodes in a track or path are using. Slotframes can be split in resource blocks reserving the needed capacity to certain needs. Periodic and bursty traffic can be handled independently in the schedule, using active and reactive policies and taking advantage of certain cell overprovision. Along a track, resource blocks can be chained so nodes in previous hops transmit their data before those that come later. This provides a tight control to latency along a track. Redundancy is achieved in a best effort manner by overprovision, giving time to the management plane of the network to request more resources if needed. -time synchronization - scheduling capabilities, discuss such things as Resource Units, time slots or resource blocks. Can we reserve periodic resources vs. ask each time, what precision can we get in latency control. - diversity scenarios, what's available, - gap analysis, e.g. discuss multihop, or what's missing how to do PAREO features.
This specification does not require IANA action.
Most RAW technologies integrate some authentication or encryption mechanisms that were defined outside the IETF.
Many thanks to the participants of the RAW WG where a lot of the work discussed here happened.
| [I-D.ietf-6tisch-architecture] | Thubert, P., "An Architecture for IPv6 over the TSCH mode of IEEE 802.15.4", Internet-Draft draft-ietf-6tisch-architecture-20, March 2019. |
| [I-D.ietf-detnet-architecture] | Finn, N., Thubert, P., Varga, B. and J. Farkas, "Deterministic Networking Architecture", Internet-Draft draft-ietf-detnet-architecture-13, May 2019. |
| [RFC5673] | Pister, K., Thubert, P., Dwars, S. and T. Phinney, "Industrial Routing Requirements in Low-Power and Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October 2009. |
| [RFC8200] | Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017. |
| [RFC8480] | Wang, Q., Vilajosana, X. and T. Watteyne, "6TiSCH Operation Sublayer (6top) Protocol (6P)", RFC 8480, DOI 10.17487/RFC8480, November 2018. |