| NWCRG | J. Heide |
| Internet-Draft | Steinwurf Aps |
| Intended status: Informational | S. Shi |
| Expires: August 15, 2019 | K. Fouli |
| M. Médard | |
| Code On Network Coding LLC | |
| V. Chook | |
| Inmarsat PLC | |
| February 11, 2019 |
Random Linear Network Coding (RLNC): Background and Practical Considerations
draft-heide-nwcrg-rlnc-background-00
This document describes the use of Random Linear Network Coding (RLNC) schemes for reliable data transport. Both block and sliding window RLNC code implementations are described. By providing erasure correction using randomly generated repair symbols, such RLNC-based schemes offer advantages in accommodating varying frame sizes and dynamically changing connections, reducing the need for feedback, and lowering the amount of state information needed at the sender and receiver. The practical considerations' section identifies RLNC-encoded symbol representation as a valuable target for standardization.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 15, 2019.
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Network Coding is a coding discipline that jointly improves network reliability and efficiency. In general communication networks, source coding is performed as a compression mechanism to reduce data redundancy and to reduce resources necessary for data transportation over the network. Channel coding, on the other hand, introduces redundancy for data transmission reliability over lossy channels. Network coding adds another layer of coding in-between these two. Random Linear Network Coding (RLNC) is an efficient network coding approach that enables network nodes to generate independently and randomly linear mappings of input to output data symbols over a finite field.
This document provides background on RLNC operation. The document also includes an open section for practical considerations where topics such as standardization and RLNC-encoded symbol representations are addressed.
Unlike conventional communication systems based on the "store-and-forward" principle, RLNC allows network nodes to independently and randomly combine input source data into coded symbols over a finite field [HK03]. Such an approach enables receivers to decode and recover the original source data as long as enough linearly independent coded symbols, with sufficient degrees of freedom, are received. At the sender, RLNC can introduce redundancy into data streams in a granular way. At the receiver, RLNC enables progressive decoding and reduces feedback necessary for retransmission. Collectively, RLNC provides network utilization and throughput improvements, high degrees of robustness and decentralization, reduces transmission latency, and simplifies feedback and state management.
To encode using RLNC, original source data are divided into symbols of a given size and linearly combined. Each symbol is multiplied with a scalar coding coefficient drawn randomly from a finite field, and the resulting coded symbol is of the same size as the original data symbols.
Thus, each RLNC encoding operation can be viewed as creating a linear equation in the data symbols, where the random scalar coding coefficients can be grouped and viewed as a coding vector. Similarly, the overall encoding process where multiple coded symbols are generated can be viewed as a system of linear equations with randomly generated coefficients. Any number of coded symbols can be generated from a set of data symbols, similarly to expandable forward error correction codes specified in [RFC5445] and [RFC3453]. Coding vectors must be implicitly or explicitly transmitted from the sender to the receiver for successful decoding of the original data. For example, sending a seed for generating pseudo-random coding coefficients can be considered as an implicit transmission of the coding vectors. In addition, while coding vectors are often transmitted together with coded data in the same data packet, it is also possible to separate the transmission of coding coefficient vectors from the coded data, if desired.
To reconstruct the original data from coded symbols, a network node collects a finite but sufficient number of degrees of freedom for solving the system of linear equations. This is beneficial over conventional approaches as the network node is no longer required to gather each individual data symbol. In general, the network node needs to collect slightly more independent coded symbols than there are original data symbols, where the slight overhead arises because coding coefficients are drawn at random, with a non-zero probability that a coding vector is linearly dependent on another coding vector, and that one coded symbol is linearly dependent on another coded symbol. This overhead can be made arbitrarily small, provided that the finite field used is sufficiently large.
A unique advantage of RLNC is the ability to re-encode or "recode" without first decoding. Recoding can be performed jointly on existing coded symbols, partially decoded symbols, or uncoded systematic data symbols. This feature allows intermediate network nodes to re-encode and generate new linear combinations on the fly, thus increasing the likelihood of innovative transmissions to the receiver. Recoded symbols and recoded coefficient vectors have the same size as before and are indistinguishable from the original coded symbols and coefficient vectors.
In practical implementations of RLNC, the original source data are often divided into multiple coding blocks or "generations" where coding is performed over each individual generation to lower the computational complexity of the encoding and decoding operations. Alternatively, a convolutional approach can be used, where coding is applied to overlapping spans of data symbols, possibly of different spanning widths, viewed as a sliding coding window. In generation-based RLNC, not all symbols within a single generation need to be present for coding to start. Similarly, a sliding window can be variable-sized, with more data symbols added to the coding window as they arrive. Thus, innovative coded symbols can be generated as data symbols arrive. This "on-the-fly" coding technique reduces coding delays at transmit buffers, and together with rateless encoding operations, enables the sender to start emitting coded packets as soon as data is received from an upper layer in the protocol stack, adapting to fluctuating incoming traffic flows. Injecting coded symbols based on a dynamic transmission window also breaks the decoding delay lower bound imposed by traditional block codes and is well suited for delay-sensitive applications and streaming protocols.
When coded symbols are transmitted through a communication network, erasures may occur, depending on channel conditions and interactions with underlying transport protocols. RLNC can efficiently repair such erasures, potentially improving protocol response to erasure events to ensure reliability and throughput over the communication network. For example, in a point-to-point connection, RLNC can proactively compensate for packet erasures by generating Forward Erasure Correcting (FEC) redundancy, especially when a packet erasure probability can be estimated. As any number of coded symbols may be generated from a set of data symbols, RLNC is naturally suited for adapting to network conditions by adjusting redundancy dynamically to fit the level of erasures, and by updating coding parameters during a session. Alternatively, packet erasures may be repaired reactively by using feedback requests from the receiver to the sender, or by a combination of FEC and retransmission. RLNC simplifies state and feedback management and coordination as only a desired number of degrees of freedom needs to be communicated from the receiver to the sender, instead of indications of the exact packets to be retransmitted. The need to exchange packet arrival state information is therefore greatly reduced in feedback operations.
The advantages of RLNC in state and feedback management are apparent in a multicast setting. In this one-to-many setup, uncorrelated losses may occur, and any retransmitted data symbol is likely to benefit only a single receiver. By comparison, a transmitted RLNC coded symbol is likely to carry a new degree of freedom that may correct different errors at different receivers simultaneously. Similarly, RLNC offers advantages in coordinating multiple paths, multiple sources, mesh networking and cooperation, and peer-to-peer operations.
A more detailed introduction to network coding including RLNC is provided in the books [MS11] and [HL08].
This section describes a generation-based RLNC scheme.
In generation-based RLNC, input data as received from an upper layer in the protocol stack is segmented into equal-sized blocks, denoted as generations, and each generation is further segmented into equal-sized data symbols for encoding, with paddings added when necessary. Encoding and decoding are performed over each individual generation. Figure 1 below provides an illustrative example where each generation includes four data symbols, and a systematic RLNC code is generated with rate 2/3.
Data Symbols:
Generation-1 Generation-2
+============================++============================+
| +---+ +---+ +---+ +---+ || +---+ +---+ +---+ +---+ |
| | 1 | | 2 | | 3 | | 4 | || | 5 | | 6 | | 7 | | 8 | | ...
| +---+ +---+ +---+ +---+ || +---+ +---+ +---+ +---+ |
+============================++============================+
| |
| |
Systematic | |
Symbols: V V
+---++---++---++---++---++---+ +---++---++---++---++---++---+
| 1 || 2 || 3 || 4 || C1|| C2| | 5 || 6 || 7 || 8 || C3|| C4| ...
+---++---++---++---++---++---+ +---++---++---++---++---++---+
Figure 1: Generation-based RLNC with rate 2/3, systematic encoding performed on data symbols within each generation.
Symbols can be of any size, although symbol sizes typically depend on application or system specifications. In scenarios with highly varying input data frame sizes, a small symbol size may be desirable for achieving flexibility and transmission efficiency, with one or more symbols concatenated to form a coded data packet. In this context, existing basic FEC schemes [RFC5445] do not support the use of a single header for multiple coded symbols, whereas the symbol representation design as described in [Symbol-Representation] provides this option.
For any protocol that utilizes generation-based RLNC, a setup process is necessary for establishing a connection and conveying coding parameters from the sender to the receiver. Such coding parameters can include one or more of field size, code specifications, index of the current generation being encoded at the sender, generation size, code rate, and desired feedback frequency or probability. Some coding parameters are updated dynamically during the transmission process, reflecting the coding operations over sequences of generations, and adjusting to channel conditions and resource availability. For example, an outer header can be added to the symbol representation specified in [Symbol-Representation] to indicate the current generation encoded within the symbol representation. Such information is essential for proper recoding and decoding operations, but the exact design of the outer header is outside the scope of the current document. At the minimum, an outer header should indicate the current generation, generation size, symbol size, and field size. Section 2 provides a detailed discussion of coding parameter considerations.
This section describes a sliding-window RLNC scheme. Sliding window RLNC was first described in [SS09].
In sliding-window RLNC, input data as received from an upper layer in the protocol stack is segmented into equal-sized data symbols for encoding. In some implementations, the sliding encoding window can expand in size as new data packets arrive, until it is closed off by an explicit instruction, such as a feedback message that re-initiates the encoding window. In some implementations, the size of the sliding encoding window is upper bounded by some parameter, fixed or dynamically determined by online behavior such as packet loss or congestion estimation. Figure 3 below provides an example of a systematic finite sliding window code with rate 2/3.
Data Symbols:
+---+ +---+ +---+ +---+ +---+ +---+ +---+ +---+
| 1 | | 2 | | 3 | | 4 | | 5 | | 6 | | 7 | | 8 | ...
+--- +---+ +---+ +---+ +---+ +---+ +---+ +---+
| C1 | | | |
+==========+ | | |
| C2 | | |
+========================+ | |
| C3 | |
+========================+ |
| C4 |
+========================+ ...
|
|
Systematically |
Coded Symbols: V
+---++---++---++---++---++---++---++---++---++---++---++---+
| 1 || 2 || C1|| 3 || 4 || C2|| 5 || 6 || C3|| 7 || 8 || C4|...
+---++---++---++---++---++---++---++---++---++---++---++---+
Figure 3: Finite sliding-window RLNC with code rate 2/3, systematic encoding performed on data symbols within the sliding coding window.
For any protocol that utilizes sliding-window RLNC, a setup process is necessary for establishing a connection and conveying coding parameters from the sender to the receiver. Such coding parameters can include one or more of field size, code specifications, symbol ordering, encoding window position, encoding window size, code rate, and desired feedback frequency or probability. Some coding parameters can also be updated dynamically during the transmission process in accordance to channel conditions and resource availability. For example, an outer header can be added to the symbol representation specified in [Symbol-Representation] to indicate an encoding window position, as a starting index for current data symbols being encoded within the symbol representation. Again, such information is essential for proper recoding and decoding operations, but the exact design of the outer header is outside the scope of the current document. At the minimum, an outer header should indicate the current encoding window position, encoding window size, symbol size, and field size. Section 2 provides a detailed discussion of coding parameter considerations.
Once a connection is established, RLNC coded packets comprising one or more coded symbols are transmitted from the sender to the receiver. The sender can transmit in either a systematic or coded fashion, with or without receiver feedback. In progressive decoding of RLNC coded symbols, the notion of “seen” packets can be utilized to provide degree of freedom feedbacks. Seen packets are those packet that have contributed to a received coded packet, where generally the oldest such packet that has yet to be declared seen is declared as seen [SS09].
Recoding is the process where coded or partially decoded symbols are re-encoded without first being fully decoded. To recode, both coded symbols and corresponding coding coefficient vectors are linearly combined, respectively, with new randomly generated recoding coefficients. Recoded symbols and recoded coefficient vectors generally have the same size as before and are indistinguishable from the original coded symbols and coding coefficient vectors. Recoding is typically performed at intermediate network nodes, in either an intra-session or an inter-session fashion. Intra-session coding refers to coding between packets of the same flow or session, while inter-session coding refers to combination of packets from different flows or sessions in a network.
As recoding requires the same operations on the coding coefficient vectors as applied to the coded symbols, coding coefficients must be updated by recoding coefficients. This is generally achieved by having the coding coefficients accessible at recoding nodes so that they may be updated. Thus, either the original coding coefficients or reversible representations of the coding coefficients need to be communicated from upstream nodes to the recoding nodes.
This is an open section describing various practical considerations such as standardization approaches and implementation-related topics.
This sub-section argues for the specification of symbol representation as a starting point for network coding standardization and provides relevant coding parameter design considerations.
Symbol representation specifies the format of the symbol-carrying data unit that is to be coded, recoded, and decoded. In other words, symbol representation defines the format of the coding-layer data unit, including header format and symbol concatenation.
Network Coding has fundamentally different requirements from conventional point-to-point codes. Network coding owes its distinct requirements to its dynamic structure, leading to a highly reconfigurable symbol set. For example:
Useful symbol representations should include provisions for the major coding functions that are relevant to the application, such as recoding, feedback, or inter-session network coding. For example, recoding requires the coefficients to be accessible at the intermediate recoding nodes. Hence, architectures and protocols requiring recoding must specify coefficient location.
Furthermore, the example of Figure 4 illustrates how the knowledge of coefficient location affects the way a coded payload is fragmented, coded, and transported across the network.
(a) Code-aware fragmentation
+---+---------+
| C | D1 |
+---+---------+---------+ +---+---------+
| C | D1 | D2 | +---->
+---+---------+---------+ +---+---------+
| C | D2 |
+---+---------+
(b) Conventional fragmentation
+-----------+
| D1 |
+-----------+-----------+ +-----------+
| D1 | D2 | +---->
+-----------+-----------+ +-----------+
| D2 |
+-----------+
Figure 4: Network operations such as fragmentation may be affected by symbol representation. For example, whether coefficient location is (a) specified or (b) hidden may affect the way fragmentation is carried out.
In Figure 4 (a), coefficient locations are known, allowing the association of the coefficient set C to both fragments D1 and D2 of original payload. The ability to manipulate the original coefficients allows the newly formed packets to be recoded and decoded at the same coding layer. Figure 4 (b) shows a case where the coding coefficient location are unknown. This may occur when the file is pre-coded by a higher layer such as the application layer, or when coefficients are deliberately hidden for security reasons, leading to typical fragmentation without coefficient replication.
The absence of information on coefficient location has important implications. One such implication is that any additional coding needs to be carried out within a new coding layer, potentially leading to higher computational and transport overheads.
The elements discussed above demonstrate that the design choices related to symbol representation have a direct impact on the viability of protocols, topologies, and architecture. The importance of symbol representation is illustrated in Figure 5, where the term "architecture" includes coding architecture (e.g., generation or sliding window), the layer placement of coding operations, and coding objectives (e.g., erasure correction, multisourcing, etc.).
+---------------+
|Architecture |
| | Symbol
| | Representation
| |
+-------------------+ | ^
|Topology | | | |
| | +-------------------+ |
| | |----| | | |
| | |----| <----------------+
| | |----| | |
| +---------------+ |
| | | |
+-------------------+ |
| |
| Protocol|
+-------------------+
Figure 5: The specification of symbol representation has major implications on system architecture, topology, and protocol.
Since symbol representation has implications on core design elements, it is expected that coding implementations that share protocol, architecture, and topology elements are likely to reuse the same symbol representation. For example, implementations with security requirements can reuse a common symbol representation that hides coefficient locations.
Another example can be found in [Symbol-Representation], which specifies symbol representation designs for generation-based and sliding window RLNC implementations. These designs introduce highly reusable formats that concatenate multiple symbols and associate them with a single symbol representation header.
For any protocol that utilizes generation-based or sliding-window RLNC, several coding parameters must be communicated from the sender to the receiver as part of the protocol design. Without elaborating on all such parameters, this section examines those essential for symbol representation design, including field size, symbol size, maximum number of symbols, and maximum generation or window size.
As RLNC is performed over a finite field, field size determines the complexity of the required mathematical computations. Larger field sizes translate to higher probability of successful decoding, as randomly generated coding coefficient vectors are more likely to be independent from each other. However, larger field sizes may also result in higher computational complexity, leading to longer decoding latency, higher energy usage, and other hardware requirements for both the encoder and the decoder. A finite field size of 2 or the binary Finite Field FF(2) should always be supported since addition, multiplication, and division over FF(2) are equivalent to elementary XOR, AND, and IDENTITY operations respectively. It is also desirable to support a field size of 2^8, corresponding to a single byte, and where operations are performed over the binary extension field FF(2^8) with polynomial x^8+x^4+x^3+x^2+1.
The choice of symbol size typically depends on the application or system specification. For example, a symbol size may be chosen based on the size of a maximum transmission unit (MTU) so that datagrams are not fragmented as they traverse a network, while also ensuring no symbol bits are unnecessarily wasted. A symbol representation design can be flexible and accommodate any symbol size in bytes. For example, an IP packet is typically in the range between 500 and 1500 bytes, while a much smaller datagram having a size of 90 bytes may be used by satellite communication networks. The symbol representation in [Symbol-Representation] can be configured to support either or both cases in different implementations.
The generation size or coding window size is a tradeoff between the strength of the code and the computational complexity of performing the coding operations. With a larger generation/window size, fewer generations or coding windows are needed to enclose a data message of a given size, thus reducing protocol overhead for coordinating individual generations or coding windows. In addition, a larger generation/window size increases the likelihood that a received coded symbol is innovative with respect to previously received symbols, thus amortizing retransmission or FEC overheads. Conversely, when coding coefficients are attached, larger generation/window sizes also lead to larger overheads per packet. The generation/window size to be used can be signaled between the sender and receiver when the connection is first established.
Lastly, to successfully decode RLNC coded symbols, sufficient degrees of freedom are required at the decoder. The maximum number of redundant symbols that can be transmitted is therefore limited by the number of linearly independent coding coefficient vectors that can be supported by the system. For example, if coding vectors are constructed using a pseudo-random generator, the maximum number of redundant symbols that can be transmitted is limited by the number of available generator states.[RFC5445]
This document does not present new security considerations.
This document has no actions for IANA.
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC3453] | Luby, M., Vicisano, L., Gemmell, J., Rizzo, L., Handley, M. and J. Crowcroft, "The Use of Forward Error Correction (FEC) in Reliable Multicast", RFC 3453, DOI 10.17487/RFC3453, December 2002. |
| [RFC5445] | Watson, M., "Basic Forward Error Correction (FEC) Schemes", RFC 5445, DOI 10.17487/RFC5445, March 2009. |
| [Symbol-Representation] | Heide, J., Shi, S., Fouli, K., Médard, M. and V. Chook, "Random Linear Network Coding (RLNC)-Based Symbol Representation", February 2018. |
| [HK03] | Ho, T., Koetter, R., Médard, M., Karger, D. and M. Effros, "The Benefits of Coding over Routing in a Randomized Setting", July 2003. |
| [HL08] | Ho, T. and D. Lun, "Network Coding: An Introduction", April 2008. |
| [MS11] | Médard, M. and A. Sprintson, "Network Coding: Fundamentals and Applications", October 2011. |
| [SS09] | Sundararajan, J., Shah, D., Médard, M., Mitzenmacher, M. and J. Barros, "Network Coding Meets TCP", April 2009. |