Internet DRAFT - draft-guo-detnet-compensation-reference
draft-guo-detnet-compensation-reference
DetNet D. Guo
Internet Draft X. You
Intended status: Informational R. Liu
Expires: 11 June 2024 S. Zhu
New H3C Technologies Co., Ltd
11 December 2023
Compensation Reference for Jitter Reduction Mechanism
draft-guo-detnet-compensation-reference-00.txt
Abstract
This document describes several methods for obtaining reference delay
applicable to different scenarios. These methods are in conjunction
with mechanisms aimed at reducing end-to-end delay jitter in a
scalable deterministic network. The goal is to meet specific business
requirements and provide greater flexibility in the multipath
planning for service protection.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 11, 2024.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
Guo, et al. Expires June 11, 2024 [Page 1]
Internet-Draft Compensation Reference for JRM December 2023
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
Abstract..........................................................1
Status of this Memo...............................................1
Table of Contents.................................................2
1. Introduction...................................................2
2. Terminology....................................................3
3. Network Model..................................................4
3.1. SN Model..................................................6
3.2. AN Model..................................................7
4. Compensation Reference Delay Collection Methods...............10
4.1. Compensation Reference Collection for SN.................12
4.1.1. Implementation Principle............................12
4.1.2. Data Plane..........................................13
4.1.3. Control Plane.......................................13
4.2. One-Way Compensation Reference Collection for AN.........15
4.2.1. Implementation Principle............................15
4.2.2. Data Plane..........................................18
4.2.3. Control Plane.......................................18
4.3. Round-Trip Compensation Reference Collection for AN......20
4.3.1. Implementation Principle............................21
4.3.2. Data Plane..........................................24
4.3.3. Control Plane.......................................25
5. Security Considerations.......................................26
6. IANA Considerations...........................................26
7. Acknowledgments...............................................26
8. Contributors..................................................26
9. References....................................................27
9.1. Informative References...................................27
Authors' Addresses...............................................29
1. Introduction
In scaling deterministic networks, as defined in [I-D.ietf-detnet-
scaling-requirements], end-to-end services may traverse multiple
network domains and employ various queuing mechanisms within each
domain. [I-D.guo-detnet-jitter-reduction-mechanism]describes a
compensation mechanism designed to reduce end-to-end jitter and meet
the requirements of applications with tightly bounded jitter. This
Guo, et al. Expires June 11, 2024 [Page 2]
Internet-Draft Compensation Reference for JRM December 2023
document expands on the Jitter Reduction Mechanism for DetNet [I-
D.guo-detnet-jitter-reduction-mechanism].
For many applications, it is crucial that different copies of the
same data transmitted through multiple paths maintain consistent
delays within specified upper and lower bounds. This ensures that in
the event of a path failure, the application remains unaware of the
fault. Therefore, a unified value is required as the compensation
delay reference for the transmission delays in multi-path transport
providing service protection. This value is referred to as the Path
Delay Reference, denoted as PthRefD in [I-D.guo-detnet-jitter-
reduction-mechanism]. This value will be greater than the actual
transmission delays on each path to accommodate additional delays
introduced by anticipated future path changes (e.g., when a new path
is added to the service protection path group after a path failure,
and the new path is longer than the original paths in the service
protection path group). By using the Path Delay Reference to
compensate for transmission delays, applications remain unaware of
such fault occurrences and recoveries within a bounded delay range.
In certain scenarios, complete transmission delays for data during
transmission may not be obtainable. According to [I-D.guo-detnet-
jitter-reduction-mechanism], it is sufficient to obtain the upper
limit of the compensation delay required for the corresponding path.
In this document, the "upper limit of the compensation delay required
for the corresponding path" is referred to as Cap_i, simplified as
the compensation reference delay value in this document.
Given the existence of both time-synchronized and non-time-
synchronized scenarios in practical requirements, and considering the
diversity of actual network deployments and the complexity of the
issues themselves, it is almost impossible to have a unified method
to address all requirements. Our approach is to abstract the network
into two typical models and provide suitable methods for each model.
In the following sections, we will first categorize and describe the
network models and then provide methods for collecting the
compensation reference delay values corresponding to different types.
Different methods will be implemented for different network types in
actual deployment to simplify the realization.
2. Terminology
The following terminology is introduced in this document on the basis
of [I-D.guo-detnet-jitter-reduction-mechanism]:
HN
Guo, et al. Expires June 11, 2024 [Page 3]
Internet-Draft Compensation Reference for JRM December 2023
Abbreviation of Head Node. For the definition of Head Node,
please refer to [I-D.guo-detnet-jitter-reduction-mechanism].
CN
Abbreviation of Compensation Node. For the definition of
Compensation Node, please refer to [I-D.guo-detnet-jitter-
reduction-mechanism].
Synchronous Networking (SN)
It is a networking model in which time synchronization is
implemented between HN and CN.
Asynchronous Networking (AN):
It is a networking model in which there is no time
synchronization between HN and CN.
Synchronous Domain (SD)
If time synchronization is achieved between the I-GW and E-GW of
the deterministic network domain through which the deterministic
flow path passes, the deterministic network domain where the I-GW
and E-GW are located is considered a synchronous domain.
3. Network Model
Before deploying deterministic service flows, the controller plane
performs path planning for the network, identifies the networking
model to which the completed planned paths belong, and labels the
paths accordingly. In the path from the Head Node (HN) to the
Compensation Node (CN), if time synchronization is achieved between
HN and CN, we consider the networking model for the deployment of
deterministic flows to be SN, and we collect reference values using
the SN method. In the path from HN to CN, if time synchronization is
not achieved between HN and CN, we consider the networking model for
the deployment of deterministic flows to be AN, and we collect
reference values using the AN method.
For AN, the path from HN to CN may contain Synchronous Domains (SD)
consisting of multiple network nodes, each with independent Ingress
Gateway (I-GW) and Egress Gateway (E-GW) devices. There may also be
individual network nodes within the path that are not time-
synchronized with other nodes. These nodes can be considered as
special SDs, where a single node forms its own domain, with its I-GW
Guo, et al. Expires June 11, 2024 [Page 4]
Internet-Draft Compensation Reference for JRM December 2023
and E-GW being the node itself, and time synchronization is required
within the node.
Upon entering an SD from the I-GW, Packet Replication Functions (PRF)
may be implemented, and PRF and Packet Elimination Functions (PEF)
may be implemented before or within the E-GW. The packets output from
the E-GW of the SD may not be the original packets but rather
multiple copies of the original packets. The copies generated by the
internal PRF function within the SD may follow different transmission
paths, and the transmission delays on different paths are different.
The differences in delay introduced by the selection of packets from
different paths by the PEF are considered as jitter within the SD.
The controller plane actually performs path planning to plan out
candidate paths that meet application requirements. Some of the
candidate paths are selected to form a service protection group,
while the remaining candidate paths can serve as backup paths in the
event of a failure. The method described in this document is based on
the paths that have already been planned, to collect compensation
reference delay values.
Guo, et al. Expires June 11, 2024 [Page 5]
Internet-Draft Compensation Reference for JRM December 2023
3.1. SN Model
HN(PE1) and CN(PE2) synchronized
^ ^
^ ^
^ ^
^ P1--------------P2 ^
^ /| |\ ^
^ / | | \ ^
PE1 + | | + PE2
(HN) + | | + (CN)
| \ | | / |
| \| |/ |
| P3--------------P4 |
| |
| |
| |
| |
Talker Listener
(a) an example of SN
+---------------------+
Tanlker------+ I-GW(HN)---E-GW(CN) +-------Listener
+---------------------+
(b) the model of SN
Figure 1 Example and Model Abstraction of SN
Figure 1(a) shows an example of SN. The Talker is connected to PE1,
which acts as the Head Node (HN), while the Listener is connected to
PE2, which acts as the Compensation Node (CN). Time synchronization
exists between PE1 and PE2.
The application networking for accessing SN adopts the abstraction
model shown in Figure 1(b), which is represented by the first I-GW
(i.e., HN) and the last E-GW (i.e., CN). The residence delay between
HN and CN is considered as variable delay within the SN, without
focusing on the specific paths within the SN, thus simplifying
implementation.
Guo, et al. Expires June 11, 2024 [Page 6]
Internet-Draft Compensation Reference for JRM December 2023
3.2. AN Model
no time synchronization between HN and CN
^ ^
^ ^
^ ^
^ P1--------------P2 ^
^ /| |\ ^
^ / | | \ ^
PE1 + | | + PE2
(HN) + | | + (CN)
| \ | | / |
| \| |/ |
| P3--------------P4 |
| |
| |
| |
| |
Talker Listener
(a) an example of AN
+------+ +------+
| +---P1----P2------------------+ |
| | | |
| +---P1----P2----P4------------+ |
| | | |
| +---P1----P3----P4------------+ |
| HN | | CN |
|(PE1) +---P1----P3----P4----P2------+ (PE2)|
| | | |
| +---P3----P1----P2----P4------+ |
| | | |
| +---P3----P1----P2------------+ |
| | | |
| +---P3----P4----P2------------+ |
| | | |
| +---P3----P4------------------+ |
+------+ +------+
(b) the model of AN
Figure 2 Example and Model Abstraction of AN
Guo, et al. Expires June 11, 2024 [Page 7]
Internet-Draft Compensation Reference for JRM December 2023
Figure 2(a) shows a simple example of AN. PE1 is connected to the
Talker and acts as the Head Node (HN), while PE2 is connected to the
Listener and acts as the Compensation Node (CN). There is no
requirement for time synchronization between the various P, PE1, and
PE2 nodes (Note: time synchronization can be present but is not
utilized). Assuming each P node is a physical node, two-downstream-
interface PRF is planned for each P node. Figure 2(b) shows the
abstract model of AN. In this diagram, there are 8 different paths
formed between PE1 and PE2 through different nodes.
After path planning, a portion or all of the paths are used to form
service protection members between HN and CN.
Guo, et al. Expires June 11, 2024 [Page 8]
Internet-Draft Compensation Reference for JRM December 2023
no time synchronization between HN and CN
^ ^
^ ^
^ 1 2 ^
^ SD1--------------SD2 ^
^ 2/|0 0|\1 ^
^ 1 / | | \ ^
PE1 + | | + PE2
(HN) + | | + (CN)
| 0 \ | | / |
| 2\|1 1|/0 |
| SD3--------------SD4 |
| 0 2 |
| |
| |
| |
Talker Listener
(a) an example of complex AN
+------+ +------+
| +--2-SD1-1--2-SD2-1--------------------+ |
| | | |
| +--2-SD1-1--2-SD2-0--1-SD4-0-----------+ |
| | | |
| +--2-SD1-0--1-SD3-0--2-SD4-0-----------+ |
| HN | | CN |
|(PE1) +--2-SD1-0--1-SD3-0--2-SD4-1--0-SD2-1--+ (PE2)|
| | | |
| +--2-SD3-1--0-SD1-1--2-SD2-0-----------+ |
| | | |
| +--2-SD3-1--0-SD1-1--2-SD-1------------+ |
| | | |
| +--2-SD3-0--2-SD4-1--0-SD2-1-----------+ |
| | | |
| +--2-SD3-0--2-SD4-0--------------------+ |
+------+ +------+
(b) a model of complex AN
Figure 3 Complex AN Example and Model Abstraction
Guo, et al. Expires June 11, 2024 [Page 9]
Internet-Draft Compensation Reference for JRM December 2023
Figure 3(a) shows an example of complex AN. PE1 is connected to the
Talker and acts as the Head Node (HN), while PE2 is connected to the
Listener and acts as the Compensation Node (CN). SD represents a
deterministic network domain after time synchronization by edge
gateways. There is no requirement for time synchronization between
SD, PE1, and PE2 (Note: time synchronization can be present but is
not utilized). Assuming two-downstream-interface PRF copies are
planned for each SD, Figure 3(b) shows the abstract model of this
complex AN. Taking "2-SD1-1" in the figure as an example, 2
represents the input interface number of the I-GW of SD1, the first 1
represents the corresponding SD number in Figure 3(a), and the second
1 represents the output interface number of E-GW of SD1. The
numbering of SDs and interfaces is for illustrative purposes,
primarily for the sake of simplicity and clarity in describing the
network topology as multiple paths. In Figure 3(b), the differences
in transmission delay within the SD are considered as jitter within
the domain, and 8 different paths are formed between PE1 and PE2
through different nodes. After path planning, a portion or all of the
paths are used to form service protection members between HN and CN.
4. Compensation Reference Delay Collection Methods
As mentioned earlier, time synchronization between the I-GW and E-GW
of SD (treated the same as SN) allows for the direct acquisition of
residence delay within it, eliminating the need to consider where
copies are generated inside the SD/SN, which path each copy is
transmitted on, and the final selection of the copy. Instead, the
differences in transmission delay of copies from different paths are
uniformly considered as jitter within the SD/SN, greatly simplifying
implementation without compromising the reliability of deterministic
elements, covering the application scenarios proposed in [I-D.draft-
ietf-detnet-mpls-over-ip-preof].
In AN, there is no time synchronization between HN and CN, making it
impossible to directly obtain end-to-end network residence time. In a
complex AN containing SD, the residence delay within the SD through
which the HN->CN path passes can be obtained through the cooperation
of the SD's I-GW and E-GW, independent of other nodes within the SD.
The input interface of the SD's I-GW and the output interface of the
E-GW participate in path identification, as do the input and output
interfaces of individual nodes.
Guo, et al. Expires June 11, 2024 [Page 10]
Internet-Draft Compensation Reference for JRM December 2023
+------------+
+---------------E1---+ | |
+---+ | | +---R3---+ | +---+
|src|------R1 +---+ | E3----O----+dst|
+---+ | | E2-------+ +---+
+----------R2 |
+-----------------+
R: replication function (PRF)
E: elimination function (PEF)
O: ordering function (POF)
Figure 4 PREOF scenario in a DetNet network
Figure 4 is copied from [I-D.ietf-detnet-mpls-over-ip-preof]. During
data forwarding, the E-node needs to implement PEF to remove the
redundant copies generated by multiple PRFs. However, when collecting
compensation reference delay values in OAM packets, the copies
generated by PRFs need to be retained, and the CN discards them after
compensation reference delay values are applied.
Next, we will introduce the compensation reference value collection
methods for synchronous and asynchronous networks, respectively.
Guo, et al. Expires June 11, 2024 [Page 11]
Internet-Draft Compensation Reference for JRM December 2023
4.1. Compensation Reference Collection for SN
4.1.1. Implementation Principle
| |
|<------------------PthRefD------------------------>|
| |
|t0 |t1 |t2 |t3 |tr
+------------------------+-----+-----------+------->+
| | | | |
| | | | |
| Copy3 /------------>+ | | |
| / | | | |
Copy2| /-----/---------------------+---------->+<-Tr_2->+
| / | | |
Copy1+/---------------------------->+ |
| | |
Figure 5 SN Compensation Reference Delay
For the SN, the compensation reference delay is denoted as the path
reference delay (PthRefD).
When the timestamp entering the HN is set as the starting time t0,
and the arrival times of the copies generated by each PRF at the CN
are denoted as t_i (where 0 < i < n + 1, and n is the number of
received copies). Let t_k (where 0 < k < n + 1) represent the latest
arrival time of all the copies at the CN. Based on t_k - t_0, an
additional value is added as the compensation reference value,
determined by the user or controller based on engineering
requirements. This additional value includes the extra transmission
delay, the margin for variable delay, and the processing delay at the
CN before scheduling. This value is the PthRefD.
Note: To simulate obtaining the HN receive timestamp, after
constructing the OAM message, the HN sends it to the receive
interface, and the receive interface then loops it back to the HN.
As shown in Figure 5, the timestamp for receiving the OAM loopback
message at the HN is t0. On the HN, PRF is implemented, generating
Copy1 and Copy2. Copy2 is further processed by a node inside the SD,
generating Copy3. The timestamps for the arrival of the three copies
at the CN are denoted as t3. Adding Tr_3 to t3 provides the value tr,
where Tr_3 is a value chosen by the user or controller based on
Guo, et al. Expires June 11, 2024 [Page 12]
Internet-Draft Compensation Reference for JRM December 2023
engineering requirements. The value tr represents the reference time
point for the path reference delay (PthRefD) relative to t0 on the
CN. It can also be understood as adding the user or controller-
selected value Tr_3 to (t3-t0) to obtain the path reference delay
(PthRefD).
Therefore, we have:
PthRefD = tr - t0, or
PthRefD = (t3 - t0) + Tr_3.
The PthRefD for the SN represents the compensation reference.
4.1.2. Data Plane
For the one-way collection of compensation reference for the SN, the
data plane behaviors of each node are described as follows:
1. The HN data plane receives the OAM message for collecting
compensation reference delay and transforms it into an OAM packet for
collecting compensation reference delay.
2. The HN sends the OAM packet for collecting compensation reference
delay. (e.g.: To simulate the reception of app-flow through loopback
on the HN: assuming the Talker is connected to interface if_0 of the
HN, an OAM packet with loopback indication is sent to if_0. Upon
receiving this packet after looping back, the receiving timestamp is
recorded. After performing the PRF on the HN, the OAM copies are sent
to the CN via different paths planned by the controller plane.)
3. At each node implementing PEF on the HN->CN path, when identifying
OAM for collecting SN compensation reference delay, the node does not
eliminate duplicate copies and directly forwards the message.
4. At the CN node, upon receiving the OAM packet for collecting SN
compensation reference delay, the measurement data is reported to the
control plane.
At the CN, it is necessary to ensure that a set of copies of an OAM
packet covers all the PREOF paths within the domain.
4.1.3. Control Plane
For the one-way collection of compensation reference for the SN, the
control plane behaviors of each node are described as follows:
Guo, et al. Expires June 11, 2024 [Page 13]
Internet-Draft Compensation Reference for JRM December 2023
1. Plan network paths.
2. Configure operations for nodes HN and CN.
3. The control plane of CN needs to specify conditions of integrity
check (ensuring the delays for all planned paths are collected).
4. The control plane needs to allocate OAM packet sequence numbers
and send messages to HN to construct OAM packets with these sequence
numbers. These OAM packets are used to collect the compensation
reference delay.
5. Coordinate with the data plane to perform measurements and
calculate the reference delay. The control plane receives measurement
data reported by CN, calculates the longest transmission delay
between HN and CN, and adds a value chosen by the user or controller
based on engineering requirements (including additional values for
line transmission, margin for variable delay, and processing delay at
CN before scheduling) to obtain the compensation reference delay for
HN->CN. Subsequently, the transmission delay for packets of DetNet
flow is compensated based on the actual residence delay and this
compensation reference delay.
6. Configure the compensation reference delay value to HN or CN. If
configured on the HN node, the HN node can determine the point in
time after compensation, simplifying CN implementation. If configured
on the CN node, the CN node determines the point in time after
compensation.
7. Runtime maintenance: periodically collect and calculate the full
path reference delay for HN->CN, and refresh the reference delay for
HN or CN. Isolate paths that have faults, and release isolation after
recovery.
For the collection of residence delay, the control plane needs to
provide relevant information to the data plane of each network node
in two ways:
Option 1: The control plane globally assigns node identifiers and
distributes these identifiers to each network node. The control plane
sends delay collection-related operations to HN.
Option 2: The control plane configures operations on a per-flow basis
to HN and CN.
Guo, et al. Expires June 11, 2024 [Page 14]
Internet-Draft Compensation Reference for JRM December 2023
4.2. One-Way Compensation Reference Collection for AN
The one-way collection method is suitable for cases where the round-
trip paths are different. Refer to [RFC9016] for the description of
BiDir. According to the compensation delay formula in [I-D.guo-
detnet-jitter-reduction-mechanism]:
CompD_i = Cap_i - (T'1_i + T'2_i + T'3_i + T'4_i)
Cap_i is obtained before the deployment of DetNet-flow sessions and
is used as the compensation reference delay value. This value is the
difference between PthRefD and FixD_i. Since time synchronization has
not been performed, PthRefD and FixD_i cannot be directly obtained.
Therefore, the compensation reference delay value Cap_i needs to be
transformed into a comparable relative value.
4.2.1. Implementation Principle
| |
|<------------------PthRefD------------------------>|
| |
|t_0 |t_1 |t_3 |tr
+--------------------------------+---------+------->+
| | | |
| | | |
| | | |
| | | |
Path2+--------------------------------+-------->+<-Tr_2->+
| | | |
Path1+------------------------------->+<------Tr_1------>+
| | |
Figure 6 Multi-path reference delay and compensation delay
According to section 3.2, the paths between HN and CN can be
abstracted as multiple paths. To simplify, consider Figure 6 with 2
paths and describe the method for calculating the compensation
reference delay value. Let t0 be the time when the measurement data
packet is received at HN, and let t_1 and t_2 be the times when two
copies of the same measurement data packet arrive at CN via Path1 and
Path2, respectively. Obviously, t_2 is the arrival time of the last
copy. tr is the reference time point based on t_2 delayed by Tr_2,
Guo, et al. Expires June 11, 2024 [Page 15]
Internet-Draft Compensation Reference for JRM December 2023
where the value of Tr_2 is chosen by the user or SDN controller based
on engineering requirements (including additional values for line
transmission, margin for variable delay, and processing delay at CN
before scheduling). The time difference between t_0 and tr
corresponds to PthRefD as described in [I-D.guo-detnet-jitter-
reduction-mechanism]. Since t_0 represents the time at HN, and t_1,
t_2, and tr represent the time at CN, and there is no time
synchronization between HN and CN, PthRefD cannot be directly
obtained by subtraction. However, t_1, t_2, and tr are all times at
CN, and the relative delay differences for each path can be obtained.
PthRefD = (FixD_1 + ActRefD_1) + Tr_1
= FixD_1 + (ActRefD_1 + Tr_1)
PthRefD - FixD_1 = ActRefD_1 + Tr_1
Where Cap_1 = PthRefD - FixD_1,
Therefore,
Cap_1 = ActRefD_1 + Tr_1
That is, Cap_1 is the sum of (ActRefD_1 + Tr_1), FixD_1 is the fixed
delay of Path1, which is constant and not considered in compensation.
ActRefD_1 is the sum of the residence delays of the measurement
packets in each domain, and Tr_1 is the delay from t1 to tr, i.e.,
tr-t1. Based on the method for obtaining actual transmission delay
within the domain as described in [I-D.guo-detnet-jitter-reduction-
mechanism], ActRefD_1 can be obtained, and thus Cap_1 can be
obtained; similarly, Cap_2 can also be obtained.
Path1
-------------------------------->
---- ----
/ \ Path2 / \
| HN | --------------------------------> | CN |
\ / \ /
---- Path3 ----
-------------------------------->
Figure 7 Illustration of OAM packet transmission
Guo, et al. Expires June 11, 2024 [Page 16]
Internet-Draft Compensation Reference for JRM December 2023
As shown in Figure 7, according to the service protection
implementation, HN will send multiple copies of the same OAM packet
on multiple paths, and CN will collect information and calculate the
compensation reference delay value for each member path. The process
to obtain the Cap_i value is as follows:
1. Record the corresponding path and reception time at CN;
simultaneously, according to the method described in [I-D.guo-detnet-
jitter-reduction-mechanism], obtain the residence delays in each
domain and accumulate them together to obtain the actual residence
delay of the OAM packets for each path. For ease of description,
let's assume that Path i is an arbitrary path for service protection,
and the accumulated value of the actual residence delay within each
domain after the OAM packet passes through this path is denoted as
ActRefD_i.
2. Collect the latest arrival time t_2 (as shown in Figure 7).
3. Based on engineering requirements, select the required delay
adjustment Tr_2 (as shown in Figure 7), and delay the latest arrival
time to obtain the reference time point tr, i.e., tr = t_2 + Tr_2.
4. Calculate the difference between the arrival time of the
measurement packet for each path relative to tr, denoted as Tr_i
(such as Tr_1, Tr_2 in Figure 7).
Tr_i = tr - t_i;
5. Calculate the compensation reference delay value for each path i
as Cap_i = ActRefD_i + Tr_i.
When calculating the compensation delay for each path, if a data
packet passes through Path i and the residence delay in each domain
is ActD_i (ActD_i is a variable delay and may not be the same after
each transmission), then the required compensation delay is
CompD_i = Cap_i - ActD_i.
Cap_i can be distributed to the HN node, encapsulated into the
DetNet-flow data packet, and used by CN to perform delay compensation
using Cap_i and ActD_i obtained from the data packet; or Cap_i can be
distributed to the flow-related compensation information table at CN
for delay compensation.
Guo, et al. Expires June 11, 2024 [Page 17]
Internet-Draft Compensation Reference for JRM December 2023
4.2.2. Data Plane
Description of the data plane behavior for the one-way collection of
compensation reference delay at each node:
1. HN's data plane receives the OAM message for collecting
compensation reference delay values and converts it into an OAM
packet for collecting compensation reference delay values.
2. HN sends the OAM packet for collecting compensation reference
delay.
Example: Simulating the reception of deterministic data via loopback
at HN: Assuming the Talker is connected to interface if_0 of HN, an
OAM packet marked for loopback is sent to if_0. When this packet is
looped back and received, the reception timestamp is recorded. After
performing the PRF, different copies of the OAM packet are sent to CN
via different paths planned by the controller plane.
3. Each node (if it is an edge gateway I-GW/E-GW of the SD, it is
handled by the SD's I-GW and E-GW) collects the residence delay of
the OAM packets passing through the path and, with the assistance of
the control plane, generates the necessary identification information
for identifying the path. Similar to the requirements in [I-D.ietf-
detnet-pof], when CN nodes receive an OAM packet or a service data
packet, they need to obtain path-related information.
4. At each node implementing PEF on the HN->CN path, when identifying
OAM for collecting compensation reference delay, the node does not
eliminate duplicate copies and directly forwards the message.
5. When CN nodes receive OAM packets, they record the reception time
of each OAM packet and send it to the control plane.
4.2.3. Control Plane
The control plane behavior of each node for one-way collection of
compensation reference delay for AN is described as follows:
1. Plan network paths and identify the network as SN or AN.
2. Configure the necessary information for generating path
identifiers at each node. Similar to the requirements in [I-D.ietf-
detnet-pof], when CN nodes receive an OAM packet or a service data
packet, they need to obtain path-related information.
Guo, et al. Expires June 11, 2024 [Page 18]
Internet-Draft Compensation Reference for JRM December 2023
3. Configure the path identifier information on CN to identify
different paths on which OAM packets or app-flow packets transmitted.
4. Set up a timeout mechanism in the control plane to identify if the
received OAM packet arrival time is acceptable. The historical window
mechanism on TSN in [IEEE802.1CB] can be used as a reference.
5. Configure operations at intermediate nodes to collect residence
delays.
6. Perform integrity checks in the control plane.
7. Allocate OAM packet sequence numbers and send messages to HN for
sending OAM packets to collect compensation reference delays.
8. Coordinate with the data plane to perform measurements and
calculate reference delay values. The control plane receives
measurement data reported by CN, calculates compensation reference
delay values, and records the arrival timestamps of multiple copies
of the same OAM packet received on each path. It also records the
total number of copies received on each path (used for integrity
checks) and the sum of residence delays within each SD and non-SD
node on that path. Based on the latest timestamp recorded on all
paths, the control plane adds a value chosen by the user or
controller based on engineering requirements (including additional
values for line transmission, variable delay upper limit margin, and
processing delay at CN before scheduling) to establish a common
compensation delay reference point tr for all paths from HN to CN. tr
is the timestamp at CN. The compensation reference delay values for
each path are calculated as described in section 4.2.1.
9. Configure the compensation reference delay values to HN or CN.
10. Runtime maintenance: periodically collect and calculate the full
path reference delay and update the reference delays to HN or CN.
For delay collection in each domain, the control plane provides the
relevant information required by the data plane and determines the
provision method. The control plane can provide this information in
the following ways:
Method 1: Globally allocate node identifiers in the control plane and
distribute this identifier to each node. The control plane
distributes the information required for delay collection to HN.
Guo, et al. Expires June 11, 2024 [Page 19]
Internet-Draft Compensation Reference for JRM December 2023
Method 2: The control plane configures per-flow operations at
intermediate nodes, and if SD exists, configures it at the SD's I-GW
and E-GW.
4.3. Round-Trip Compensation Reference Collection for AN
The round-trip collection method is suitable for situations where the
round-trip paths are the same. See [RFC9016] for a description of
BiDir.
Due to the lack of global clock synchronization, we cannot directly
obtain the end-to-end path delay. It would be very complicated to
directly obtain the delay of each path segment and then accumulate
the delay of each segment. Therefore, we draw on the idea of 4.2 and
use relative-delay method. By using the relative-delay method to
accurately obtain the approximate invariant part of the delay,
combined with the variable residence delay on the path and other
delays, PthRefD is obtained by comprehensive evaluation. At the same
time, the difference between the approximate invariant part of the
delay for the specified path and PthrefD is adjusted into the
compensation reference delay Cap_i. According to the formula of
compensation delay in [I-D.guo-detnet-jitter-reduction-mechanism]:
CompD_i = (PthRefD - FixD_i - (T1_i + T2_i + T3_i + T4_i)
Let Cap_i = PthRefD - FixD_i,
We need to design a method to obtain PthRefD and FixD_i in order to
obtain Cap_i before deploying DetNet flows. During the runtime, the
approximate invariant part of the delay FixD_i in the service
protection member path_i is periodically obtained, and the newly
obtained value is used to update the previous FixD_i. When the
DetNet-flow packets are actually transmitted, the new FixD_i and
PthRefD are used to calculate the new Cap_i.
Guo, et al. Expires June 11, 2024 [Page 20]
Internet-Draft Compensation Reference for JRM December 2023
4.3.1. Implementation Principle
HN CN
+----------+ Req_1 +----------+
| th_R_1|<---------------------------|tc_R_1 |
| | | |
| th_A_1| Ack_1 | |
| |--------------------------->|tc_A_1 |
| | Ack_2 | |
| th_A_2|--------------------------->|tc_A_2 |
| | ...... |...... |
| | | |
| | Ack_n | |
| th_A_n|--------------------------->|tc_A_n |
+----------+ +----------+
(a) Round-Trip Collection Message
|<-------t_Req_1------>|<-------------PthRefD----------->|
| | |
|tc_R_1 |th_R_1 |tc_A_1 |tc_A_2 |
|----------------------+------------+--------+-----------|->time
| | | | |
| | | | |
| Path_2|------------+------->|<---Tr_2-->|
| | | |
| Path_1|----------->|<--------Tr_1------>|
| | | |
| | | |
(b) Delay Model for Round-Trip Collection Method
Figure 9 Round-Trip Collection Message and Delay model
According to section 3.2, the paths between HN and CN can be
abstracted as multiple paths.
Figure 9(a) is a diagram for measuring and collecting message delay
information. CN sends a request type OAM message, Req_1, to HN
through Path_1 (Path_1 being any path within the service protection
group). Upon receiving the message, HN generates n response type OAM
messages, Ack_1 to Ack_n, and sends them through Path_1 to Path_n
back to CN. CN records the transmission timestamp, tc_R_1, when
sending the request message. During the transmission of Req_1, each
node (including the SD's I-GW and E-GW, or non-SD nodes) collects and
Guo, et al. Expires June 11, 2024 [Page 21]
Internet-Draft Compensation Reference for JRM December 2023
calculates the residence delay on the path of CN->HN at SD or non-SD
nodes, which is designated as ActD_Req (excluding the line delay
between SDs, or between non-SD nodes, or between SD and non-SD
nodes). During the transmission of Act_i (i = 1 to n), each node
records the residence delay on the path of HN->CN at SD or non-SD
nodes, denoted as ActD_Act_i (i = 1 to n) (excluding the line delay
between SD, non-SD nodes, or between SD and non-SD nodes).
HN records the reception timestamp th_R_1 for Req_1 and the
transmission timestamps th_A_1 to th_A_n for the n response-type OAM
messages Ack_1 to Ack_n. HN calculates n residence delays and uses
them as the initial values for ActD_Act_1 to ActD_Act_n:
ActD_Act_i = th_A_i - th_R_1 (i = 1 to n)
Along with carrying ActD_Act_i (i = 1 to n), Act_i (i = 1 to n) also
carries ActD_Req. CN records the reception timestamp tc_A_i (i = 1 to
n) for receiving Act_i. CN can obtain ActD_Req and ActD_Ack_i from
Act_i.
Figure 9(b) shows a comprehensive delay model for multiple stages,
using an example with two paths. In this figure, th_R_1 and th_A_1~2
are timestamps at HN, while tc_R_1 and tc_A_1~2 are timestamps at CN.
The tr represents the compensation delay reference point for all
paths from HN to CN, calculated as the latest timestamp of receiving
the response-type OAM in CN (tc_A_2 in the figure), with additional
values determined by the user or controller based on engineering
requirements. These additional values include extra link delay,
variable residence delay upper limit, and any processing delay at CN
before scheduling.
The specific processing steps are described as follows:
1. delay information collection. As shown in Figure 9(a), Path_1 to
Path_n are n paths with the same HN and CN, used for service
protection.
a) CN sends Req_j message to HN via any path j (0<j<n+1), records
the transmission time tc_R_j, and sets the accumulated residence
delay of the SD or non-SD nodes that Req_j passes through as
ActD_Req. In the request message, ActD_Req is individually carried in
a field (Note: ActD_Req does not include the link delay between SDs,
or between non-SD nodes, or between non-SD nodes, as there is no time
synchronization).
b) Upon receiving Req_j at HN, the reception time is recorded as
th_R_j. HN constructs n response messages Ack_1 to Ack_n and sends
Guo, et al. Expires June 11, 2024 [Page 22]
Internet-Draft Compensation Reference for JRM December 2023
them to CN. The accumulated residence delay of the nodes that Ack_i
(i=1 to n, same below) passes through is set as ActD_Ack_i. In each
response message, in addition to carrying ActD_Req in a field, there
is an additional field to carry ActD_Ack_i. The time for sending a
response message to path i on HN is set to th_A_i, and the residence
delay on HN (th_A_i - th_R_j) is used as the initial value of
ActD_Ack_i;
c) When CN receives Ack_i, the reception time is recorded as
tc_A_i.
2. Calculate the FixD_j for path j based on the information carried
by Ack_j and the time it arrives at CN. The time tc_A_j - tc_R_j
consists of: (1) the transmission delay of the request message Req_j
from CN to HN; (2) processing delay at HN; (3) transmission delay of
the response message Ack_j from HN to CN. Ack_j carries the
accumulated residence delay ActD_Req and ActD_Ack_j of the nodes that
the request and response messages pass through. Assuming the round-
trip link delay is approximately equal, then:
FixD_j= [(tc_A_j - tc_R_j) - (ActD_Req_j + ActD_Ack_j)]/2;
3. Calculate the FixD_i for each path based on the information
carried by the response messages Ack_i (0<i<n+1, i!=j) on each path,
the time it arrives at CN, and the calculated FixD_j. Since (tc_A_i -
tc_R_j) already contains the fixed transmission delay FixD_j of the
request message, the fixed transmission delay for the other paths is:
FixD_i = [(tc_A_i - tc_R_j) - (ActD_Req + ActD_Ack_i)] - FixD_j;
4. Before deploying DetNet flow, calculate PthRefD. Taking two paths
as an example as shown in Figure 9(b), the transmission delay t_Req_1
from CN to HN is common and can be ignored. The delay from HN to CN
on Path_1 is approximately from the time th_R_1 at HN to the time
tc_A_1 at CN; similarly, the delay from HN to CN on Path_2 is
approximately from the time th_R_1 at HN to the time tc_A_2 at CN.
Select the latest arrival time tc_A_2 (approximating the transmission
delay of the longest path) as the basis, and add the value tr_2,
which is selected by the user or controller based on engineering
requirements (including extra link delay, variable delay upper limit,
and any processing delay before CN scheduling), as the reference
point tr. Here, PthRefD is the time from th_R_1 at HN to the time tr
at CN. Without time synchronization, direct subtraction cannot obtain
the path delay. However, it can be indirectly obtained. Assuming that
the response message via Path_2 arrives the latest, then:
Guo, et al. Expires June 11, 2024 [Page 23]
Internet-Draft Compensation Reference for JRM December 2023
PthRefD = FixD_2 + ActD_Ack_2 + Tr_2;
5. Calculate the compensation reference delay value:
Cap_i = PthRefD - FixD_i;
6. Update Cap_i into CN or HN for forwarding compensation.
4.3.2. Data Plane
To collect the compensation reference delay for AN by round-trip
method, the data plane behaviors of each node are described as
follows:
1. CN's data plane receives the compensation reference delay
collection message and converts it into an OAM packet.
2. CN sends the request OAM packet for compensation reference delay
collection to HN, with the path specified by the control plane. CN
records the transmission time and forwards it to the control plane.
3. Each node collects the residence delay of the request message
passing through the path(if the node is SD's I-GW or E-GW, this is
done by the SD's I-GW and E-GW). With coordination from the control
plane, necessary identification information for identifying the path
is generated. Similar requirements are specified in [I-D.ietf-detnet-
pof]. When a CN node receives an OAM packet or an app-flow packet, it
needs to obtain path-associated information.
4. HN records the reception delay, constructs several response-type
OAM messages, which include the residence delay of the request OAM
message and the residence delay of the response OAM message
(initially the residence delay on the HN), and sends them to the CN
through different paths planned by the control plane.
5. Each node collects the residence delay of the response messages
passing through the path(if the node is SD's I-GW or E-GW, this is
done by the SD's I-GW and E-GW). With coordination from the control
plane, necessary identification information for identifying the path
is generated.
6. At each node implementing PEF on the HN->CN path, when identifying
OAM for collecting compensation reference delay, the node does not
eliminate duplicate copies and directly forwards the message.
7. At the CN node, the reception time of each response message is
recorded and sent to the control plane.
Guo, et al. Expires June 11, 2024 [Page 24]
Internet-Draft Compensation Reference for JRM December 2023
4.3.3. Control Plane
The control plane configures gateway identifiers, path information,
and related operations to the nodes through signaling negotiation or
controller configuration.
For the collection of compensation reference delay for AN, the
control plane needs to coordinate with the data plane to accomplish
the following process:
1. Plan network paths and identify the network as SN or AN.
2. Configure the necessary information for generating path
identifiers to each node. Similar to the requirements in [I-D.ietf-
detnet-pof], HN nodes and CN nodes need to obtain path-associated
information when receiving OAM messages or service data packets.
3. Configure path identifier information on CN (for response-type
OAM) and HN (for request-type OAM) for identifying the OAM messages
transmitted along different paths and forwarding service data
packets.
4. Set a timeout mechanism. Reference can be made to the historical
window mechanism on TSN in [IEEE 802.1 CB].
5. Define and configure operations for collecting residence delays at
nodes (including SD's I-GW and E-GW) along the path.
6. Conduct integrity checks in the control plane.
7. Allocate OAM message sequence numbers and send messages to CN to
send compensation reference delay collection OAM messages.
8. Upon receiving an OAM message, the CN control plane cooperates
with the data plane to complete the measurement and calculate the
reference delay. For each copy of the same OAM message received by
the CN control plane, the maximum residence delay and fixed
transmission delay within each node are calculated, based on which
the compensation reference delay for each path is determined as
described in section 4.3.1.
9. Configure the compensation reference delay value to HN or CN.
10. Runtime maintenance: periodically collect and calculate the full
path reference delay and update the reference delay to CN or HN.
Guo, et al. Expires June 11, 2024 [Page 25]
Internet-Draft Compensation Reference for JRM December 2023
For the delay collection in each domain, the control plane
coordinates to provide the relevant information required by the data
plane and determines the providing method. The control plane can
provide in the following ways:
Method one: The control plane globally allocates node identifiers and
distributes this identifier to each node (including SD's I-GW and E-
GW). The control plane sends down the required information for delay
collection to HN.
Method two: The control plane configures operations on a per-flow
basis at the intervening nodes (including SD's I-GW and E-GW).
5. Security Considerations
TBD
6. IANA Considerations
TBD
7. Acknowledgments
TBD
8. Contributors
The editor wishes to thank and acknowledge the following contributors
for contributing text to this document.
Guo, et al. Expires June 11, 2024 [Page 26]
Internet-Draft Compensation Reference for JRM December 2023
Shenchao Xu
New H3C Technologies Co., Ltd
310052
Email: xushenchao@h3c.com
Ning Pan
New H3C Technologies Co., Ltd
100094
Email: panning@h3c.com
Xusheng Chen
New H3C Technologies Co., Ltd
100094
Email: cxs@h3c.com
Wei Wang
New H3C Technologies Co., Ltd
100094
Email: david_wang@h3c.com
Xinmin Liu
New H3C Technologies Co., Ltd
100094
Email: liuxinmin@h3c.com
9. References
9.1. Informative References
[I-D.guo-detnet-jitter-reduction-mechanism]
Guo, D., Xu, S., "Jitter Reduction Mechanism for
DetNet",Work in Progress, Internet-Draft, draft-guo-detnet-
jitter-reduction-mechanism-01, 23 October 2023, <
https://datatracker.ietf.org/doc/draft-guo-detnet-jitter-
reduction-mechanism/>.
[I-D.ietf-detnet-scaling-requirements]
Liu, P., Li, Y., Eckert, T., Xiong, Q., and J. Ryoo,
"Requirements for Scaling Deterministic Networks",Work in
Progress, Internet-Draft, draft-ietf-detnet-scaling-
requirements-05, November 2023,
<https://datatracker.ietf.org/doc/draft-ietf-detnet-
scaling-requirements/>.
[I-D.ietf-detnet-mpls-over-ip-preof]
Guo, et al. Expires June 11, 2024 [Page 27]
Internet-Draft Compensation Reference for JRM December 2023
Varga, B., Farkas, J., and A. G. Malis, "Deterministic
Networking (DetNet): DetNet PREOF via MPLS over
UDP/IP",Work in Progress, Internet-Draft, draft-ietf-
detnet-mpls-over-ip-preof-07, 25 July 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-detnet-
mpls-over-ip-preof-07>.
[I-D.ietf-detnet-pof]
Varga, B., Farkas, J., Kehrer, S., Heer, T., "Deterministic
Networking (DetNet): Packet Ordering Function", Work in
Progress, Internet-Draft, draft-ietf-detnet-pof-06, 8
November 2023, <https://datatracker.ietf.org/doc/draft-
ietf-detnet-pof/>.
[RFC9016]
Varga, B., Farkas, J., Cummings, R., Jiang, Y., Fedyk, D.,
"Flow and Service Information Model for Deterministic
Networking (DetNet)", RFC 8964, DOI 10.17487/RFC9016, March
2021, <https://www.rfc-editor.org/info/rfc9016>.
[IEEE802.1CB]
IEEE,"IEEE Standard for Local and metropolitan area
networks-Frame Replication and Elimination for
Reliability", IEEE Std 802.1CB-2017 (2017), 1--102.
<https://doi.org/10.1109/IEEESTD.2017.8091139>.
Guo, et al. Expires June 11, 2024 [Page 28]
Internet-Draft Compensation Reference for JRM December 2023
Authors' Addresses
Daorong Guo
New H3C Technologies Co., Ltd
Beijing
100094
China
Email: guodaorong@h3c.com
XUEJUN YOU
New H3C Technologies Co., Ltd
Beijing
100094
China
Email: yoe@h3c.com
Rubing Liu
New H3C Technologies Co., Ltd
Hangzhou
310052
China
Email: liurubing@h3c.com
Shiyin Zhu
New H3C Technologies Co., Ltd
Beijing
100094
China
Email: zhushiyin@h3c.com
Guo, et al. Expires June 11, 2024 [Page 29]
