Internet DRAFT - draft-lyy-detnet-ref-delay-measurement
draft-lyy-detnet-ref-delay-measurement
Network Working Group H. Yang
Internet-Draft K. Yao
Intended status: Standards Track China Mobile
Expires: 1 January 2023 J-Y. Le Boudec
EPFL
30 June 2022
One-way Delay Measurement Based on Reference Delay
draft-lyy-detnet-ref-delay-measurement-01
Abstract
The end-to-end network one-way delay is an important performance
metric in the 5G network. For realizing the accurate one-way delay
measurement, existing methods requires the end-to-end deployment of
accurate clock synchronization mechanism, such as PTP or GPS, which
results in relatively high deployment cost. Another method can
derive the one-way delay from the round-trip delay. In this case,
since the delay of the downlink and uplink of the 5G network may be
asymmetric, the measurement accuracy is relatively low. Hence, this
document introduces a method to measure the end-to-end network one-
way delay based on a reference delay guaranteed by deterministic
networking without clock synchronization.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 1 January 2023.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
Yang, et al. Expires 1 January 2023 [Page 1]
Internet-Draft Network Working Group June 2022
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions Used in This Document . . . . . . . . . . . . . . 4
2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4
2.2. Requirements Language . . . . . . . . . . . . . . . . . . 4
3. Theoretical analysis of One-way Delay Measurement Based on
Reference Delay . . . . . . . . . . . . . . . . . . . . . 4
4. One-way Delay Measurement Procedure . . . . . . . . . . . . . 7
5. Packet and Measurement Header Format . . . . . . . . . . . . 9
6. Acquisition of Reference Delay . . . . . . . . . . . . . . . 10
7. Security Considerations . . . . . . . . . . . . . . . . . . . 10
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10
9.1. Normative References . . . . . . . . . . . . . . . . . . 11
9.2. Informative References . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11
1. Introduction
With the gradual promotion of new-generation network technologies
(such as 5G networks) and their application in various industries,
SLA guarantees for network quality become more and more important.
For example, different 5G services have different requirements for
network performance indicators such as delay, jitter, packet loss,
and bandwidth. Among them, the 5G network delay is defined as end-
to-end one-way delay of the network. Real-time and accurate
measurement of the end-to-end one-way delay is very important for the
SLA guarantee of network services, and has become an urgent and
important requirement.
Yang, et al. Expires 1 January 2023 [Page 2]
Internet-Draft Network Working Group June 2022
As shown in figure 1, 5G network HD video surveillance service is a
common scenario having requirement of end-to-end one-way delay
measurement. In this case, one end of the network is a high-
definition surveillance camera in the wireless access side, and the
other end of the network is a video server. The end-to-end one-way
delay from the surveillance camera to the video server is the sum of
T1, T2, T3 and T4, which is composed of delay in wireless access
network, optical transmission network, 5G core network, and IP data
network.
+--------+ +-------+ +-------+ +-------+
+------+ |Wireless| |Optical| |5G Core| | IP | +------+
|Camera+<->+ Access +<->+ Trans +<->+Network+<->+ Data +<->+Server|
+------+ |Network | |Network| | | |Network| +------+
+--------+ +-------+ +-------+ +-------+
|<---- T1 ---->|<--- T2 -->|<--- T3 -->|<--- T4 ---->|
Figure 1: A Scenario for End-to-end One-way Delay
The existing one-way delay measurement solutions are divided into two
types. One type of mechanism to calculate one-way delay is based on
the measurement of round-trip delay. However, for example, because
upstream traffic and downstream traffic do not share the same path in
5G network, the accuracy of the end-to-end one-way delay calculated
from the round-trip delay is low. Another type of mechanism is in-
band OAM with accurate network time synchronization mechanism , such
as NTP[RFC5905] or PTP[IEEE.1588.2008].
The one-way delay measurement solution based on precise network time
synchronization requires the deployment of an end-to-end time
synchronization mechanism. The current time synchronization accuracy
based on the NTP protocol can only reach millisecond level, which
cannot fully meet the measurement accuracy requirements. The time
synchronization accuracy based on the GPS module or the PTP protocol
can meet the requirements. However, because many data centers are
actually located underground or in rooms without GPS signals, so GPS
clock information cannot be continuously obtained for time
synchronization. For time synchronization solutions based on the PTP
protocol, each device in the wireless access network, 5G transport
network, and 5G core network must support the PTP protocol, which is
unrealistic at the moment. So the one-way delay measurement solution
based on precise end-to-end time synchronization is expensive and
difficult to be deployed.
This document introduces a one-way delay measurement mechanism for
Deterministic Networking (DetNet) [RFC8655]. The one-way delay
measurement is based on a stable one-way delay of a reference DetNet
Yang, et al. Expires 1 January 2023 [Page 3]
Internet-Draft Network Working Group June 2022
packet, named as reference delay, which is known in advance and has
extremely low jitter. We can use the reference delay provided by the
reference DetNet packet to derive the one-way delay of other common
service packets.
2. Conventions Used in This Document
2.1. Terminology
NTP Network Time Protocol
PTP Precision Time Protocol
SLA Service Level Agreement
2.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14[RFC2119][RFC8174] when, and only when, they appear in all
capitals, as shown here.
3. Theoretical analysis of One-way Delay Measurement Based on Reference
Delay
The end-to-end one-way delay of a packet with bounded delay that's
sent through a deterministic network path can be used as a reference
delay, which is known in advance and has extremely low jitter. This
section will describe the end-to-end one-way delay measurement method
based on reference delay in details . Assume that the end-to-end one-
way delay of a target packet is being measured, as shown in figure 2,
the target packet is transimitted through a normal network path while
the reference packet is sent through a deterministic network path.
At the meantime, we assume that there is a global clock which could
offer very precisive timing capabilities, and we denote its current
time to be true time, t. That is to say, for local clocks at sender
and receiver, their current time are Cs(t) and Cr(t) respectively.
The reference packet is sent at first from the sender with its local
timestamp Cs(Ts1) marked inside, and at true time Tr1, the reference
packet arrives at the receiver, and the receiver shows Cr(Tr1).
Similarly, the departure and arrival timestamps of the target packet
are Cs(Ts2) and Cr(Tr2). Since clocks at sender and receiver are not
time synchronized, target delay can not be directly measured by
making subtraction.
Yang, et al. Expires 1 January 2023 [Page 4]
Internet-Draft Network Working Group June 2022
Target
Packet +------+ +------+
+-----------> |Normal| +---> |Normal| +------------+
| |Switch| |Switch| |
| +------+ +---+--+ |
| | |
| Reference v |
+---+--+ Packet +------+ +---+--+ +----v---+
|Sender| +------> |Detnet| +---> |Detnet| +-----> |Receiver|
+------+ |Switch| |Switch| +--------+
+------+ +------+
Reference +-------+ DTrue_ref +-------+
Packet: |Cs(Ts1)| +--------------------------> |Cr(Tr1)|
+-------+ +-------+
Target +-------+ DTrue_target +-------+
Packet: |Cs(Ts2)| +--------------------------> |Cr(Tr2)|
+-------+ +-------+
Figure 2: Topology of One-way Delay Measurement Based on
Reference Delay
However, the boundedness of the reference delay can be leveraged for
measurement, as regulated in
[I.D.draft-ietf-detnet-bounded-latency-10]. The boundedness of the
reference delay can be formulated by equation 1.
Equation 1: L - J <= DTrue_ref <= L
In equation 1, L is the maximum value of the reference delay and J is
the peak-to-peak value of the reference delay. L and J are usually
measured in tens of microsecond level precision. DTrue_ref refers to
the true reference delay, and it is the difference between Tr1 and
Ts1, which can not be directly measured. DTrue_target denotes the
true target delay. They follow equation 2 and 3 respectively.
Equation 2: DTrue_ref = Tr1 - Ts1
Equation 3: DTrue_target = Tr2 - Ts2
Now we can get a relationship between the reference delay and the
target delay by equation 4.
Equation 4: DTrue_target = (Tr2 - Tr1) + DTrue_ref - (Ts2 - Ts1)
Yang, et al. Expires 1 January 2023 [Page 5]
Internet-Draft Network Working Group June 2022
Here we follow the clock model proposed by [ThomasTime] to formulate
the time variation of clocks at sender and receiver. The clock model
states that for a TSN-grade clock, its local time Ci(t) always
follows equation 5.
Equation 5: (Ci(T2) - Ci(T1) - eta) * (1/rho) <=T2 - T1 <= (Ci(T2) -
Ci(T1))*rho + eta (T2 >= T1)
In equation 5, rho refers to the time stability bound, i.e. 1.0001
for TSN-grade clock, and eta is the timing jitter bound, i.e. 2ns for
TSN-grade clock. The model can be adopted for analyzing behaviors of
clocks at the sender and receiver, because they are the both ends of
a deterministic network path and their clocks are TSN-grade. In this
way, inequality 6 for sender clock Cs(t) and inequality 7 for
receiver clock Cr(t) are formulated below.
Equation 6: (Cs(Ts2) - Cs(Ts1) - eta) * (1/rho) <=Ts2 - Ts1 <=
(Cs(Ts2) - Cs(Ts1))*rho + eta (Ts2 >= Ts1)
Equation 7: (Cr(Tr2) - Cr(Tr1) - eta) * (1/rho) <=Tr2 - Tr1 <=
(Cr(Tr2) - Cr(Tr1))*rho + eta (Tr2 >= Tr1)
Now, equation 4 can be extended with known values to express its
upper and lower bound. Upper bound is shown in equalition 8 and
lower bound is shown in equation 9.
Equation 8: DTrue_target <= Cr(Tr2) - Cr(Tr1) - Cs(Ts2) + Cs(Ts1) + L
+ eta * (1+(1/rho)) + (Cr(Tr2)-Cr(Tr1)+Cs(Ts2)-Cs(Ts1))(rho - 1)
Equation 9: DTrue_target >= Cr(Tr2) - Cr(Tr1) - Cs(Ts2) + Cs(Ts1) + L
- J - eta * (1+(1/rho)) - (Cr(Tr2)-Cr(Tr1)+Cs(Ts2)-Cs(Ts1))(rho - 1)
Accordingly, a point estimate of DTrue_target is expressed by
equation 10, and its corresponding inaccuracy is shown by equation
11.
Equation 10: DEst_target = Cr(Tr2) - Cr(Tr1) - Cs(Ts2) + Cs(Ts1) + L
- J/2
Equation 11: delta DEst_target = J/2 + eta * (1+(1/rho)) + (Cr(Tr2)-
Cr(Tr1)+Cs(Ts2)-Cs(Ts1))(rho - 1)
The derivation above can be used as a theoretical proof for the one-
way delay measurement approach based on the characteristics of
reference delay within deterministic network.
Yang, et al. Expires 1 January 2023 [Page 6]
Internet-Draft Network Working Group June 2022
4. One-way Delay Measurement Procedure
The measurement steps are shown in figure 3, which describe the
measurement steps at the sender side and receiver side respectively.
For the sender side, a reference packet is sent. In the first step,
the sender gets ready to send a reference packet; in the second step,
the sender marks an egress timestamp Cs(Ts1) for the reference
packet; in the third step, the sender encapsulates the egress
timestamp of the reference packet in the measurement header of the
reference packet; in the fourth step, the sender sends the reference
packet. For the target packet, the sender side procedures are the
same,we omit it for simplicity. The sending time of the target
packet is according to the traffic model of real applications.
Reference packets are sent for many times at first, in order to get
accurate bounds of reference delay, until which the target packet can
not be sent for measurement.
For the reference packet, the processing steps at the receiver are
shown in figure 3. In the first step, the reference packet arrives
at the receiver, and the receiver receives the reference packet; in
the second step, the receiver timestamps the reference packet at the
entrance, which is denoted as Cr(Tr1); in the third step, the
receiver decapsulates the measurement header of the reference packet
to obtain the sender side timestamp Cs(Ts1); in the fourth step, the
receiver records the timestamp information of Cs(Ts1) and Cr(Tr1); in
the fifth step, the receiver uses the source/destination pair
obtained by decapsulation in the third step as the search key,
queries the reference delay table and records the reference delay
search result, upper bound L and peak-to-peak value J.
Yang, et al. Expires 1 January 2023 [Page 7]
Internet-Draft Network Working Group June 2022
Sender Side Procedures for both Reference and Target Packet:
+-------+ +------------+ +-------------+ +-------+
|Sender | |Sender Side | |Sender Side | |Sending|
|Ready +-->+Timestamping+-->+Encapsulation+-->+ Packet|
| | | | | | | |
+-------+ +------------+ +-------------+ +-------+
Receiver Side Procedures for Reference Packet:
+---------+ +-------------+ +-------------+ +---------+ +---------+
|Reference| |Receiver Side| |Receiver Side| |Timestamp| |Query for|
|Packet +->+Timestamping +->+Decapsulation+->+Recorded +->+Reference|
|Arrival | | | | | | | |Delay, L,|
| | | | | | | | | and J |
+---------+ +-------------+ +-------------+ +---------+ +---------+
Receiver Side Procedures for Target Packet:
+-------+ +-------------+ +-------------+ +---------+ +-----------+
| Target| |Receiver Side| |Receiver Side| |Timestamp| | One-way |
| Packet+->+Timestamping +->+Decapsulation+->+Recorded +->+ Delay |
|Arrival| | | | | | | |Calculation|
+-------+ +-------------+ +-------------+ +---------+ +-----------+
Figure 3: Measurement steps for Sender and Receiver Respectively
For the target packet, the processing steps at the receiver are also
shown in figure 3. In the first step, the target packet arrives at
the receiver, and the receiver receives the target packet; in the
second step, the receiver timestamps the target packet at the
entrance, which is denoted as Cr(Tr2); in the third step, the
receiver decapsulates the measurement header of the target packet to
obtain the sender side timestamp Cs(Ts2); in the fourth step, the
receiver records the timestamp information of Cs(Ts2) and Cr(Tr2); in
the fifth step, the receiver calculates the target one-way delay,
which we want to measure, according to the recorded timestamp
information Cs(Ts1), Cs(Ts2), Cr(Tr1), Cr(Tr2) and reference delay
with its upper bound and peak-to-peak jitter. The upper and lower
bound of target one-way delay can be caculated by equation 8 and 9,
and at the meantime, a rough exstimation can be made by using
equation 10.
Yang, et al. Expires 1 January 2023 [Page 8]
Internet-Draft Network Working Group June 2022
5. Packet and Measurement Header Format
The sender encapsulates the timestamp information and sender-receiver
pair information in the measurement header of the sent packet, as
shown in figure 4. The position of measurement header is in the
option field of the TCP protocol header. The delay measurement
option format is defined in figure 5. The Length value is 8 octets,
which is in accordance with TCP option. The sender ID is one octet,
and the receiver ID is also one octet. The sender side timestamp is
4 octets, which can store accurate timestamp information.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Ethernet header (14 octets) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IP header (20 octets) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| TCP header (20 octets) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TCP Delay Measurement Option (8 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Format of Reference or Target Packet
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Kind=TBA | Length | Sender ID | Receiver ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sender Side Timestamp (4 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: TCP Delay Measurement Option Format
Yang, et al. Expires 1 January 2023 [Page 9]
Internet-Draft Network Working Group June 2022
6. Acquisition of Reference Delay
The end-to-end one-way delay includes three parts, namely the
transmission delay, the internal processing delay of the network
devices, and the internal queueing delay of the network devices.
Among them, fixed parts of the delay include transmission delay and
internal processing delay. The transmission delay is related to
transmission distance and transmission media. For example, in
optical fiber, it is about 5ns per meter. With transmission path and
media determined, it is basically a fixed value. The internal
processing delay of a network device includes processing delay of the
device's internal pipeline or processor and serial-to-parallel
conversion delay of the interface, which is related to in/out port
rate of the device, message length and forwarding behavior. The
magnitude of the internal processing delay is at microsecond level,
and it is basically a fixed value related to the chip design
specifications of a particular network device. Variable part of the
delay is the internal queueing delay. The queueing delay of the
device internal buffer is related to the queue depth, queue
scheduling algorithm, message priority and message length. For each
device along the end-to-end path, the queueing delay can reach
microsecond or even millisecond level, depending on values of the
above parameters and network congestion state.
With the continuous development of networking technologies and
application requirements, a series of new network technologies have
emerged which can guarantee bounded end-to-end delay and ultra small
jitter. For example, deterministic network[RFC8655], by leveraging
novel scheduling algorithms and packet priority settings, can
stabilize queuing delay of network device on the end-to-end path. As
a result, the end-to-end one-way delay is extremely low and bounded.
So packets transmitted by a deterministic network with delay
guarantee can be used as reference packets, and their end-to-end one-
way delay can be used as reference delays. The acquisition method of
reference delay is not limited to the above method based on
deterministic network technology.
7. Security Considerations
TBD.
8. IANA Considerations
This document requests IANA to assign a Kind Number in TCP Option to
indicate TCP Delay Measurement option.
9. References
Yang, et al. Expires 1 January 2023 [Page 10]
Internet-Draft Network Working Group June 2022
9.1. Normative References
[IEEE.1588.2008]
IEEE, "IEEE Standard for a Precision Clock Synchronization
Protocol for Networked Measurement and Control Systems",
July 2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
<https://www.rfc-editor.org/info/rfc5905>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>.
9.2. Informative References
[I.D.draft-ietf-detnet-bounded-latency-10]
N. Finn, J-Y. Le Boudec, E. Mohammadpour, J. Zhang, B.
Varga, "DetNet Bounded Latency",
<https://datatracker.ietf.org/doc/draft-ietf-detnet-
bounded-latency/>.
[ThomasTime]
L. Thomas and J.-Y. Le Boudec, "On Time Synchronization
Issues in Time-Sensitive Networks with Regulators and
Nonideal Clocks",
<https://dl.acm.org/doi/10.1145/3393691.3394206>.
Authors' Addresses
Hongwei Yang
China Mobile
Beijing
100053
China
Email: yanghongwei@chinamobile.com
Yang, et al. Expires 1 January 2023 [Page 11]
Internet-Draft Network Working Group June 2022
Kehan Yao
China Mobile
Beijing
100053
China
Email: yaokehan@chinamobile.com
Jean-Yves Le Boudec
EPFL
IC Station 14
CH-1015 Lausanne EPFL
Switzerland
Email: jean-yves.leboudec@epfl.ch
Yang, et al. Expires 1 January 2023 [Page 12]
