Internet DRAFT - draft-cfb-ippm-spinbit-measurements
draft-cfb-ippm-spinbit-measurements
IPPM M. Cociglio
Internet-Draft Telecom Italia
Intended status: Experimental G. Fioccola
Expires: January 4, 2021 Huawei Technologies
M. Nilo
F. Bulgarella
Telecom Italia
R. Sisto
Politecnico di Torino
July 3, 2020
Client-Server Explicit Performance Measurements
draft-cfb-ippm-spinbit-measurements-02
Abstract
This document introduces an additional single bit signal to enhance
the spin bit [I-D.trammell-ippm-spin] performance in presence of
network impairments and application limited flow. In addition, it
defines two new explicit per-flow transport-layer signals for hybrid
measurement of connection loss rate. The former is a spin-bit
dependent signal and uses a single bit. The latter is a standalone
solution based on a two bits loss signal and on alternate marking RFC
8321 [RFC8321].
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 4, 2021.
Cociglio, et al. Expires January 4, 2021 [Page 1]
Internet-Draft July 2020
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Spin bit and Delay bit mechanism . . . . . . . . . . . . . . 4
2.1. Delay Sample generation . . . . . . . . . . . . . . . . . 5
2.1.1. The recovery process . . . . . . . . . . . . . . . . 6
2.2. Delay Sample reflection . . . . . . . . . . . . . . . . . 6
3. Using the Spin bit and Delay bit for Hybrid RTT Measurement . 7
3.1. End-to-end RTT measurement . . . . . . . . . . . . . . . 7
3.2. Half-RTT measurement . . . . . . . . . . . . . . . . . . 8
3.3. Intra-domain RTT measurement . . . . . . . . . . . . . . 9
4. Observer's algorithm and Waiting Interval . . . . . . . . . . 10
5. Adding a Loss signal for Packet loss measurement . . . . . . 11
5.1. Round Trip Packet Loss measurement . . . . . . . . . . . 13
6. Packet Loss using one bit loss signal . . . . . . . . . . . . 14
6.1. Observer's logic for one bit loss signal . . . . . . . . 16
7. Two Bits packet loss measurement using alternate marking . . 16
7.1. Setting the square bit (Q) on outgoing packets . . . . . 16
7.2. Setting the reflection square bit (R) on outgoing packets 17
7.2.1. Determining the completion of an incoming marking
period . . . . . . . . . . . . . . . . . . . . . . . 18
7.3. Observer's logic and passive loss measurements . . . . . 18
7.3.1. Upstream one-way loss . . . . . . . . . . . . . . . . 19
7.3.2. Three-quarters connection loss . . . . . . . . . . . 19
7.3.3. Full one-way loss in the opposite direction . . . . . 20
7.3.4. Half round-trip loss . . . . . . . . . . . . . . . . 21
7.3.5. Downstream one-way loss . . . . . . . . . . . . . . . 21
7.4. Enhancement of reflection period size computation . . . . 22
7.5. Improvement of the resilience to out of sequence . . . . 22
8. Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.1. QUIC . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.2. TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
9. Security Considerations . . . . . . . . . . . . . . . . . . . 23
Cociglio, et al. Expires January 4, 2021 [Page 2]
Internet-Draft July 2020
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 24
12.1. Normative References . . . . . . . . . . . . . . . . . . 24
12.2. Informative References . . . . . . . . . . . . . . . . . 24
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25
1. Introduction
Both [I-D.trammell-tsvwg-spin] and [I-D.trammell-ippm-spin] define an
explicit per-flow transport-layer signal for hybrid measurement of
end-to-end RTT. This signal consists of three bits: a spin bit,
which oscillates once per end-to-end RTT, and a two-bit Valid Edge
Counter (VEC), which compensates for loss and reordering of the spin
bit to increase fidelity of the signal in less than ideal network
conditions.
In this document it is introduced the delay bit, that is a single bit
signal that can be used together with the spin bit by passive
observers to measure the RTT of a network flow, avoiding the spin bit
ambiguities that arise as soon as network conditions deteriorate.
Unlike the spin bit, which is actually set in every packet
transmitted on the network, the delay bit is set only once per round
trip.
Regarding loss rate measurement, two new algorithms are introduced.
The first algorithm enables end-to-end round trip loss rate
measurement using a single bit signal called loss bit. This signal
is used to mark a train of packets (a portion of traffic) which
bounces back an forth two times between endpoints, realizing a two
round trip reflection. A passive on-path observer, placed on
whatever direction, can trivially count and compare the number of
marked packets seen during the two reflections estimating
statistically the loss rate experienced by the connection. The
second algorithm uses a double square signal and RFC 8321 [RFC8321]
to mark the whole traffic exchanged between endpoints. This solution
enables different types of measurements providing a complete picture
of connection loss events.
This document defines hybrid measurement RFC 7799 [RFC7799] path
signals to be embedded into a transport layer protocol, explicitly
intended for exposing end-to-end RTT and loss rate information to
measurement devices on path.
The document introduces mechanisms applicable to any transport-layer
protocol, then explains how to bind the signals to a variety of IETF
transport protocols, and in particular to QUIC and TCP.
Cociglio, et al. Expires January 4, 2021 [Page 3]
Internet-Draft July 2020
The application of the spin bit to QUIC is described in
[I-D.ietf-quic-spin-exp] which adds the spin bit to QUIC for
experimentation purposes.
Note that spin bit, delay bit and loss bits explained in this
document are inspired by RFC 8321 [RFC8321]. This is also mentioned
in [I-D.trammell-quic-spin].
Note that additional details about the Performance Measurements for
QUIC are also described in the paper [ANRW19-PM-QUIC].
2. Spin bit and Delay bit mechanism
The main idea is to have a single packet, with a second marked bit
(the delay bit), that bounces between client and server during the
entire connection life. This single packet is called Delay Sample.
A simple observer placed in an intermediate point, tracking the delay
sample and the relative timestamp in every spin bit period, can
measure the end-to-end round trip delay of the connection. In the
same way as seen with the spin bit, it is possible to carry out other
types of measurements using this additional bit. The next paragraphs
give an overview of the observer capabilities.
In order to describe the delay sample working mechanism in detail, we
have to distinguish two different phases which take part in the delay
bit lifetime: initialization and reflection. The initialization is
the generation of the delay sample, while the reflection realizes the
bounce behavior of this single packet between the two endpoints.
The next figure describes the Delay bit mechanism: the first bit is
the spin bit and the second one is the delay bit.
+--------+ -- -- -- -- -- +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ -- -- -- -- -- +--------+
(a) No traffic at beginning.
+--------+ 00 00 01 -- -- +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ -- -- -- -- -- +--------+
Cociglio, et al. Expires January 4, 2021 [Page 4]
Internet-Draft July 2020
(b) The Client starts sending data and
sets the first packet as Delay Sample.
+--------+ 00 00 00 00 00 +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ -- -- 01 00 00 +--------+
(c) The Server starts sending data
and reflects the Delay Sample.
+--------+ 10 10 11 00 00 +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ 00 00 00 00 00 +--------+
(d) The Client inverts the spin bit and
reflects the Delay Sample.
+--------+ 10 10 10 10 10 +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ 00 00 11 10 10 +--------+
(e) The Server reflects the Delay Sample.
+--------+ 00 00 01 10 10 +--------+
| | -----------> | |
| Client | | Server |
| | <----------- | |
+--------+ 10 10 10 10 10 +--------+
(f) The client reverts the spin
bit and reflects the Delay Sample.
Figure 1: Spin bit and Delay bit
2.1. Delay Sample generation
During this first phase, endpoints play different roles. First of
all a single delay sample must be bouncing per round trip period (and
so per spin bit period). According to that statement and in order to
simplify the general algorithm, the delay sample generation is in
charge of just one of the two endpoints:
Cociglio, et al. Expires January 4, 2021 [Page 5]
Internet-Draft July 2020
o the client, when connection starts and spin bit is set to 0,
initializes the delay bit of the first packet to 1, so it becomes
the delay sample for that marking period. Only this packet is
marked with the delay bit set to 1 for this round trip period; the
other ones will carry only the spin bit;
o the server never initializes the delay bit to 1; its only task is
to reflect the incoming delay bit into the next outgoing packet
only if certain conditions occur.
Theoretically, in absence of network impairments, the delay sample
should bounce between client and server continuously, for the entire
duration of the connection. Actually, that is highly unlikely mainly
for two different reasons:
1) the packet carrying the delay bit might be lost during its journey
on the network which is unreliable by definition;
2) one of the two endpoints could stop or delay sending data because
the application is limiting the amount of traffic transmitted;
To deal with these problems, the algorithm provides a procedure to
regenerate the delay sample and to inform a possible observer that a
problem has occurred, and then the measurement has to be restarted.
2.1.1. The recovery process
In order to relieve the server from tasks that go beyond the mere
reflection of the sample, even in this case the recovery process
belongs to the client. A fundamental assumption is that a delay
sample is strictly related to its spin bit period. Considering this
rule, the client verifies that every spin bit period ends with its
delay sample. If that does not happen and a marking period
terminates without a delay sample, the client waits a further empty
period; then, in the following period, it reinitializes the mechanism
by setting the delay bit of the first outgoing packet to 1, making it
the new delay sample. The empty period is needed to inform the
intermediate points that there was an issue and a new delay
measurement session is starting.
2.2. Delay Sample reflection
The reflection is the process that enables the bouncing of the delay
sample between client and server. The behavior of the two endpoints
is slightly different. With the exception of the client that, as
previously exposed, generates a new delay sample, by default the
delay bit is set to 0.
Cociglio, et al. Expires January 4, 2021 [Page 6]
Internet-Draft July 2020
Server side reflection: when a packet with the delay bit set to 1
arrives, the server marks the first packet in the opposite direction
as the delay sample, if it has the same spin bit value. While if it
has the opposite spin bit value this sample is considered lost.
Client side reflection: when a packet with delay bit set to 1
arrives, the client marks the first packet in the opposite direction
as the delay sample, if it has the opposite spin bit value. While if
it has the same spin bit value this sample is considered lost.
In both cases, if the outgoing marked packet is transmitted with a
delay greater than a predetermined threshold after the reception of
the incoming delay sample (1ms by default), reflection is aborted and
this sample is considered lost.
Note that reflection takes place for the packet that is carrying the
delay bit regardless of its position within the period. For this
reason it is necessary to introduce that condition of validation in
order to identify and discard those samples that, due to reordering,
might move to a contiguous period. Furthermore, by introducing a
threshold for the retransmission delay of the sample, it is possible
to eliminate all those measurements which, due to lack of traffic on
the endpoints, would be overestimated and not true. Thus, the
maximum estimation error, without considering any other delays due to
flow control, would amount to twice the threshold (e.g. 2ms) per
measurement, in the worst case.
3. Using the Spin bit and Delay bit for Hybrid RTT Measurement
Unlike what happens with the spin bit for which it is necessary to
validate or at least heuristically evaluate the goodness of an edge,
the delay sample can be used by an intermediate observer as a simple
demarcator between a period and the following one eliminating the
ambiguities on the calculation of the RTT found with the analysis of
the spin-bit only. The measurement types, that can be done from the
observation of the delay sample, are exactly the same achievable with
the spin bit only.
3.1. End-to-end RTT measurement
The delay sample generation process ensures that only one packet
marked with the delay bit set to 1 runs back and forth on the wire
between two endpoints per round trip time. Therefore, in order to
determine the end-to-end RTT measurement of a QUIC flow, an on-path
passive observer can simply compute the time difference between two
delay samples observed in a single direction. Note that a
measurement, to be valid, must take into account the difference in
time between the timestamps of two consecutive delay samples
Cociglio, et al. Expires January 4, 2021 [Page 7]
Internet-Draft July 2020
belonging to adjacent spin-bit periods. For this reason, an
observer, in addition to intercepting and analyzing the packets
containing the delay bit set to 1, must maintain awareness of each
spin period in such a way as to be able to assign each delay sample
to its period and, at the same time, identifying those periods that
do not contain it.
=======================|======================>
= ********** -----Obs----> ********** =
= * Client * * Server * =
= ********** <------------ ********** =
<==============================================
(a) client-server RTT
==============================================>
= ********** ------------> ********** =
= * Client * * Server * =
= ********** <----Obs----- ********** =
<======================|=======================
(b) server-client RTT
Figure 2: Round-trip time (both direction)
3.2. Half-RTT measurement
An on-path passive observer that is sniffing traffic in both
directions -- from client to server and from server to client -- can
also use the delay sample to measure "upstream" and "downstream" RTT
components. Also known as the half-RTT measurement, it represents
the components of the end-to-end RTT concerning the paths between the
client and the observer (upstream), and the observer and the server
(downstream). It does this by measuring the delay between a delay
sample observed in the downstream direction and the one observed in
the upstream direction, and vice versa. Also in this case, it should
verify that the two delay samples belong to two adjacent periods, for
the upstream component, or to the same period for the downstream
component.
Cociglio, et al. Expires January 4, 2021 [Page 8]
Internet-Draft July 2020
=======================>
= ********** ------|-----> **********
= * Client * Obs * Server *
= ********** <-----|------ **********
<=======================
(a) client-observer half-RTT
=======================>
********** ------|-----> ********** =
* Client * Obs * Server * =
********** <-----|------ ********** =
<=======================
(b) observer-server half-RTT
Figure 3: Half Round-trip time (both direction)
3.3. Intra-domain RTT measurement
Taking advantage of the half-RTT measurements it is also possible to
calculate the intra-domain RTT which is the portion of the entire RTT
used by a QUIC flow to traverse the network of a provider (or part of
it). To achieve this result two observers, able to watch traffic in
both directions, must be employed simultaneously at ingress and
egress of the network to be measured. At this point, to determine
the delay between the two observers, it is enough to subtract the two
computed upstream (or downstream) RTT components.
Cociglio, et al. Expires January 4, 2021 [Page 9]
Internet-Draft July 2020
=========================================>
= =====================>
= = ********** ---|--> ---|--> **********
= = * Client * Obs Obs * Server *
= = ********** <--|--- <--|--- **********
= <=====================
<=========================================
(a) client-observer RTT components (half-RTTs)
==================>
********** ---|--> ---|--> **********
* Client * Obs Obs * Server *
********** <--|--- <--|--- **********
<==================
(b) the intra-domain RTT resulting from the
subtraction of the above RTT components
Figure 4: Intra-domain Round-trip time (client-observer: upstream)
The spin bit is an alternate marking generated signal and the only
difference than RFC 8321 [RFC8321] is the size of the alternation
that will change with the flight size each RTT. So it can be useful
to segment the RTT and deduce the contribution to the RTT of the
portion of the network between two on-path observers and it can be
easily performed by calculating the delay between two or more
measurement points on a single direction by applying RFC 8321
[RFC8321].
4. Observer's algorithm and Waiting Interval
Given below is a formal summary of the functioning of the observer
every time a delay sample is detected. A packet containing the delay
bit set to 1:
o if it has the same spin bit value of the current period and no
delay sample was detected in the previous period, then it can be
used as a left edge (i.e. to start measuring an RTT sample), but
not as a right edge (i.e. to complete and RTT measurement since
the last edge). If the observation point is symmetric (i.e. it
can see both upstream and downstream packets in the flow) and in
the current period a delay sample was detected in the opposite
direction (i.e. in the upstream direction), the packet can also be
used to compute the downstream RTT component.
Cociglio, et al. Expires January 4, 2021 [Page 10]
Internet-Draft July 2020
o if it has the same spin bit value of the current period and a
delay sample was detected in the previous period, then it can be
used at the same time as a left or right edge, and to compute RTT
component in both directions.
Like stated previously, every time an empty period is detected, the
observer must restart the measurement process and consider the next
delay sample that will come as the beginning of a new measure, then
as a left edge. As a result, being able to assign the delay sample
to the corresponding spin period becomes a crucial factor for the
proper functioning of the entire algorithm.
Considering that the division into periods is realized by exploiting
the spin bit square wave, it is easy to understand that the presence
of spurious spin edges -- caused by packet reordering -- would
inevitably lead the observer to overestimate the amount of periods
actually present in the transmission. This results in a greater
number of empty periods detected and the consequent decrease of the
actual RTT samples achievable. Therefore, in order to maximize the
performance of the whole algorithm, the observer must implement a
mechanism to filter out spurious spin edges.
To face this problem the waiting interval has to be introduced.
Basically, every time a spin bit edge is detected, the observer sets
a time interval during which it rejects every potential spurious
edges observed on the wire. While, at the end of the interval it
starts again to accept changes in the spin bit value. This
guarantees a proper protection against the spurious edges in relation
to the size of the interval itself. For instance, an interval of 5ms
is able to filter out edges that have been reordered by a maximum of
5ms. Clearly, the mechanism does its job for intervals smaller than
the RTT of the observed connection (if RTT is smaller than the
waiting interval the observer can't measure the RTT).
5. Adding a Loss signal for Packet loss measurement
It is possible to introduce a mechanism to evaluate also the packet
loss together with the delay measurement. This can be achieved by
introducing the loss signal, a single bit signal whose purpose is to
mark a variable number of packets (from live traffic) which are
exchanged two times between the endpoints realizing a two round-trip
reflection. The overall exchange comprises:
o The client first selects, generates and consequent transmits to
the server a first train of packets, by marking the loss bit to 1;
o The server, upon reception from the client of each one of the
packets included in the first train, reflects to the client a
Cociglio, et al. Expires January 4, 2021 [Page 11]
Internet-Draft July 2020
respective second train of packets of the same size as the first
train received, by marking the loss bit to 1;
o The client, upon reception from the server of each one of the
packets included in the second train, reflects to the server a
respective third train of packets of the same size as the second
train received, by marking the loss bit to 1;
o The server, upon reception from the client of each one of the
packets included in the third train, finally reflects to the
client a respective fourth train of packets of the same size as
the third train received, by marking the loss bit to 1.
Packets belonging to the first round (first and second train)
represent the Generation Phase while those belonging to the second
round (third and fourth train) represent the Reflection Phase.
A passive on-path observer, placed on whatever direction, can
trivially count and compare the number of marked packets seen during
the two mentioned phases (i.e. the first and third or the second and
the fourth trains of packets, depending on which direction is
observed) and estimate the loss rate experienced by the connection.
This process is repeated continuously to obtain more measurements as
long as the endpoints exchange traffic. These measurements can be
called Round Trip(RT) losses
The general algorithm shown above gives an idea of its underlying
principles but is not enough to make the whole process working
properly.
Firstly, there is the issue that packet rates in the two directions
may be different. Therefore, the right number of packets to be
marked has to be chosen in order to avoid their congestion on the
slowest traffic direction. As a consequence, this number is
inevitably equal to the amount of packets transited, indeed, on the
slowest direction. This problem can be easily addressed by a method
wherein the two endpoints of a communication exchange marked packets
interleaved with unmarked packets. From an implementation point of
view, this result can be achieved by introducing a single token
system that adjusts the number of outgoing marked packets.
Basically, the token is enabled every time a packet arrives and
disabled when a marked packet is transmitted. Since the creation of
the initial train of marked packets is carried out by the client, the
management and use of this single token is also assigned to it, which
in fact "calculates" the correct number of packets to be marked each
time.
Cociglio, et al. Expires January 4, 2021 [Page 12]
Internet-Draft July 2020
Secondly, a mechanism to individually identify each train of packets
must be provided to enable the observer to distinguish between trains
belonging to different phases (Generation and Reflection).
5.1. Round Trip Packet Loss measurement
Since the measurements are performed on a portion of the traffic
exchanged between client and server, the observer calculates the end-
to-end Round Trip Packet Loss that, statistically, will be equal to
the loss rate experienced by the connection along the entire network
path. So this measurement can be simply referred as the Round Trip
Packet Loss (RTPL).
=======================|======================>
= ********** -----Obs----> ********** =
= * Client * * Server * =
= ********** <------------ ********** =
<==============================================
(a) client-server RTPL
==============================================>
= ********** ------------> ********** =
= * Client * * Server * =
= ********** <----Obs----- ********** =
<======================|=======================
(b) server-client RTPL
Figure 5: Round-trip packet loss (both direction)
In addition, this methodology allows the Half-RTPL measurement and
the Intra-domain RTPL measurement, in the same way as described in
the previous sections for RTT measurement.
Cociglio, et al. Expires January 4, 2021 [Page 13]
Internet-Draft July 2020
=======================>
= ********** ------|-----> **********
= * Client * Obs * Server *
= ********** <-----|------ **********
<=======================
(a) client-observer half-RTPL
=======================>
********** ------|-----> ********** =
* Client * Obs * Server * =
********** <-----|------ ********** =
<=======================
(b) observer-server half-RTPL
Figure 6: Half Round-trip packet loss (both direction)
=========================================>
=====================> =
********** ---|--> ---|--> ********** = =
* Client * Obs Obs * Server * = =
********** <--|--- <--|--- ********** = =
<===================== =
<=========================================
(a) observer-server RTPL components (half-RTPLs)
==================>
********** ---|--> ---|--> **********
* Client * Obs Obs * Server *
********** <--|--- <--|--- **********
<==================
(b) the intra-domain RTPL resulting from the
subtraction of the above RTPL components
Figure 7: Intra-domain Round-trip packet loss (observer-server)
6. Packet Loss using one bit loss signal
The single bit loss signal, whose basic mechanism was generalized in
the previous section, is implemented using just one bit: marked
packets have this bit set to 1, whereas unmarked ones have it set to
0. This solution requires a working spin-bit signal used to separate
Cociglio, et al. Expires January 4, 2021 [Page 14]
Internet-Draft July 2020
different trains of packets. In particular, a "pause" of at least
one empty spin-bit period is introduced between each phase of the
algorithm. An on-path observer can determine in this way if a phase
(and therefore a train of packets) is ended and a new one is
starting.
The client is in charge of almost the entire complexity of the
algorithm. Its task can be summarized in 4 different points:
1. The client starts generating marked packets for two consecutive
spin-bit periods; it maintains a generation token that is enabled
every time a packet arrives and disabled when another one is
forwarded. When this token is disabled, the generation process
is paused (i.e. outgoing packets are transmitted unmarked) and
resumes as soon as its value returns true, and that happens as
soon as a packet is received. In addition, at the end of the
first spin-bit period spent in generation, the reflection counter
is unlocked to start counting incoming marked packets which will
be later reflected;
2. When the generation is completed, the client waits to see in
input an empty spin-bit period so as to be sure that everyone has
seen at least that empty period. This one will be used by the
observer as a divider between generated and reflected packets.
During this phase, all the outgoing packets are forwarded with
the loss bit set to 0. The reflection counter is still
incremented every time a marked packet arrives;
3. The client starts reflecting marked packets until the reflection
counter is zeroed; the generation token is also used (in the same
way) during this phase to avoid congestion on the slowest traffic
direction. In addition, at the end of the first spin-period
spent in reflection, the reflection counter is locked to avoid
incoming reflected packets incrementing it;
4. When the reflection is completed, the client waits to see in
input an empty spin-bit period so as to be sure that everyone has
seen at least that empty period. This one will be used by the
observer as a divider between reflected and newly generated
packets. During this phase, all the outgoing packets are
forwarded with the loss bit set to 0. The whole process restarts
going back to the first point.
As previously anticipated, the server simply reflects each incoming
marked packet sent by the client. It maintains a simple counter that
is incremented every time a marked packet arrives and decremented
when a marked one is sent in the opposite direction.
Cociglio, et al. Expires January 4, 2021 [Page 15]
Internet-Draft July 2020
6.1. Observer's logic for one bit loss signal
The on-path observer, placed in any direction, counts marked packets
and separates different trains detecting empty spin-bit periods
between them (one or more). Then, it simply computes the difference
between a Generation train and a Reflection train to produce a
statistical measurement of the Round Trip Packet Loss (RTPL) and of
the connection end-to-end loss rate.
Here is an example. Packets are represented by two digits (first one
is the spin bit, second one is the loss bit):
Generation Pause Reflection Pause
____________________ ______________ ____________________ ________
| | | | |
01 01 00 01 11 10 11 00 00 10 10 10 01 00 01 01 10 11 10 00 00 10
Figure 8: one bit loss signal example
Note that 5 marked packets have been generated of which 4 reflected.
7. Two Bits packet loss measurement using alternate marking
An alternative methodology, based on the classical alternate marking
RFC 8321 [RFC8321], can be deployed to enable passive packet loss
measurement in a connection oriented communication. This section
explains its fundamentals and all the metrics that can be achieved by
exploiting this mechanism.
Two new loss bits are introduced:
o Square Bit (Q): this bit is toggled every N outgoing packets
generating a square signal as already seen in the alternate
marking methodology RFC 8321 [RFC8321].
o Reflection Square Bit (R): this bit is used to reflect the
incoming square signal (the one generated by the opposite
endpoint) according to the algorithm explained in next Section; in
a nutshell, it is used to report the losses found in the opposite
transmission channel.
7.1. Setting the square bit (Q) on outgoing packets
The sQuare value is initialized to 0 and is applied to the Q-bit of
every outgoing packet. The sQuare value is toggled after sending N
packets (e.g. 64). By doing so, each endpoint splits its outgoing
Cociglio, et al. Expires January 4, 2021 [Page 16]
Internet-Draft July 2020
traffic into blocks of N packets with different "packet color" as
defined by RFC 8321 [RFC8321]. A single block of N packets is called
"marking period". Observation points can estimate upstream losses by
counting the number of packets included in a marking period of the
produced square signal.
7.2. Setting the reflection square bit (R) on outgoing packets
Unlike the sQuare signal for which packets are transmitted into
blocks of fixed size, the Reflection square signal (being an
alternate marking signal too) produces blocks of packets whose size
varies according to these simple rules:
o when the transmission of a new block starts, its size is set equal
to the size of the last marking period whose reception has been
completed;
o if, before transmission of the block is terminated, the reception
of at least one further marking period is completed, the size of
the block is updated to the average size of the further received
marking periods. Implementation details follow.
The Reflection square value is initialized to 0 and is applied to the
R-bit of every outgoing packet. The Reflection square value is
toggled for the first time when the completion of a marking period is
detected in the incoming sQuare signal (produced by the opposite node
using the Q-bit). When this happens, the number of packets (p),
detected within this first marking period, is used to generate a
reflection square signal which toggles every M=p packets (at first).
This new signal produces blocks of M packets (marked using the R-bit)
and each of them is called "reflection marking period".
The M value is then updated every time a completed marking period in
the incoming sQuare signal is received, following this formula:
M=round(avg(p)).
The parameter avg(p) is the average number of packets in a marking
period computed considering all the marking periods received since
the beginning of the current reflection marking period.
Looking at the R-bit, observation points have clear indication of
losses experienced by the entire opposite channel plus those occurred
in the path from the sender up to them (if losses occur in this
latter portion of path).
Cociglio, et al. Expires January 4, 2021 [Page 17]
Internet-Draft July 2020
7.2.1. Determining the completion of an incoming marking period
A simple sQuare bit transition cannot be used to determine the
completion of a marking period. Indeed, packet reordering can lead
to the generation of spurious edges in the sQuare signal. To address
this problem, a marking period is considered ended when at least X
packets (e.g. 5) with reverse marking (i.e. belonging to the
following marking period) have been received.
This same approach can be used by observation points to clean both
sQuare and Reflection square signals.
7.3. Observer's logic and passive loss measurements
Since both sQuare and Reflection square bits are toggled at most
every N packets (except for the first transition of the R-bit as
explained before), an on-path observer can trivially count the number
of packets of each marking block and, knowing the value of N, can
estimate the amount of loss experienced by the connection. Different
metrics can be measured depending on which direction the observer is
looking to.
One direction observer:
o upstream one-way loss: the loss between the sender and the
observation point
o "three-quarters" connection loss: the loss between the receiver
and the sender in the opposite direction plus the loss between the
sender and the observation point in the observed direction
o full one-way loss in the opposite direction: the loss between the
receiver and the sender in the opposite direction
Two directions observer (same metrics seen previously applied to both
direction, plus):
o client-observer half round-trip loss: the loss between the client
and the observation point in both directions
o observer-server half round-trip loss: the loss between the
observation point and the server in both directions
o downstream one-way loss: the loss between the observation point
and the receiver (valid for both directions)
Cociglio, et al. Expires January 4, 2021 [Page 18]
Internet-Draft July 2020
7.3.1. Upstream one-way loss
Since packets are continuously Q-bit marked into alternate blocks of
size N, knowing the value of N, an on-path observer can estimate the
amount of loss occurred from the sender up to it after observing at
least N packets. The upstream one-way loss rate ("uowl") is one
minus the average number of packets in a block of packets with the
same Q value ("p") divided by N ("uowl=1-avg(p)/N").
=====================>
********** -----Obs----> **********
* Client * * Server *
********** <------------ **********
(a) in client-server channel (uowl_up)
********** ------------> **********
* Client * * Server *
********** <----Obs----- **********
<=====================
(b) in server-client channel (uowl_down)
Figure 9: Upstream one-way loss
7.3.2. Three-quarters connection loss
Except for the very first block in which there is nothing to reflect
(a complete marking period has not been yet received), packets are
continuously R-bit marked into alternate blocks of size lower or
equal than N. Knowing the value of N, an on-path observer can
estimate the amount of loss occurred in the whole opposite channel
plus the loss from the sender up to it in the observation channel.
As for the previous metric, the "three-quarters" connection loss rate
("tql") is one minus the average number of packets in a block of
packets with the same R value ("t") divided by N ("tql=1-avg(t)/N").
Cociglio, et al. Expires January 4, 2021 [Page 19]
Internet-Draft July 2020
=======================>
= ********** -----Obs----> **********
= * Client * * Server *
= ********** <------------ **********
<============================================
(a) in client-server channel (tql_up)
============================================>
********** ------------> ********** =
* Client * * Server * =
********** <----Obs----- ********** =
<=======================
(b) in server-client channel (tql_down)
Figure 10: Three-quarters connection loss
The following metrics derive from these first two metrics.
7.3.3. Full one-way loss in the opposite direction
Using the previous metrics, full one-way loss can be computed:
fowl_down = tql_up - uowl_up
fowl_up = tql_down - uowl_down
********** -----Obs----> **********
* Client * * Server *
********** <------------ **********
<==========================================
(a) in client-server channel (fowl_down)
==========================================>
********** ------------> **********
* Client * * Server *
********** <----Obs----- **********
(b) in server-client channel (fowl_up)
Figure 11: Full one-way loss in the opposite direction
Cociglio, et al. Expires January 4, 2021 [Page 20]
Internet-Draft July 2020
7.3.4. Half round-trip loss
Using the previous metrics, the two half round-trip loss measurements
can be computed:
hrtl_co = tql_up - uowl_down
hrtl_os = tql_down - uowl_up
=======================>
= ********** ------|-----> **********
= * Client * Obs * Server *
= ********** <-----|------ **********
<=======================
(a) client-observer half round-trip loss (hrtl_co)
=======================>
********** ------|-----> ********** =
* Client * Obs * Server * =
********** <-----|------ ********** =
<=======================
(b) observer-server half round-trip loss (hrtl_os)
Figure 12: Half Round-trip loss (both direction)
7.3.5. Downstream one-way loss
Using the previous metrics, downstream one-way loss can be computed:
dowl_up = hrtl_os - uowl_down
dowl_down = hrtl_co - uowl_up
Cociglio, et al. Expires January 4, 2021 [Page 21]
Internet-Draft July 2020
=====================>
********** ------|-----> **********
* Client * Obs * Server *
********** <-----|------ **********
(a) in client-server channel (dowl_up)
********** ------|-----> **********
* Client * Obs * Server *
********** <-----|------ **********
<=====================
(b) in server-client channel (dowl_down)
Figure 13: Downstream one-way loss
7.4. Enhancement of reflection period size computation
The use of the rounding function used in the M computation introduces
errors. However, these errors can be minimized by storing the
rounding applied each time M is computed, and using it during the
computation of the M value in the following reflection marking
period.
This can be achieved introducing the new r_avg parameter in the
previous M formula. The new formula is M=round(avg(p)+r_avg) where
r_avg is computed as not rounded M minus rounded M; its initial value
is equal to 0.
7.5. Improvement of the resilience to out of sequence
Since endpoints have clear indication about reordered packets, we can
use this information to absorb out of sequences in the incoming
sQuare wave, even when the marking period threshold (see 7.2.1
Section) has been reached.
This can be achieved by updating the size of the current reflection
block while this is being transmitted. The reflection block size is
then updated every time an incoming reordered packet of the previous
marking period is detected. This can be done if and only if the
transmission of the current reflection block is in progress and no
packets of the following marking period (Q-bit) have been received.
Cociglio, et al. Expires January 4, 2021 [Page 22]
Internet-Draft July 2020
8. Protocols
8.1. QUIC
The binding of the delay bit signal to QUIC is partially described in
[I-D.ietf-quic-transport], which adds the spin bit to the first byte
of the short packet header, leaving two reserved bits for future
experiments.
To implement the additional signals discussed in this document, the
first byte of the short packet header can be modified as follows:
the delay bit (D) can be placed in the first reserved bit (i.e.
the fourth most significant bit _0x10_) while the loss bit in the
second reserved bit (i.e. the fifth most significant bit _0x08_);
the proposed scheme is:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|0|1|S|D|L|K|P|P|
+-+-+-+-+-+-+-+-+
Figure 14: scheme 1
alternatively, the standalone two bits loss signal (QR) can be
placed in both reserved bits; the proposed scheme, in this case,
is:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|0|1|S|Q|R|K|P|P|
+-+-+-+-+-+-+-+-+
Figure 15: scheme 2
8.2. TCP
The signals can be added to TCP by defining bit 4 of bytes 13-14 of
the TCP header to carry the spin bit, and eventually bits 5 and 6 to
carry additional information, like the delay bit and the 1 bit loss
signal (or the two bits loss signal).
9. Security Considerations
The privacy considerations for the hybrid RTT measurement signal are
essentially the same as those for passive RTT measurement in general.
Cociglio, et al. Expires January 4, 2021 [Page 23]
Internet-Draft July 2020
10. Acknowledgements
tbc
11. IANA Considerations
tbc
12. References
12.1. Normative References
[I-D.ietf-quic-spin-exp]
Trammell, B. and M. Kuehlewind, "The QUIC Latency Spin
Bit", draft-ietf-quic-spin-exp-01 (work in progress),
October 2018.
[I-D.ietf-quic-transport]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", draft-ietf-quic-transport-29 (work
in progress), June 2020.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli,
L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi,
"Alternate-Marking Method for Passive and Hybrid
Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321,
January 2018, <https://www.rfc-editor.org/info/rfc8321>.
12.2. Informative References
[ANRW19-PM-QUIC]
ACM/IRTF Applied Networking Research Workshop 2019
(ANRW'19), "Performance measurements of QUIC
communications", DOI 10.1145/3340301.3341127, 2019.
[I-D.trammell-ippm-spin]
Trammell, B., "An Explicit Transport-Layer Signal for
Hybrid RTT Measurement", draft-trammell-ippm-spin-00 (work
in progress), January 2019.
Cociglio, et al. Expires January 4, 2021 [Page 24]
Internet-Draft July 2020
[I-D.trammell-quic-spin]
Trammell, B., Vaere, P., Even, R., Fioccola, G., Fossati,
T., Ihlar, M., Morton, A., and S. Emile, "Adding Explicit
Passive Measurability of Two-Way Latency to the QUIC
Transport Protocol", draft-trammell-quic-spin-03 (work in
progress), May 2018.
[I-D.trammell-tsvwg-spin]
Trammell, B., "A Transport-Independent Explicit Signal for
Hybrid RTT Measurement", draft-trammell-tsvwg-spin-00
(work in progress), July 2018.
Authors' Addresses
Mauro Cociglio
Telecom Italia
Via Reiss Romoli, 274
Torino 10148
Italy
Email: mauro.cociglio@telecomitalia.it
Giuseppe Fioccola
Huawei Technologies
Riesstrasse, 25
Munich 80992
Germany
Email: giuseppe.fioccola@huawei.com
Massimo Nilo
Telecom Italia
Via Reiss Romoli, 274
Torino 10148
Italy
Email: massimo.nilo@telecomitalia.it
Fabio Bulgarella
Telecom Italia
Via Reiss Romoli, 274
Torino 10148
Italy
Email: fabio.bulgarella@guest.telecomitalia.it
Cociglio, et al. Expires January 4, 2021 [Page 25]
Internet-Draft July 2020
Riccardo Sisto
Politecnico di Torino
Corso Duca degli Abruzzi, 24
Torino 10129
Italy
Email: riccardo.sisto@polito.it
Cociglio, et al. Expires January 4, 2021 [Page 26]