Internet DRAFT - draft-csfx-ippm-hipmetrics
draft-csfx-ippm-hipmetrics
Network Working Group A. Clemm
Internet-Draft J. Strassner
Intended status: Standards Track Futurewei
Expires: April 23, 2022 J. Francois
Inria
October 20, 2021
High-Precision Service Metrics
draft-csfx-ippm-hipmetrics-00
Abstract
This document defines a set of metrics for high-precision networking
services. These metrics can be used to assess the service levels
that are being delivered for a networking flow. Specifically, they
can be used to determine the degree of compliance with which service
levels are being delivered relative to service level objectives that
were defined for the flow. The metrics can be used as part of flow
records and/or accounting records. They can also be used to
continuously monitor the quality with which high-precision networking
service are being delivered.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 23, 2022.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
Clemm, et al. Expires April 23, 2022 [Page 1]
Internet-Draft High-Precision Service Metrics October 2021
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Key Words . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Definitions and Acronyms . . . . . . . . . . . . . . . . . . 3
4. Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
5. Discussion Items . . . . . . . . . . . . . . . . . . . . . . 7
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
7. Security Considerations . . . . . . . . . . . . . . . . . . . 7
8. Normative References . . . . . . . . . . . . . . . . . . . . 8
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9
1. Introduction
Many networking applications increasingly rely on high-precision
networking services that have clearly defined service level
objectives (SLOs), for example with regards to end-to-end latency.
Applications requiring such services include industrial networks, for
example cloud-based industrial controllers for precision machinery,
vehicular applications, for example tele-driving in which a vehicle
is remotely controlled by a human operators, or Augmented Reality /
Virtual Reality (AR/VR) applications involving rendering of point
clouds remotely. Many of those applications are not tolerant of
degrading service levels. A slight miss in SLOs does not merely
result in a slight deterioration of the Quality of Experience to end
users, but may render the application inoperable. At the same time,
many of those applications are mission critical, in which sudden
failures can jeopardize safety or have other adverse consequences.
However, clearly those applications represent significant business
opportunity demanding dependable technical solutions.
Because of this, efforts such as Deterministic Networking (DetNet)
[RFC8655] are attempting to create solutions in which clear bounds on
parameters such as end-to-end latency and jitter can be defined in
order to make service levels being delivered predictable and,
ideally, deterministic. However, one area that has not kept pace
concerns metrics that can account for service levels with which
services are delivered, specifically the degree of precision for
agreed-upon service level objectives. Such metrics, and the
instrumentation to support them, are important for a number of
purposes, including monitoring (to ensure that networking services
Clemm, et al. Expires April 23, 2022 [Page 2]
Internet-Draft High-Precision Service Metrics October 2021
are performing according to their objectives) as well as accounting
(to maintain a record of service levels actually delivered, important
for monetization of such services as well as for triaging of
problems).
The current state-of-the-art of such metrics includes (for example)
interface metrics, useful to obtain data on traffic volume and
behavior that can be observed at an interface [RFC2863] [RFC8343] but
agnostic of actual end-to-end service levels and not specific to
distinct flows. Flow records [RFC7011] [RFC7012] maintain statistics
about flows, including flow volume and flow duration, but again
contain very little information about end-to-end service levels, let
alone whether the service levels delivered meet their targets, i.e.
their associated SLOs.
This specification introduces a new set of metrics aimed at capturing
end-to-end service levels for a flow, specifically the degree to
which flows comply with the SLOs that are in effect.
It should be noted that at this point, the set of metrics proposed
here is intended as a "starter set" that is intended to spark further
discussion. Other metrics are certainly conceivable; we expect that
the list of metrics will evolve over time as part of Working Group
discussions.
2. Key Words
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
3. Definitions and Acronyms
MTBF: Mean Time Between Failures
SL: Service Level
SLA: Service Level Agreement
SLO: Service Level Objective
4. Metrics
The following section proposes a set of accounting metrics focus on
end-to-end latency objectives. They indicate whether any violations
of end-to-end latency occurred at the packet level. These metrics
Clemm, et al. Expires April 23, 2022 [Page 3]
Internet-Draft High-Precision Service Metrics October 2021
are intended to be applied on a per-flow basis and are intended to
assess the degree to which a flow's end-to-end service levels comply
with the SLO in effect for that flow.
While the focus in this document concerns end-to-end latency
objectives, analogous metrics could also be defined for other end-to-
end service level parameters, such as loss (which is distinct from
loss occurring at any one given interface) or delay variation.
o Violated Packets. This indicates the number of packets for which
a violation of a latency SLO occurred.
o Violated Time Units (e.g. violated seconds, violated
milliseconds). This indicates the number of time units during
which one or more violations of SLOs were observed, regardless of
how many violations took place during the same interval. This
measure is useful in scenarios where bursts of violations might
suddenly occur (e.g. due to temporary network congestion, during
route convergence etc.) and the count of violated packets by
itself might paint a misleading picture.
The following additional set of metrics may be useful in certain
scenarios as well. However, their precise definition may be subject
to policy and further discussion is needed:
o Significantly Violated Packets. This indicates the number of
packets for which a "significant" violation occurred, where
"significant" implies an SLO that was not merely a near-miss but
that missed the objective by a degree determined especially
significant.
o Significantly Violated Time Units (e.g. significantly violated
seconds, significantly violated milliseconds). This indicates the
number of time units during which any significant violation
occurred.
o Severely Violated Time Units (e.g. severely violated seconds,
severely violated milliseconds). "Severe" here refers to the
occurrence of multiple violations within the same time unit. The
definition of "severe" may be subject to policy; it may also take
into account the significance of the violations that occur.
Note that there is no definition of Severely Violated Packets. The
term "severe" is used in conjunction with the occurrence of multiple
violations related to multiple packets, not any one packet in
isolation.
Clemm, et al. Expires April 23, 2022 [Page 4]
Internet-Draft High-Precision Service Metrics October 2021
From these first-order metrics, second-order metrics can be defined
that build on the first set of metrics. Some of these metrics are
modeled after Mean Time Between Failure, or MTBF metrics - a
"failure" in this context referring to a failure to deliver a packet
according to its SLO.
o Time since last violated time unit (i.e., since last violated ms,
since last violated second). (This parameter is particularly
useful for the monitoring of the current health.)
o Packets since last violated packet. (This parameter is
particularly useful for the monitoring of the current health.)
o Mean time between violated time units (i.e. between violated
milliseconds, between violated seconds). This refers to the
arithmetic mean of time between violations such as violated time
units.
o Mean packets between violations. This refers to the arithmetic
mean of the number of SLO-compliant packets between SLO
violations. (Another variation of "MTBF" in a service setting.)
The same set of metrics can also be applied to significant
violations, and to severe violations:
o Time since last significantly violated time unit (i.e., since last
significantly violated ms, since last significantly violated
second).
o Time since last severely violated time unit (i.e., since last
severely violated ms, since last severely violated second).
o Packets since last significatly violated packet.
o Mean time between significantly violated time units (i.e. between
significantly violated milliseconds, between significantly
violated seconds).
o Mean time between severely violated time units (i.e. between
severely violated milliseconds, between severely violated
seconds).
o Mean packets between significant violations. This refers to the
arithmetic mean of the number of SLO-compliant packets between
significant SLO violations.
The next set of metrics puts the violations in relationship to non-
violations. It is intended to provide an analogous measure to that
Clemm, et al. Expires April 23, 2022 [Page 5]
Internet-Draft High-Precision Service Metrics October 2021
of availability, typically defined as the number of time units during
which a system (or service) is unavailable divided by the total
number of time units. In analogy, a time unit that is "violated" can
be viewed as one in which a service is not available with the
advertised precision:
o Precision availability (of milliseconds, of seconds): the ratio
between violated time units (seconds, milliseconds) and the total
time units for the duration of the service.
o Analogous metrics for precision availability re: severely violated
time units, re: significantly violated time units.
It should be noted that certain Service Level Agreements may be
statistical in nature, requiring the service levels of packets in a
flow to adhere to certain distributions. For example, an SLA might
state that any given SLO applies only to a certain percentage of
packets, allowing for a certain amount of violations to take place.
A "violated packet" in that case does not necessarily constitute an
SLO violation. However, it is still useful to maintain those
statistics, as the number of violated packets still matters when
looked at in proportion to the total number of packets.
Along that vein, an SLA might establish an SLO of, say, end-to-end
latency to not exceed 20ms for 99% of packets, to not exceed 25ms for
99.999% of packets, and to never exceed 30ms for anything beyond. In
that case, any individual packet missing the 20 ms latency target
cannot be considered an SLO violation in itself, but compliance with
the SLO may need to be assessed after the fact.
To support statistical SLAs more directly, it is feasible to support
additional metrics, such as metrics that represent histograms for
service level parameters with buckets corresponding to individual
service level objectives. For the example just given, a histogram
for a given flow could be maintained with three buckets: one
containing the count of packets within 20ms, a second with a count of
packets between 20 and 25ms (or simply all within 25ms), a third with
a count of packet between 25 and 30ms (or simply all packets within
30ms, and a fourth with a count of anything beyond (or simply a total
count). Of course, the number of buckets and the boundaries between
those buckets should correspond to the needs of the application
respectively SLA, i.e. to the specific guarantees and SLOs that were
provided. The definition of histogram metrics is for further study.
Clemm, et al. Expires April 23, 2022 [Page 6]
Internet-Draft High-Precision Service Metrics October 2021
5. Discussion Items
The following is a list of items for which further discussion is
needed as to whether they should be included in the scope of this
specification:
o A YANG data model
o A set of IPFIX Information Elements
o Statistical metrics: e.g. histograms/buckets
o Policies regarding the definition of "significant" and "severe"
violations
o Additional second-order metrics, such as "longest disruption of
service time" (measuring consecutive time units with violations)
6. IANA Considerations
TBD
7. Security Considerations
Instrumentation for metrics that are used to assess compliance with
SLOs consitute an interesting target for an attacker. By interfering
with the maintaining of such metrics, services could be falsely
identified as being in compliance (when they are not), or vice-versa
flagged as being non-compliant (when indeed they are). While this
document does not specify how networks should be instrumented to
maintain the identified metrics, such instrumentation needs to be
properly secured to ensure accurate measurements and prohibit
tampering with metrics being kept.
Where metrics are being defined relative to an SLO, the configuration
of those SLOs needs to be properly secured. Likewise, where SLOs can
be adjusted, it needs to be clear which particular SLO any given
metrics instance refers to. The same service levels that constitute
SLO violations for one flow, and that should be maintained as part of
the "violated time units", "violated packets", and related metrics,
may be perfectly compliant for another flow. Where it is not
possible to properly tie together SLOs and violation metrics, it will
be preferrable to merely maintain statistics about sevice levels that
were delivered (for example, overall histograms of end-to-end
latency), without assessing which of these constitute violations.
By the same token, where the definition of what constitutes a
"severe" violation or a "significant" violation depends on policy or
Clemm, et al. Expires April 23, 2022 [Page 7]
Internet-Draft High-Precision Service Metrics October 2021
context, the configuration of such policy or context needs to be
specially secured and the configuration of this policy be bound to
the metrics being maintained. This way it will be clear which policy
was in effect when those metrics were being assessed. An attacker
that is able to tamper with such policies will render the
corresponding metrics useless (in the best case) or misleading (in
the worst case).
8. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group
MIB", RFC 2863, DOI 10.17487/RFC2863, June 2000,
<https://www.rfc-editor.org/info/rfc2863>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model
for IP Flow Information Export (IPFIX)", RFC 7012,
DOI 10.17487/RFC7012, September 2013,
<https://www.rfc-editor.org/info/rfc7012>.
[RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language",
RFC 7950, DOI 10.17487/RFC7950, August 2016,
<https://www.rfc-editor.org/info/rfc7950>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8343] Bjorklund, M., "A YANG Data Model for Interface
Management", RFC 8343, DOI 10.17487/RFC8343, March 2018,
<https://www.rfc-editor.org/info/rfc8343>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>.
Clemm, et al. Expires April 23, 2022 [Page 8]
Internet-Draft High-Precision Service Metrics October 2021
Authors' Addresses
Alexander Clemm
Futurewei
2330 Central Expressway
Santa Clara CA 95050
USA
Email: ludwig@clemm.org
John Strassner
Futurewei
2330 Central Expressway
Santa Clara CA 95050
USA
Email: strazpdj@gmail.com
Jerome Francois
Inria
615 Rue du Jardin Botanique
Villers-les-Nancy 54600
France
Email: jerome.francois@inria.fr
Clemm, et al. Expires April 23, 2022 [Page 9]
