Internet DRAFT - draft-ietf-opsawg-ipfix-on-path-telemetry
draft-ietf-opsawg-ipfix-on-path-telemetry
Network Working Group T. Graf
Internet-Draft Swisscom
Intended status: Standards Track B. Claise
Expires: 17 July 2024 Huawei
A. Huang Feng
INSA-Lyon
14 January 2024
Export of On-Path Delay in IPFIX
draft-ietf-opsawg-ipfix-on-path-telemetry-06
Abstract
This document introduces new IP Flow Information Export (IPFIX)
information elements to expose the On-Path Telemetry measured delay
on the IOAM transit and decapsulation nodes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 17 July 2024.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Graf, et al. Expires 17 July 2024 [Page 1]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . 6
3.1. IP One-Way Delay Hybrid Type I Passive Performance
Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.1. Summary . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.2. Description . . . . . . . . . . . . . . . . . . . . . 7
3.1.3. Change Controller . . . . . . . . . . . . . . . . . . 7
3.1.4. Version of Registry Format . . . . . . . . . . . . . 7
3.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 7
3.2.1. Reference Definition . . . . . . . . . . . . . . . . 7
3.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 8
3.3. Method of Measurement . . . . . . . . . . . . . . . . . . 8
3.3.1. Reference Methods . . . . . . . . . . . . . . . . . . 8
3.3.2. Packet Stream Generation . . . . . . . . . . . . . . 8
3.3.3. Traffic Filtering (Observation) Details . . . . . . . 8
3.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 9
3.3.5. Runtime Parameters and Data Format . . . . . . . . . 9
3.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . . 9
3.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.2. Reference Definition . . . . . . . . . . . . . . . . 10
3.4.3. Administrative Items . . . . . . . . . . . . . . . . 12
3.4.4. Comments and Remarks . . . . . . . . . . . . . . . . 13
4. IPFIX Information Elements . . . . . . . . . . . . . . . . . 13
5. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 14
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
6.1. Performance Metrics . . . . . . . . . . . . . . . . . . . 15
6.2. IPFIX Entities . . . . . . . . . . . . . . . . . . . . . 15
6.2.1. PathDelayMeanDeltaMicroseconds . . . . . . . . . . . 16
6.2.2. PathDelayMinDeltaMicroseconds . . . . . . . . . . . . 16
6.2.3. PathDelayMaxDeltaMicroseconds . . . . . . . . . . . . 17
6.2.4. PathDelaySumDeltaMicroseconds . . . . . . . . . . . . 17
7. Operational Considerations . . . . . . . . . . . . . . . . . 17
7.1. Time Accuracy . . . . . . . . . . . . . . . . . . . . . . 17
7.2. Mean Delay . . . . . . . . . . . . . . . . . . . . . . . 18
7.3. Reduced-size encoding . . . . . . . . . . . . . . . . . . 18
7.4. IOAM Application . . . . . . . . . . . . . . . . . . . . 18
8. Security Considerations . . . . . . . . . . . . . . . . . . . 18
9. Implementation Status . . . . . . . . . . . . . . . . . . . . 19
9.1. FD.io VPP . . . . . . . . . . . . . . . . . . . . . . . . 19
9.2. Huawei VRP . . . . . . . . . . . . . . . . . . . . . . . 19
9.3. Fluvia . . . . . . . . . . . . . . . . . . . . . . . . . 19
9.4. Pmacct Data Collection . . . . . . . . . . . . . . . . . 20
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 20
Graf, et al. Expires 17 July 2024 [Page 2]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
11.1. Normative References . . . . . . . . . . . . . . . . . . 20
11.2. Informative References . . . . . . . . . . . . . . . . . 20
Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . 24
A.1. Aggregated On-Path Dealay Examples . . . . . . . . . . . 24
A.1.1. Template Record and Data Set with Mean Delta . . . . 24
A.1.2. Template Record and Data Set with Sum Delta . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28
1. Introduction
Network operators want a statistical delay view of their networks.
They want to understand where in the network, for which customer
traffic, how much and why delay is being accummlated. In order to
answer why and where, delay needs to be reported into device and
control-plane context. In order to understand which customer traffic
is affected, delay needs to be reported into customer data-plane
context. That enables network operators to quickly identify when the
control-plane updates the current path with a different next-hop and
therefore the forwarding path changes to different nodes and
interfaces, how the path delay changes for which customer traffic.
With On-Path Telemetry, described in the Network Telemetry Framework
[RFC9232] and applied in In-situ OAM [I-D.ietf-ippm-ioam-deployment]
and Alternate Marking Deployment Framework
[I-D.ietf-ippm-alt-mark-deployment], the path delay between two
endpoints is measured by inserting a timestamp in the packet.
On-Path Telemetry can be distinguished between two modes. Passport
mode, [RFC9197], where only the last hop in the forwarding path of
the On-Path Telemetry domain exposes all the metrics, and postcard
mode, [I-D.song-ippm-postcard-based-telemetry], where the metrics are
also exposed in the transit nodes. In both modes the forwarding path
exposes performance metrics allowing to determine how much delay has
been accumulated on which hop.
This document defines four new IPFIX Information Elements (IEs),
exposing the On-Path delay on IOAM transit and decapsulation nodes,
following the postcard mode principles. Since these IPFIX IEs are
performance metrics [RFC8911], they must be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
Following the guidelines for Registered Performance Metric requesters
and reviewers [RFC8911], the different characteristics of the
performance metrics (Identifier, Name, URI, Status, Requester,
Revision, Revision Date, Description, etc) must be clearly specified
in the "IANA Performance Metric Registry [IANA-PERF-METRIC] in order
for the results of measurements using the Performance Metrics to be
comparable even if they are performed by different implementations
Graf, et al. Expires 17 July 2024 [Page 3]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
and in different networks. These characteristics start by selecting
a meaningful name, following the "MetricType_Method_SubTypeMethod_...
Spec_Units_Output" naming convention (See Section 7.1.2 of
[RFC8911]).
+------------------------------------+-------------------------------+
| Performance Metric | IPFIX Information Element |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMeanDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Mean (TBD1)|(TBD5) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMinDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Min (TBD2) |(TBD6) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMaxDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Max (TBD3) |(TBD7) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelaySumDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Sum (TBD4) |(TBD8) |
+------------------------------------+-------------------------------+
Table 1: Correspondance between IPFIX IE and Performance Metric
The delay is measured by calculating the difference between the
timestamp imposed with On-Path Telemetry in the packet at the IOAM
encapsulation node and the timestamp exported in the IPFIX flow
record from the IOAM transit and decapsulation nodes. The lowest,
highest, mean, and/or the sum of measured path delay can be exported,
thanks to the different IPFIX IE specifications.
On-Path Telemetry Domain
.........................................
. .
. D1 .
. <------> .
. .
. D2 .
. <--------------------> .
. .
. D3 .
. <-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1 Encapsulation Transit Transit Decapsulation Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
Graf, et al. Expires 17 July 2024 [Page 4]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Figure 1: Delay use case. Packets flow from host 1 to host 2.
On the usecase showed in Figure 1 using On-path Telemetry to export
the delay metrics, the node R2 exports the delay D1, the node R3
exports the delay D2 and the decapsulation node R4 exports the total
delay D3 using IPFIX.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document makes use of the terms defined in [RFC7011] and
[I-D.ietf-ippm-ioam-deployment].
The following terms are used as defined in [RFC7011].
* IPFIX
* IPFIX Information Elements (IEs)
* Flow Record
* Exporter
The following terms are used as defined in [RFC8911].
* Performance Metric
* Registered Performance Metric
* Performance Metrics Registry
The following terms are used as defined in
[I-D.ietf-ippm-ioam-deployment].
* IOAM encapsulation node
* IOAM transit node
* IOAM decapsulation node
Graf, et al. Expires 17 July 2024 [Page 5]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
3. Performance Metrics
This section defines and describes the new performance metrics by
applying the template defined in Section 11 of [RFC8911].
3.1. IP One-Way Delay Hybrid Type I Passive Performance Metrics
This section specifies four performance metrics for the Hybrid Type I
Passive assessment of IP One-Way Delay, to be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
All column entries besides the ID, Name, Description, and Output
Reference Method categories are the same; thus, this section defines
four closely related performance metrics. As a result, IANA has
assigned corresponding URLs to each of the four registered
performance metrics.
3.1.1. Summary
This category includes multiple indexes of the registered performance
metrics: the element ID and Metric Name.
3.1.1.1. ID (Identifier)
<insert a numeric Identifier, an integer, TBD>
3.1.1.2. Name
IANA has allocated the numeric Identifiers TBD1-4 for the four Named
Metric Entries in this section
3.1.1.3. Name
TBD1: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
TBD2: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
TBD3: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
TBD4: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
3.1.1.4. URI
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
Graf, et al. Expires 17 July 2024 [Page 6]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
3.1.2. Description
This metric assesses the one-way delay of IP packets constituting a
single connection between two hosts. We consider the measurement of
one-way delay based on a single Observation Point (OP) [RFC7011]
somewhere in the network. The output is the one-way delay for all
successfully forwarded packets expressed as the <statistic> of their
conditional delay distribution, where <statistic> is one of:
* Mean
* Min
* Max
* Sum
3.1.3. Change Controller
IETF
3.1.4. Version of Registry Format
1.0
3.2. Metric Definition
This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable
document reference and values of input factors, called "Fixed
Parameters".
3.2.1. Reference Definition
Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-
Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC
7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-
editor.org/info/rfc7679>. [RFC7679]
Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC
6049, DOI 10.17487/RFC6049, January 2011, <https://www.rfc-
editor.org/info/rfc6049>. [RFC6049]
Graf, et al. Expires 17 July 2024 [Page 7]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Section 3.4 of [RFC7679] provides the reference definition of the
singleton (single value) one-way delay metric. Section 4.4 of
[RFC7679] provides the reference definition expanded to cover a
multi-value sample. Note that terms such as "singleton" and "sample"
are defined in section 2 of [RFC2330].
With the OP [RFC7011] typically located between the hosts
participating in the IP connection, the one-way delay metric requires
one individual measurement between the OP and sourcing host, such
that the Spatial Composition [RFC6049] of the measurements yields a
one-way delay singleton.
3.2.2. Fixed Parameters
Traffic Filters:
IPv4 header values:
DSCP: Set to 0
IPv6 header values:
DSCP: Set to 0
Hop Count: Set to 255
Flow Label: Set to 0
Extension Headers: None
3.3. Method of Measurement
This category includes columns for references to relevant sections of
the RFC(s) and any supplemental information needed to ensure an
unambiguous method for implementations.
3.3.1. Reference Methods
The foundational methodology for this metric is defined in section 4
of [RFC7323] using the Timestamps option with modifications that
allow application at a mid-path OP [RFC7011].
3.3.2. Packet Stream Generation
N/A
3.3.3. Traffic Filtering (Observation) Details
The Fixed Parameters above give a portion of the Traffic Filter.
Other aspects will be supplied as Runtime Parameters (below).
Graf, et al. Expires 17 July 2024 [Page 8]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
3.3.4. Sampling Distribution
This metric requires a partial sample of all packets that qualify
according to the Traffic Filter criteria.
3.3.5. Runtime Parameters and Data Format
Runtime Parameters are input factors that must be determined,
configured into the measurement system, and reported with the results
for the context to be complete.
The hybrid type I metering parameters must must be reported to
provide the complete measurement context. As an example, if the
IPFIX metering process is used, then the IPFIX metering process
parameters (IPFIX template record used, potential traffic filters,
and potential sampling method and parameters) that generates the flow
records must be reported to provide the complete measurement context.
Src: The IP address of the host in the host A Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see section 4 of [RFC6991].
Dst: The IP address of the host in the host B Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see section 4 of [RFC6991].
TTL or Hop Limit: Set at desired value.
DSCP: Set at desired value.
IPv6 Flow Label: Set at desired value.
Timestamp: The timestamp when the packet is being received at IOAM
encapsulation node. Format depends on On-Path Telemetry
implementation. For IOAM, Section 4.4.1 of [RFC9197] describes
what kind of timestamps are supported. Section 4.4.2.3 and
4.4.2.4 describe where the timestamp is being inserted. For the
Enhanced Alternate Marking Method, Section 2 of
[I-D.zhou-ippm-enhanced-alternate-marking] describes timestamp
encoding and granularity.
3.3.6. Roles
host A: Launches the IP packet to open the connection. The Role of
"host A" is synonymous with the IP address used at host A.
host B: Receives the IP packet to open the connection. The Role of
"host B" is synonymous with the IP address used at host B.
Graf, et al. Expires 17 July 2024 [Page 9]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Encapsulation Node: Receives the IP packet to open the connection
and encapsulates the timestamp into the packet. The Role of
"Encapsulation Node" is synonymous with the timestamp inserted in
the packet.
Transit Node: Receives the IP packet to open the connection and
measures the delay between the timestamp in the packet and the
timestamp when the packet was received.
Decapsulation Node: Receives the IP packet to open the connection
and measures the delay between the timestamp in the packet and the
timestamp when the packet was received and removes the IOAM header
from the packet.
3.4. Output
This category specifies all details of the output of measurements
using the metric.
3.4.1. Type
OWDelay Types are discussed in the subsections below.
3.4.2. Reference Definition
For all output types:
OWDelay_HybridType1_Passive_IP: The one-trip delay of one IP packet
is a Singleton
For each <statistic> Singleton one of the following subsections
applies.
3.4.2.1. Mean
The mean SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.2.2 of [RFC6049] for details on calculating this
statistic; see also section 4.2.3 of [RFC6049].
Mean: The time value of the result is expressed in units of seconds,
Graf, et al. Expires 17 July 2024 [Page 10]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.2. Min
The minimum SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for details on calculating this
statistic; see also section 4.3.3 of [RFC6049].
Min: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.3. Max
The maximum SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for a closely related method for
calculating this statistic; see also section 4.3.3 of [RFC6049]. The
formula is as follows:
Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
where all packets n = 1 through N have finite singleton delays.
Max: The time value of the result is expressed in units of seconds,
Graf, et al. Expires 17 July 2024 [Page 11]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.4. Sum
The sum SHALL be calculated using the conditional distribution of all
packets with a finite value of one-way delay (undefined delays are
excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.3.5 of [RFC6049] for details on calculating this
statistic. However in this case FiniteDelay or MaxDelay MAY be used.
Sum: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.5. Metric Units
The <statistic> of one-way delay is expressed in seconds, where
<statistic> is one of:
* Mean
* Min
* Max
* Sum
The one-way delay of the IP connection singleton is expressed in
seconds.
3.4.2.6. Calibration
Passive Measurements at an OP could be calibrated against an Active
Measurement at host A where the Active Measurement represents the
ground truth.
3.4.3. Administrative Items
Graf, et al. Expires 17 July 2024 [Page 12]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
3.4.3.1. Status
Current
3.4.3.2. Requester
This RFC
3.4.3.3. Revision
1.0
3.4.3.4. Revision Date
RFC Date
3.4.4. Comments and Remarks
none
4. IPFIX Information Elements
This section defines and describes the new IPFIX IEs.
PathDelayMeanDeltaMicroseconds
32-bit unsigned integer that identifies the mean path delay in
microseconds, between the IOAM encapsulation node and the local
node with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).
PathDelayMinDeltaMicroseconds
32-bit unsigned integer that identifies the lowest path delay in
microseconds, between the IOAM encapsulation node and the local
node with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).
PathDelayMaxDeltaMicroseconds
32-bit unsigned integer that identifies the highest path delay in
microseconds, between the IOAM encapsulation node and the local
node with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).
PathDelaySumDeltaMicroseconds
64-bit unsigned integer that identifies the sum of the path delay
in microseconds, between the IOAM encapsulation node and the local
node with the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node).
Graf, et al. Expires 17 July 2024 [Page 13]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
5. Use Cases
The measured On-Path delay can be aggregated with Flow Aggregation as
defined in [RFC7015] to the following device and control-plane
dimensions to determine:
* With node id and egressInterface(IE14), on which node which
logical egress interfaces have been contributing to how much
delay.
* With node id and egressPhysicalInterface(253), on which node which
physical egress interfaces have been contributing to how much
delay.
* With ipNextHopIPv4Address(IE15) or ipNextHopIPv6Address(IE62), the
forwarding path to which next-hop IP contributed to how much
delay.
* With mplsTopLabelIPv4Address(IE47) or destinationIPv6Address and
srhActiveSegmentIPv6 from [RFC9487], the forwarding path to which
MPLS top label IPv4 address or IPv6 destination address and SRv6
active segment contributed to how much delay.
* BGP communities are often used for setting a path priority or
service selection. With bgpDestinationExtendedCommunityList(488)
or bgpDestinationCommunityList(485) or
bgpDestinationLargeCommunityList(491) which group of prefixes
accumulated at which node how much delay.
* With destinationIPv4Address(13), destinationTransportPort(11),
protocolIdentifier (4) and sourceIPv4Address(IE8), the forwarding
path delay on each node from each IPv4 source address to a
specific application in the network.
Taking figure 1 from section 1 as topology example. Below example
table shows the aggregated delay per each node, ingressInterface,
egressInterface, destinationIPv6Address and srhActiveSegmentIPv6.
Graf, et al. Expires 17 July 2024 [Page 14]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
+-----------+-----------+------+-------------+-------------+------------+
| ingress | egress | Node | destination | srhActive | Path Delay |
| Interface | Interface | | IPv6Address | SegmentIPv6 | |
+-----------+-----------+------+-------------+-------------+------------+
| 271 | 276 | R1 | 2001:db8::2 | 2001:db8::4 | 0 us |
+-----------+-----------+------+-------------+-------------+------------+
| 301 | 312 | R2 | 2001:db8::3 | 2001:db8::4 | 22 us |
+-----------+-----------+------+-------------+-------------+------------+
| 22 | 27 | R3 | 2001:db8::4 | 2001:db8::4 | 42 us |
+-----------+-----------+------+-------------+-------------+------------+
| 852 | 854 | R4 | 2001:db8::4 | 2001:db8::4 | 122 us |
+-----------+-----------+------+-------------+-------------+------------+
Table 2: Example table of measured delay. Ascending by delay.
6. IANA Considerations
6.1. Performance Metrics
This document requests IANA to create new performance metrics under
the "Performance Metrics" registry [RFC8911] with the values defined
in section 2.
6.2. IPFIX Entities
This document requests IANA to create new IPFIX IEs (see table 3)
under the "IPFIX Information Elements" registry [RFC7012] available
at "IANA Performance Metric Registry [IANA-PERF-METRIC] and assign
the following initial code points.
+-------+--------------------------------+
|Element| Name |
| ID | |
+-------+--------------------------------+
| TBD5 | PathDelayMeanDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD6 | PathDelayMinDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD7 | PathDelayMaxDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD8 | PathDelaySumDeltaMicroseconds |
| | |
+-------+--------------------------------+
Table 3: Creates IPFIX IEs in the "IPFIX Information Elements" registry
Graf, et al. Expires 17 July 2024 [Page 15]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Note to the RFC-Editor:
* Please replace TBD5 - TBD8 with the values allocated by IANA
* Please replace the [RFC-to-be] with the RFC number assigned to
this document
6.2.1. PathDelayMeanDeltaMicroseconds
Name: PathDelayMeanDeltaMicroseconds
ElementID: TBD5
Description: This Information Element identifies the mean path delay
between the IOAM encapsulation node and the local node with the
IOAM domain (either an IOAM transit node or an IOAM decapsulation
node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Mean in the IANA Performance Metric Registry.
6.2.2. PathDelayMinDeltaMicroseconds
Name: PathDelayMinDeltaMicroseconds
ElementID: TBD6
Description: This Information Element identifies the lowest path
delay between the IOAM encapsulation node and the local node with
the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node) in microseconds, according to the
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Min in the IANA Performance Metric Registry.
Graf, et al. Expires 17 July 2024 [Page 16]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
6.2.3. PathDelayMaxDeltaMicroseconds
Name: PathDelayMaxDeltaMicroseconds
ElementID: TBD7
Description: This Information Element identifies the highest path
delay between the IOAM encapsulation node and the local node with
the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Max in the IANA Performance Metric Registry.
6.2.4. PathDelaySumDeltaMicroseconds
Name: PathDelaySumDeltaMicroseconds
ElementID: TBD8
Description: This Information Element identifies the sum of the path
delay between the IOAM encapsulation node and the local node with
the IOAM domain (either an IOAM transit node or an IOAM
decapsulation node) in microseconds, according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Sum in the IANA Performance Metric Registry.
7. Operational Considerations
7.1. Time Accuracy
The same recommendation as defined in section 4.5 of [RFC5153] for
IPFIX applies in terms of clock precision to this document as well.
Graf, et al. Expires 17 July 2024 [Page 17]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
7.2. Mean Delay
The mean (average) path delay can be calculated by dividing the
PathDelaySumDeltaMicroseconds(TBD5) by the packetDeltaCount(2) at the
IPFIX data collection in order to offload the IPFIX Exporter from
calculating the mean for every Flow at export time.
7.3. Reduced-size encoding
Unsigned64 has been chosen as type for PathDelaySumDeltaMicroseconds
to support cases with large delay numbers and where many packets are
being accounted. As an example, a specific flow record with path
delay of 100 microseconds can not observe more than 42949 packets
without overflowing the unsigned32 counter. The procedure described
in Section 6.2 of [RFC7011] could be applied to reduce network
bandwidth between the IPFIX Exporter and Collector if unsigned32
would be large enough without wrapping around.
7.4. IOAM Application
This document is applicable in IOAM to the Edge-to-Edge and Direct
Exporting Option-Type.
In case of Edge-to-Edge Option-Type, as described in Section 4.6 of
[RFC9197], by setting bits 2 and 3, timestamps can be encoded as
defined in Section 4.4.2.3 and 4.4.2.4 of [RFC9197].
In case of Direct Exporting Option-Type, as described in Section 2 of
[I-D.ahuang-ippm-dex-timestamp-ext], by setting Extension-Flags 2 and
3, timestamps can be encoded as defined in Section 4.4.2.3 and
4.4.2.4 of [RFC9197].
For the Enhanced Alternate Marking Method, Section 2 of
[I-D.zhou-ippm-enhanced-alternate-marking] defines that within the
metaInfo a nano second timestamp can be encoded in the encapsulation
node and be read at the intermediate and decapsulation node to
calculate the on-path delay. [RFC9343] defines how this can be
appied to the IPv6 data-plane and [I-D.fz-spring-srv6-alt-mark]
defines how this can be appied to the Segment Routing Header in SRv6.
8. Security Considerations
There are no significant extra security considerations regarding the
allocation of these new IPFIX IEs compared to [RFC7012].
Graf, et al. Expires 17 July 2024 [Page 18]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
9. Implementation Status
Note to the RFC-Editor: Please remove this section before publishing.
9.1. FD.io VPP
INSA Lyon implemented the following IEs as part of a prototype in the
FD.io VPP (Vector Packet Processing) platform:
* PathDelayMeanDeltaMicroseconds
* PathDelayMaxDeltaMicroseconds
* PathDelayMinDeltaMicroseconds
* PathDelaySumDeltaMicroseconds
The open source code can be obtained here: [INSA-Lyon-VPP] and was
validated at the IETF 116 hackathon.
9.2. Huawei VRP
Huawei implemented the following IEs as part of a a production
implementation in the VRP platform:
* PathDelayMeanDeltaMicroseconds
* PathDelayMaxDeltaMicroseconds
* PathDelayMinDeltaMicroseconds
* PathDelaySumDeltaMicroseconds
The implementation was validated at the IETF 116 hackathon.
9.3. Fluvia
NTT Com implemented the following IEs in the Fluvia Exporter:
* PathDelayMeanDeltaMicroseconds
* PathDelayMaxDeltaMicroseconds
* PathDelayMinDeltaMicroseconds
* PathDelaySumDeltaMicroseconds
Graf, et al. Expires 17 July 2024 [Page 19]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
The open source code can be obtained here: [NTT-Fluvia] and was
validated at the IETF 118 hackathon.
9.4. Pmacct Data Collection
Paolo Lucente implemented the IE PathDelayMeanDeltaMicroseconds by
dividing IE PathDelaySumDeltaMicroseconds by IE packetDeltaCount in
the open source Network Telemetry data collection project pmacct.
The source code can be obtained here: [Paolo-Lucente-Pmacct] and was
validated at the IETF 116 hackathon.
10. Acknowledgements
The authors would like to thank Al Morton, Greg Mirsky and Giuseppe
Fioccola for their review and valuable comments.
11. References
11.1. Normative References
[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model
for IP Flow Information Export (IPFIX)", RFC 7012,
DOI 10.17487/RFC7012, September 2013,
<https://www.rfc-editor.org/info/rfc7012>.
[RFC8911] Bagnulo, M., Claise, B., Eardley, P., Morton, A., and A.
Akhter, "Registry for Performance Metrics", RFC 8911,
DOI 10.17487/RFC8911, November 2021,
<https://www.rfc-editor.org/info/rfc8911>.
11.2. Informative References
[I-D.ahuang-ippm-dex-timestamp-ext]
Feng, A. H., Francois, P., Claise, B., and T. Graf,
"Timestamp extension for In Situ Operations,
Administration, and Maintenance (IOAM) Direct Export",
Work in Progress, Internet-Draft, draft-ahuang-ippm-dex-
timestamp-ext-00, 15 February 2023,
<https://datatracker.ietf.org/doc/html/draft-ahuang-ippm-
dex-timestamp-ext-00>.
Graf, et al. Expires 17 July 2024 [Page 20]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
[I-D.fz-spring-srv6-alt-mark]
Fioccola, G., Zhou, T., Cociglio, M., Mishra, G. S., wang,
X., and G. Zhang, "Application of the Alternate Marking
Method to the Segment Routing Header", Work in Progress,
Internet-Draft, draft-fz-spring-srv6-alt-mark-07, 22
September 2023, <https://datatracker.ietf.org/doc/html/
draft-fz-spring-srv6-alt-mark-07>.
[I-D.ietf-ippm-alt-mark-deployment]
Fioccola, G., Zhou, T., Graf, T., Nilo, M., and L. Zhang,
"Alternate Marking Deployment Framework", Work in
Progress, Internet-Draft, draft-ietf-ippm-alt-mark-
deployment-00, 3 January 2024,
<https://datatracker.ietf.org/doc/html/draft-ietf-ippm-
alt-mark-deployment-00>.
[I-D.ietf-ippm-ioam-deployment]
Brockners, F., Bhandari, S., Bernier, D., and T. Mizrahi,
"In-situ OAM Deployment", Work in Progress, Internet-
Draft, draft-ietf-ippm-ioam-deployment-05, 4 January 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-ippm-
ioam-deployment-05>.
[I-D.song-ippm-postcard-based-telemetry]
Song, H., Mirsky, G., Zhou, T., Li, Z., Graf, T., Mishra,
G. S., Shin, J., and K. Lee, "On-Path Telemetry using
Packet Marking to Trigger Dedicated OAM Packets", Work in
Progress, Internet-Draft, draft-song-ippm-postcard-based-
telemetry-16, 2 June 2023,
<https://datatracker.ietf.org/doc/html/draft-song-ippm-
postcard-based-telemetry-16>.
[I-D.zhou-ippm-enhanced-alternate-marking]
Zhou, T., Fioccola, G., Liu, Y., Cociglio, M., Pang, R.,
Xiong, L., Lee, S., and W. Li, "Enhanced Alternate Marking
Method", Work in Progress, Internet-Draft, draft-zhou-
ippm-enhanced-alternate-marking-14, 23 November 2023,
<https://datatracker.ietf.org/doc/html/draft-zhou-ippm-
enhanced-alternate-marking-14>.
[IANA-PERF-METRIC]
"IANA Performance Metric Registry",
<https://www.iana.org/assignments/performance-metrics/
performance-metrics.xhtml>.
Graf, et al. Expires 17 July 2024 [Page 21]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
[INSA-Lyon-VPP]
"INSA Lyon, FD.io VPP implementation",
<https://github.com/network-analytics/vpp-srh-onpath-
telemetry>.
[NTT-Fluvia]
"NTT Com, Fluvia Exporter",
<https://github.com/nttcom/fluvia/>.
[Paolo-Lucente-Pmacct]
"Paolo Lucente, Pmacct open source Network Telemetry Data
Collection", <https://github.com/pmacct/pmacct>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>.
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393,
DOI 10.17487/RFC3393, November 2002,
<https://www.rfc-editor.org/info/rfc3393>.
[RFC5153] Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P.
Aitken, "IP Flow Information Export (IPFIX) Implementation
Guidelines", RFC 5153, DOI 10.17487/RFC5153, April 2008,
<https://www.rfc-editor.org/info/rfc5153>.
[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
<https://www.rfc-editor.org/info/rfc5905>.
[RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for
the Network Configuration Protocol (NETCONF)", RFC 6020,
DOI 10.17487/RFC6020, October 2010,
<https://www.rfc-editor.org/info/rfc6020>.
[RFC6049] Morton, A. and E. Stephan, "Spatial Composition of
Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011,
<https://www.rfc-editor.org/info/rfc6049>.
Graf, et al. Expires 17 July 2024 [Page 22]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
[RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
IP Network Performance Metrics: Different Points of View",
RFC 6703, DOI 10.17487/RFC6703, August 2012,
<https://www.rfc-editor.org/info/rfc6703>.
[RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types",
RFC 6991, DOI 10.17487/RFC6991, July 2013,
<https://www.rfc-editor.org/info/rfc6991>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[RFC7015] Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation
for the IP Flow Information Export (IPFIX) Protocol",
RFC 7015, DOI 10.17487/RFC7015, September 2013,
<https://www.rfc-editor.org/info/rfc7015>.
[RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Scheffenegger, Ed., "TCP Extensions for High Performance",
RFC 7323, DOI 10.17487/RFC7323, September 2014,
<https://www.rfc-editor.org/info/rfc7323>.
[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Delay Metric for IP Performance Metrics
(IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
2016, <https://www.rfc-editor.org/info/rfc7679>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi,
Ed., "Data Fields for In Situ Operations, Administration,
and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197,
May 2022, <https://www.rfc-editor.org/info/rfc9197>.
[RFC9232] Song, H., Qin, F., Martinez-Julia, P., Ciavaglia, L., and
A. Wang, "Network Telemetry Framework", RFC 9232,
DOI 10.17487/RFC9232, May 2022,
<https://www.rfc-editor.org/info/rfc9232>.
[RFC9343] Fioccola, G., Zhou, T., Cociglio, M., Qin, F., and R.
Pang, "IPv6 Application of the Alternate-Marking Method",
RFC 9343, DOI 10.17487/RFC9343, December 2022,
<https://www.rfc-editor.org/info/rfc9343>.
Graf, et al. Expires 17 July 2024 [Page 23]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
[RFC9487] Graf, T., Claise, B., and P. Francois, "Export of Segment
Routing over IPv6 Information in IP Flow Information
Export (IPFIX)", RFC 9487, DOI 10.17487/RFC9487, November
2023, <https://www.rfc-editor.org/info/rfc9487>.
Appendix A. IPFIX Encoding Examples
This appendix represents two different encodings for the newly
introduced IEs. Taking figure 1 from section 1 as topology example.
Below example Table 4 shows the aggregated delay with
ingressInterface, egressInterface, destinationIPv6Address and
srhActiveSegmentIPv6.
+------ +------+-----------+-----------+------+---------+---------+---------+---------+
|ingress|egress|destination|srhActive |packet|PathDelay|PathDelay|PathDelta|PathDelta|
|Inter |Inter |IPv6Address|SegmentIPv6|Delta |MinDelta |MaxDelta |MeanDelta|MeanDelta|
|face |face | | |Count |Micro.. |Micro.. |Micro.. |Micro.. |
+-------+------+-----------+-----------+------+---------+---------+---------+---------+
| 271 | 276 |2001:db8::4|2001:db8::2| 5 | 22 us | 74 us | 36 us | 180 us |
+-------+------+-----------+-----------+------+---------+---------+---------+---------+
Table 4: Aggregated delay with egressInterface and srhActiveSegmentIPv6
A.1. Aggregated On-Path Dealay Examples
A.1.1. Template Record and Data Set with Mean Delta
With encoding in Figure 1, the mean (average) path delay is
calculated on the exporting node.
* Ingress interface => ingressInterface
* Egress interface => egressInterface
* IPv6 destination address => destinationIPv6Address
* Active SRv6 Segment => srhIPv6ActiveSegment
* Packet Delta Count => packetDeltaCount
* Minimum One-Way Delay => PathDelayMinDeltaMicroseconds (TBD6)
* Maximum One-Way Delay => PathDelayMaxDeltaMicroseconds (TBD7)
* Mean One-Way Delay => PathDelayMeanDeltaMicroseconds (TBD5)
Graf, et al. Expires 17 July 2024 [Page 24]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 2 | Length = 40 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| ingressInterface = 10 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| egressInterface = 14 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv6Address = 28 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| srhIPv6ActiveSegment = 495 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelayMinDelta.. = TBD6 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelayMaxDelta.. = TBD7 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelayMeanDelta.. = TBD5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Template Record for PathDelayMeanDeltaMicroseconds
The data set is represented as follows:
Graf, et al. Expires 17 July 2024 [Page 25]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 256 | Length = 60 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ingressInterface = 271 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| egressInterface = 276 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv6Address = |
| ... |
| ... |
| 2001:db8::2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| srhIPv6ActiveSegment = ... |
| ... |
| ... |
| 2001:db8::4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| packetDeltaCount = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelayMinDeltaMicroseconds = 22 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelayMaxDeltaMicroseconds = 74 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelayMeanDeltaMicroseconds = 36 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Data Set Encoding for PathDelayMeanDeltaMicroseconds
A.1.2. Template Record and Data Set with Sum Delta
With encoding in Figure 3, the mean (average) path delay is
calculated on the IPFIX data collection.
* Ingress interface => ingressInterface
* Egress interface => egressInterface
* IPv6 destination address => destinationIPv6Address
* Active SRv6 Segment => srhIPv6ActiveSegment
* Packet Delta Count => packetDeltaCount
* Minimum One-Way Delay => PathDelayMinDeltaMicroseconds (TBD6)
Graf, et al. Expires 17 July 2024 [Page 26]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
* Maximum One-Way Delay => PathDelayMaxDeltaMicroseconds (TBD7)
* Sum of One-Way Delay => PathDelaySumDeltaMicroseconds (TBD8)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 2 | Length = 40 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 257 | Field Count = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| ingressInterface = 10 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| egressInterface = 14 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv6Address = 28 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| srhIPv6ActiveSegment = 495 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelayMinDelta.. = TBD6 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelayMaxDelta.. = TBD7 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| PathDelaySumDelta.. = TBD8 | Field Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: Template Record for PathDelaySumDeltaMicroseconds
The data set is represented as follows:
Graf, et al. Expires 17 July 2024 [Page 27]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 257 | Length = 60 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ingressInterface = 271 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| egressInterface = 276 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv6Address = |
| ... |
| ... |
| 2001:db8::2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| srhIPv6ActiveSegment = ... |
| ... |
| ... |
| 2001:db8::4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| packetDeltaCount = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelayMinDeltaMicroseconds = 22 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelayMaxDeltaMicroseconds = 74 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PathDelaySumDeltaMicroseconds = 180 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Data Set Encoding for PathDelaySumDeltaMicroseconds
Authors' Addresses
Thomas Graf
Swisscom
Binzring 17
CH-8045 Zurich
Switzerland
Email: thomas.graf@swisscom.com
Benoit Claise
Huawei
Email: benoit.claise@huawei.com
Graf, et al. Expires 17 July 2024 [Page 28]
Internet-Draft Export of On-Path Delay in IPFIX January 2024
Alex Huang Feng
INSA-Lyon
Lyon
France
Email: alex.huang-feng@insa-lyon.fr
Graf, et al. Expires 17 July 2024 [Page 29]
