Supporting IOAM in IPv6

In-situ OAM (IOAM) defines two trace options, pre-allocated trace option and incremental trace option, which record hop-by-hop data along a packet's forwarding path. describes the method to encapsulate IOAM pre-allocated trace option data fields in IPv6. Because the trace options requires per hop processing, such options can only be encapsulated in IPv6 Hop-by-Hop (HbH) options header. mandates that the HbH options header, if exists, must be the first extension header following the IPv6 header. However, the IOAM trace data can be large, which can amount to tens to hundreds of bytes, making accessing other headers after it difficult or even impossible in some routers. There are practical limitations on how far the hardware can reach into a packet in forwarding hardware. The IOAM trace option cannot be applied if it makes other extension headers inaccessible. Even if the other headers can be reached, the deeper they are, the higher the cost to access and process them, and the lower the forwarding performance. Note that does not support the incremental trace option because it would expand the HbH header at each hop and push back all other headers after it. The changing location of the later extension headers could further complicate the hardware implementation and affect the forwarding performance. The issue becomes more severe when SRv6 and IOAM coexist. The Segment Routing Extension Header (SRH) is encapsulated in a routing header which is after the HbH options header. SRH itself can be large. It requires read and write operations at each SRv6 segment endpoint node. If it is deeply embedded in a packet and its location keeps shifting, either it is beyond the reach of hardware or the forwarding performance degrades. We can avoid the problem by not using both at the same time, but this is not ideal, because IOAM is an important OAM tool and it is even more wanted when SRv6 brings more operational complexity into IPv6 networks. The second recourse is to limit the IOAM to SRv6 nodes only. That is, consider SRv6 as an overlay tunnel over IPv6 and apply the IOAM pipe mode as discussed in , which only collects data at each SRv6 segment endpoint nodes. To realize this, describes an approach that encapsulates the IOAM option data fields in an SRH TLV. describes another approach to enable postcard-based telemetry for SRv6 without needing IOAM option encapsulation. In either case, the SRH is close to the packet front and its location is fixed. While these approaches are useful for use cases that only need to monitor the segment endpoints, it fails to cover all the IPv6 nodes on the packet forwarding path in an IOAM domain. So the proposition of this draft is, if we need to apply IOAM on all nodes in an SRv6 network, how we can amend the approach in or use alternative approaches to circumvent the aforementioned issues. In this draft, we propose two viable approaches: (1) separating the IOAM trace data from the instruction header to a different extension header option after the routing header if it exists, and (2) applying the segment IOAM trace export scheme. We discuss the pros and cons of each approach.

An IOAM trace type data fields contain two parts: instruction and trace data. Although by convention the trace data part immediately follows the instruction part, there is not fundamental reason why these two parts must stick together. This observation provides us an optimization opportunity to amend the original proposal in . We separate the IOAM trace type data fields into the instruction part and the trace data part. We encapsulate only the instruction part in the HbH options header, and encapsulate the trace data part in another extension header option after all the IPv6 extension headers that need to be examined and processed on the packet forwarding path (e.g., a routing header). This arrangement allows us to use the incremental trace option efficiently. Even if the data trace increases its size at each node, all IPv6 extension headers before it remain a fixed size, and new data is guaranteed to be inserted at a fixed location. Figure 1 shows the HbH option format for IOAM incremental trace type instruction. The field specification is identical to that in and .

Figure 2 shows the TLV option format for IOAM trace type data. The IOAM trace type data format is compliant with .

We have basically two methods to encapsulate the IOAM incremental trace data. First, we can define a new IPv6 extension header which is dedicated to metadata. Once standardized, this extension header can also be used to host potential metadata from other applications such as NSH for SFC . Second, this option can be carried as a TLV option in another existing extension header such as the Destination Option Header (DOH). The only requirement is that this extension header should be the last one in the extension header chain. The first method is cleaner but it requires to standardize a new extension header type; the second method is simpler but it needs to overcome the access constraints exerted by .

If the overhead of the IOAM trace type data fields is under control, we may still manage to encapsulate both instruction and data in HbH options header as described in . To this end, we introduce two sub-approaches.

proposes an enhancement to IOAM trace type which can configure the allowable overhead of the IOAM trace type data fields. Once the trace data size is up to the limit at a network node (i.e., a segment or a fixed number of network nodes are traversed), the trace data will be stripped and exported so room is made to accommodate new trace data from nodes in the next segment of the forwarding path. This approach requires some moderate updates to the IOAM trace type data fields, as described in . Figure 3 shows the format of the HbH Option Header containing Segment IOAM trace type data fields. A flag bit (#23) in the Flags field is used to indicate the current header is a segment IOAM header. In this context, the last octet in the IOAM header is partitioned into two 4-bit nibbles. The first nibble (SSize) is used to save the segment size and the second nibble (RHop) is used to save the remaining hops. This limits the maximum segment size to 15.

At the beginning of each segment, the segment size (SSize) and the remaining hops (RHop) are initialized: RHop is set to equal to SSize. At each hop, if RHop is not zero, the node data is added to the node data list and then RHop is decremented by 1. If RHop is equal to 0 when receiving the packet, the node needs to remove (in incremental trace option) or clear (in pre-allocated trace option) the IOAM node data list and reset RHop to SSize. In this case, if we use the IOAM pre-allocated trace type, the size and location of each IPv6 extension header is fixed and predictable, and the hardware capability and performance can be guaranteed.

Whenever a packet with the IOAM option reaches a SRv6 segment endpoint node which needs to access the SRH, we can configure the node to export immediately the IOAM trace data accumulated so far. In this case, basically at each SRv6 segment endpoint node, after the trace data export, the HbH header size is fixed and the header contains an IOAM option with only the instruction part. After the SRH processing, this node can add local IOAM trace data in the HbH option header before forwarding the packet. The incremental trace type is more proper in this approach than the pre-allocated trace type, due to the uncertainty of the number of hops between two segment endpoints. In an extreme case when every node is also an SRv6 node, this approach regresses to a per-hop postcard-based telemetry approach such as IOAM DEX as described in .

It is worth noting that, instead of using the IOAM trace options, IOAM Direct Export (DEX) Option Type can be used for fixed and small packet overhead since it only needs to encapsulate a fix-size instruction header in the HbH option header. The scheme is covered in .

The following table compares the existing approach and the four other alternative approaches proposed in this draft.

IOAM trace option is easy to implement and extensible for data types. Complementary to , the scalable solutions for using it in IPv6 networks discussed in this document can fully carry out the benefits of IOAM trace option and meanwhile avoid potential issues associated with variable and high header overhead.

No new security issue is identified other than those for IOAM trace option , IOAM DEX , and IPv6 extension headers .

This document requires no IANA actions.