Initial Performance Metric Registry Entries Part 2: MBM
draft-morton-ippm-mbm-registry-01

Abstract

This memo defines a Registry Entry for the Performance Metrics Registry based on Model Based Metrics. This entry will be combined with the "initial-registry" draft after review.

The string "@@@@" identifies some areas for further discussion to finalize the text.

Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on September 14, 2017.

Copyright Notice

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Introduction
• 2. Scope
• 3. MBM Registry Entry
• 3.1. Summary
• 3.1.1. ID (Identifier)
• 3.1.2. Name
• 3.1.3. URIs
• 3.1.4. Description
• 3.1.5. Reference
• 3.1.6. Change Controller
• 3.1.7. Version (of Registry Format)
• 3.2. Metric Definition
• 3.2.1. Reference Definition
• 3.2.2. Fixed Parameters
• 3.3. Method of Measurement
• 3.3.1. Reference Method
• 3.3.2. Packet Stream Generation
• 3.3.3. Traffic Filtering (observation) Details
• 3.3.4. Sampling Distribution
• 3.3.5. Run-time Parameters and Data Format
• 3.3.6. Roles
• 3.4. Output
• 3.4.1. Type
• 3.4.2. Reference Definition
• 3.4.3. Metric Units
• 3.4.4. Calibration
• 3.5. Administrative items
• 3.5.1. Status
• 3.5.2. Requestor
• 3.5.3. Revision
• 3.5.4. Revision Date
• 3.6. Comments and Remarks
• 4. ver08 BLANK Registry Entry
• 4.1. Summary
• 4.1.1. ID (Identifier)
• 4.1.2. Name
• 4.1.3. URIs
• 4.1.4. Description
• 4.1.5. Reference
• 4.1.6. Change Controller
• 4.1.7. Version (of Registry Format)
• 4.2. Metric Definition
• 4.2.1. Reference Definition
• 4.2.2. Fixed Parameters
• 4.3. Method of Measurement
• 4.3.1. Reference Method
• 4.3.2. Packet Stream Generation
• 4.3.3. Traffic Filtering (observation) Details
• 4.3.4. Sampling Distribution
• 4.3.5. Run-time Parameters and Data Format
• 4.3.6. Roles
• 4.4. Output
• 4.4.1. Type
• 4.4.2. Reference Definition
• 4.4.3. Metric Units
• 4.4.4. Calibration
• 4.5. Administrative items
• 4.5.1. Status
• 4.5.2. Requestor
• 4.5.3. Revision
• 4.5.4. Revision Date
• 4.6. Comments and Remarks
• 5. Security Considerations
• 6. IANA Considerations
• 7. Acknowledgements
• 8. References
• 8.1. Normative References
• 8.2. Informative References
• Authors' Addresses

1. Introduction

Note: Efforts to synchronize structure and terminology with [I-D.ietf-ippm-metric-registry] will likely be incomplete until both drafts are stable.

This memo proposes a (set of) entry(ies) for the Performance Metric Registry, based on Model-Based Metrics (MBM). It uses terms and definitions from the IPPM literature, primarily [RFC2330].

2. Scope

This document defines one of the initial set of Performance Metrics Registry entries, for which IETF approval (following development in the IP Performance Metrics (IPPM) Working Group) will satisfy the requirement for Expert Review. Note that all are Active Performance Metrics, which are based on RFCs prepared in the IPPM working group of the IETF, according to their framework [RFC2330] and its updates.

3. MBM Registry Entry

This section gives an initial registry entry for a Model-Based Metric (MBM) Sustained Burst Metric.

3.1. Summary

This category includes multiple indexes to the registry entries, the element ID and metric name.

3.1.1. ID (Identifier)

<insert numeric identifier, an integer>

3.1.2. Name

<insert name according to metric naming convention>

OWMBM_Active_IP-TCP-SustainedBurst_RFCXXXXsecY_Enumerated_PFI

3.1.3. URIs

URI: Prefix urn:ietf:metrics:perf:<name>

URL: http:\\www.iana.org\ ... <name>

3.1.4. Description

TBD.

3.1.5. Reference

<reference to the RFC of spec where the registry entry is defined>

RFCXXXXsecY

3.1.6. Change Controller

<org or person >

IETF

3.1.7. Version (of Registry Format)

<currently 1.0>

1.0

3.2. Metric Definition

This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.

3.2.1. Reference Definition

<Full bibliographic reference to an immutable doc.>

<specific section reference and additional clarifications, if needed>

Mathis, M. and A. Morton, "Model Based Metrics for Bulk Transport Capacity", draft-ietf-ippm-model-based- metrics-10 (work in progress), February 2017.

The primary metrics of measurement are round-trip delay and one-way loss, measured under the conditions described in Section 8.5.1 of [I-D.ietf-ippm-model-based-metrics].

For loss:

Almes, G., Kalidini, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Loss Metric for IP Performance Metrics (IPPM)", RFC 7680, DOI 10.17487/RFC7680, January 2016, <http://www.rfc-editor.org/info/rfc7680>.

Section 2.4 of [RFC7680] provides the reference definition of the singleton (single value) one-way loss metric. Section 3.4 of [RFC7680] provides the reference definition expanded to cover a multi-singleton sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].

For round-trip delay:

Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999.

[RFC2681]

Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 2016, <http://www.rfc-editor.org/info/rfc7679>.

[RFC7679]

<specific section reference and additional clarifications, if needed>

Section 2.4 of [RFC2681] provides the reference definition of the singleton (single value) Round-trip delay metric. Section 3.4 of [RFC2681] provides the reference definition expanded to cover a multi-singleton sample. Note that terms such as singleton and sample are defined in Section 11 of [RFC2330].

Note that although the definition of "Round-trip-Delay between Src and Dst" is directionally ambiguous in the text, this metric tightens the definition further to recognize that the host in the "Src" role will send the first packet to "Dst", and ultimately receive the corresponding return packet from "Dst" (when neither are lost).

Finally, note that the variable "dT" is used in [RFC2681] to refer to the value of Round-trip delay in metric definitions and methods. The variable "dT" has been re-used in other IPPM literature to refer to different quantities, and cannot be used as a global variable name here.

3.2.2. Fixed Parameters

<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>

Type-P as defined in Section 13 of [RFC2330]:

• IPv4 header values:
  • DSCP: set to 0
  • TTL: set to 255
  • Protocol: Set to 6 (TCP)
• IPv6 header values:
  • DSCP: set to 0
  • Hop Count: set to 255
  • Protocol: Set to 6 (TCP)
• TCP header values:
  • Checksum: the checksum MUST be calculated and included in the header
• TCP Payload
  • see target_MTU in Run-time parameters.

Other measurement parameters:

• Tmax: a loss threshold waiting time @@@@ Should this be linked to TCP RTO, or target_RTT plus a factor ???
  • 3.0, expressed in units of seconds, as a positive value of type decimal64 with fraction digits = 4 (see section 9.3 of [RFC6020]) and with resolution of 0.0001 seconds (0.1 ms), with lossless conversion to/from the 32-bit NTP timestamp as per section 6 of [RFC5905].

3.3. Method of Measurement

This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.

3.3.1. Reference Method

<for metric, insert relevant section references and supplemental info>

The method of measurement is described in Section 8.5.1 of [I-D.ietf-ippm-model-based-metrics].

The example described in Section 9 of [I-D.ietf-ippm-model-based-metrics] may also help.

@@@@<more could be said here about loss and RTT methods>

3.3.2. Packet Stream Generation

<list of generation parameters and section/spec references if needed>

The stream generation parameters are described in Section 3 of [I-D.ietf-ippm-model-based-metrics]. They are dependent on the target parameters described in the Run-time parameters section below. They are written here with underscores because they are used in formulas in this note.

@@@@ I strongly suggest decimal64 fraction digits = 9 or 12 (i.e nanoseconds or picoseconds). I point out that the average headway for min sized packets on 100Gb/s is already down to 5.1 uS. In either case for decimal64 the max headway is way longer than ever needed (decades or months). --MM-- (@@@@ BTW I think your fraction digits are off --MM--).

@@@@ This section has a general problem that it need to better prescribe generic concepts (headway, sizes, rates) and then define multiple distinct parameters needed to properly specify each pattern.

packet_headway

Time interval between packets, specified from the start of one to the start of the next, as a positive value of type decimal64 with fraction digits = 9 seconds, for a resolution of 1 nanosecond. (see section 9.3 of [RFC6020]). @@@@ We need a convention for "back to back" independent of clock accuracy.--MM--

burst_headway

Time interval between bursts, specified from the start of the first packet one burst to the start of the first packet of the next burst, specified as a positive value of type decimal64 with fraction digits = 9 seconds, for a resolution of 1 nanosecond. (see section 9.3 of [RFC6020]).

paced_single_packets

Send individual packets at the specified packet headway, specified as a positive value of type decimal64 with fraction digits = 9 seconds, for a resolution of 1 nanosecond. (see section 9.3 of [RFC6020]). @@@@ NB: I dropped rate.--MM--

paced_bursts

Send bursts on a timer. Specify any 3 of: average data rate, packet size, burst size (number of packets) and burst headway (burst start to start), specified as a positive value of type decimal64 with fraction digits = 9 seconds, for a resolution of 1 nanosecond. (see section 9.3 of [RFC6020]).

slowstart_rate

The average data rate necessary to mimic TCP slowstart by sending 4 packet paced bursts to mimic a two level burst pattern as described in Section 6.1 of [I-D.ietf-ippm-model-based-metrics]. This rate should be chosen to be twice the implied bottleneck IP capacity (but not more than the sender interface rate). The slowstart_rate is specified as a value of type uint32 (see section 9.2 of [RFC6020]) in units of IP-layer bytes per second.

slowstart_burst

Mimic one round of TCP slowstart by sending a specified number of packets in a two level burst pattern that resembles slowstart, specified as a number of type uint16 (see section 9.2 of [RFC6020]) in units of packets, and a rate specified as a value of type uint16 (see section 9.2 of [RFC6020]) in units of packets per second.

repeated_slowstart_burst

Repeat Slowstart bursts once per target_RTT. All Slowstart bursts are the same size in measurements (different from normal TCP sending behavior), specified as a value of type boolean (see section 9.5 of [RFC6020]). @@@@@ I would change this to [slowestart] burst headway, nominally an interval mimicking the RTT and long enough to permit all of the queues to drain between slowstart bursts.

3.3.3. Traffic Filtering (observation) Details

<insert the measured results based on a filtered version of the packets observed, and this section provides the filter details (when present), and section reference>.

NA

3.3.4. Sampling Distribution

<insert time distribution details, or how this is diff from the filter>

NA

3.3.5. Run-time Parameters and Data Format

<list of run-time parameters, and any reference(s)>.

The following parameters are described in [RFC2330]

Src

the IP address of the host in the Src Role (format ipv4-address-no-zone value for IPv4, or ipv6-address-no-zone value for IPv6, see Section 4 of [RFC6991])

Dst

the IP address of the host in the Dst Role (format ipv4-address-no-zone value for IPv4, or ipv6-address-no-zone value for IPv6, see section 4 of [RFC6991])

T0

a time, the start of a measurement interval, (format "date-and-time" as specified in Section 5.6 of [RFC3339], see also Section 3 of [RFC6991]). The UTC Time Zone is required by Section 6.1 of [RFC2330]. When T0 is "all-zeros", a start time is unspecified and Tf is to be interpreted as the Duration of the measurement interval. The start time is controlled through other means.

Tf

a time, the end of a measurement interval, (format "date-and-time" as specified in Section 5.6 of [RFC3339], see also Section 3 of [RFC6991]). The UTC Time Zone is required by Section 6.1 of [RFC2330]. When T0 is "all-zeros", a end time date is ignored and Tf is interpreted as the Duration of the measurement interval.

The following MBM-specific parameters are as defined in Section 3 of [I-D.ietf-ippm-model-based-metrics], and subsequent sections of the memo.

target_data_rate

The specified application data rate required for an application's proper operation, specified as a value of type uint32 (see section 9.2 of [RFC6020]) in units of IP-layer bytes per second.

target_RTT

The specified baseline (minimum) RTT of the longest complete path over which the user expects to be able meet the target performance, specified as a positive value of type decimal64 with fraction digits = 4 (see section 9.3 of [RFC6020]) with resolution of 0.0001 seconds (0.1 ms).

target_MTU

The specified maximum MTU supported by the complete path the over which the application expects to meet the target performancespecified as a value of type uint16 (see section 9.2 of [RFC6020]) in units of IP-layer bytes.

target_window_size

The average number of packets in flight (the window size) needed to meet the Target Data Rate, for the specified Target RTT, and MTU, specified as a value of type uint32 (see section 9.2 of [RFC6020]) in units of @@@@ packets @@@@ or IP-layer bytes @@@@@. It implies the scale of the bursts that the network might experience.

subpath_???

@@@@@ Do we need a subpath-specific parameter? Such as subpath_RTT ???

derating

The modeling framework permits some latitude in relaxing or "derating" some test parameters as described in Section 5.3 of [I-D.ietf-ippm-model-based-metrics], in exchange for a more stringent TIDS validation procedures as described in Section 10 of [I-D.ietf-ippm-model-based-metrics]. The use of derated parameters is specified as a value of type boolean (see section 9.5 of [RFC6020]).

test_window

The smallest window sufficient to meet or exceed the target_rate when operating with a pure self-clock over a test path, specified as a value of type uint32 (see section 9.2 of [RFC6020]) in units of @@@@ packets @@@@ or IP-layer bytes @@@@@.

The following MBM-specific parameters are as defined in Section of 7.2 [I-D.ietf-ippm-model-based-metrics]:

H0H1_ratio

The value of the multiplier on the Null Hypothesis loss ratio used to calculate the Alternate Hypothesis loss ratio, specified as a value of type uint8 (see section 9.2 of [RFC6020]) and unit-less.

alpha_TI_err

Measurements support accepting H0 with the specified Type I error = alpha (= 0.05 for example), specified as a positive value of type decimal64 with fraction digits = 4 (see section 9.3 of [RFC6020]) with resolution of 0.0001.

beta_TII_err

Measurements support accepting H1 with the specified Type II error = beta (= 0.05 for example), specified as a positive value of type decimal64 with fraction digits = 4 (see section 9.3 of [RFC6020]) with resolution of 0.0001.

Additional MBM-specific parameters may be calculated by the measurement system itself, or they may be supplied as additional Run-time parameters: @@@@ Candidates ????

3.3.6. Roles

<lists the names of the different roles from the measurement method>

[I-D.ietf-ippm-model-based-metrics].

data_sender

Host sending data and receiving ACKs.

data_receiver

Host receiving data and sending ACKs.

as described in Section 3 of

3.4. Output

This category specifies all details of the Output of measurements using the metric.

3.4.1. Type

<insert name of the output type, raw or a selected summary statistic>

The primary output type is PFI, or Pass, Fail, Inconclusive, referring to the conclusion of the test.

Two secondary output types MAY be reported to support the primary output.

Loss Ratio: Singleton

Mean Round-trip Time: Singleton

3.4.2. Reference Definition

<pointer to section/spec where output type/format is defined>

T0

the start of a measurement interval, (format "date-and-time" as specified in Section 5.6 of [RFC3339], see also Section 3 of [RFC6991]). The UTC Time Zone is required by Section 6.1 of [RFC2330].

Tf

the end of a measurement interval, (format "date-and-time" as specified in Section 5.6 of [RFC3339], see also Section 3 of [RFC6991]). The UTC Time Zone is required by Section 6.1 of [RFC2330].

PFI

the summarized result of the measurement representing the conclusion of whether or not the target values have been achieved, (format enum as specified in section 9.6 of [RFC6020]) with one of the following enumerations: Pass, Fail, Inconclusive.

Loss_Ratio

the result of lost (or ECN marked) packet measurement from data_sender to data_receiver, expressed as the ratio of lost packets to total packets sent from the data sender (units). See section 4 of [RFC7680] for details on this calculation.

Mean_RTT

Mean Round-trip Time: The mean SHALL be calculated using the conditional distribution of all packets with a finite value of round-trip delay (undefined delays are excluded), a single value as follows:

• See section 4.1 of [RFC3393] for details on the conditional distribution to exclude undefined values of delay, and Section 5 of [RFC6703] for background on this analysis choice.
• See section 4.2.2 of [RFC6049] for details on calculating this statistic, and 4.2.3 of [RFC6049].
• The time value of the result is expressed in units of seconds, as a positive value of type decimal64 with fraction digits = 9 (see section 9.3 of [RFC6020]) with resolution of 0.000000001 seconds (1.0 ns), and with lossless conversion to/from the 64-bit NTP timestamp as per section 6 of RFC [RFC5905].

3.4.3. Metric Units

<insert units for the measured results, and the reference specification>.

PFI: Enumerated{Pass, Fail, Inconclusive}

Loss Ratio: RatioPercent

Mean Round-trip Time: Seconds

3.4.4. Calibration

<describe the error calibration, a way to indicate that the results were collected in a calbration mode of operation, and a way to report internal status metrics related to calibration, such as time offset>

3.5. Administrative items

3.5.1. Status

<current or depricated>

3.5.2. Requestor

<name of individual or Internet Draft, etc.>

3.5.3. Revision

1.0

3.5.4. Revision Date

YYYY-MM-DD

3.6. Comments and Remarks

Additional (Informational) details for this entry

4. ver08 BLANK Registry Entry

This section gives an initial registry entry for ....

4.1. Summary

This category includes multiple indexes to the registry entries, the element ID and metric name.

4.1.1. ID (Identifier)

<insert numeric identifier, an integer>

4.1.2. Name

<insert name according to metric naming convention>

4.1.3. URIs

URI: Prefix urn:ietf:params:performance:metric

URL:

4.1.4. Description

TBD.

4.1.5. Reference

<reference to the RFC of spec where the registry entry is defined>

4.1.6. Change Controller

<org or person >

4.1.7. Version (of Registry Format)

<currently 1.0>

4.2. Metric Definition

This category includes columns to prompt the entry of all necessary details related to the metric definition, including the RFC reference and values of input factors, called fixed parameters.

4.2.1. Reference Definition

<Full bibliographic reference to an immutable doc.>

<specific section reference and additional clarifications, if needed>

4.2.2. Fixed Parameters

<list and specify Fixed Parameters, input factors that must be determined and embedded in the measurement system for use when needed>

4.3. Method of Measurement

This category includes columns for references to relevant sections of the RFC(s) and any supplemental information needed to ensure an unambiguous methods for implementations.

4.3.1. Reference Method

<for metric, insert relevant section references and supplemental info>

4.3.2. Packet Stream Generation

<list of generation parameters and section/spec references if needed>

4.3.3. Traffic Filtering (observation) Details

<insert the measured results based on a filtered version of the packets observed, and this section provides the filter details (when present), and section reference>.

4.3.4. Sampling Distribution

<insert time distribution details, or how this is diff from the filter>

4.3.5. Run-time Parameters and Data Format

<list of run-time parameters, and any reference(s)>.

4.3.6. Roles

<lists the names of the different roles from the measurement method>

4.4. Output

This category specifies all details of the Output of measurements using the metric.

4.4.1. Type

<insert name of the output type, raw or a selected summary statistic>

4.4.2. Reference Definition

<pointer to section/spec where output type/format is defined>

4.4.3. Metric Units

<insert units for the measured results, and the reference specification>.

4.4.4. Calibration

<describe the error calibration, a way to indicate that the results were collected in a calbration mode of operation, and a way to report internal status metrics related to calibration, such as time offset>

4.5. Administrative items

4.5.1. Status

<current or depricated>

4.5.2. Requestor

<name of individual or Internet Draft, etc.>

4.5.3. Revision

1.0

4.5.4. Revision Date

YYYY-MM-DD

4.6. Comments and Remarks

Additional (Informational) details for this entry

5. Security Considerations

These registry entries represent no known security implications for Internet Security. Each referenced Metric contains a Security Considerations section.

6. IANA Considerations

IANA is requested to populate The Performance Metric Registry defined in [I-D.ietf-ippm-metric-registry] with the values defined above.

<more is needed here>

7. Acknowledgements

The authors thank Brian Trammell for suggesting the term "Run-time Parameters", which led to the distinction between run-time and fixed parameters implemented in this memo, for identifying the IPFIX metric with Flow Key as an example, and for many other productive suggestions. Thanks to Peter Koch, who provided several useful suggestions for disambiguating successive DNS Queries in the DNS Response time metric.

The authors also acknowledge the constructive reviews and helpful suggestions from Barbara Stark, Juergen Schoenwaelder, Tim Carey, and participants in the LMAP working group.

8. References

8.1. Normative References

8.2. Informative References

Authors' Addresses