measurement results of both one-way delays of back-to-back connection between two PERFAS+ clients at 64 Bytes

Test metric	PERFAS+
minimum [us]	26
maximum [us]	205
mean [us]	49.1
median [us]	49.5
standard deviation [us]	7.6
2.5% percentile [us]	32.5
97.5% percentile [us]	59.5

7. Summary

By an IPPM measurement system like PERFAS+ three physical measurement clients are needed to measure the round-trip delay between all sites. With the PMS the same measurements can be performed with only one client. In theory one PMS could monitor a whole MPLS-enabled backbone. The GPS receivers of two IPPM measurement agents were not available, hence the one-way delay could not be captured with the IPPM system PERFAS+. Otherwise a direct comparison with calculated one-way delay values based on the PMS measured values would have been possible. This could be done in future. The results shown in Section 4 indicate, that the PMS measurements equal those captured by an IPPM conformant measurement system. The ADK test is successful by comparing the measurement values of the round-trip delays for packets with a size of 64 bytes. The network does not include an impairment generator (which was required within a test set up to compare independent IPPM implementations, see [RFC6808]). An impairment generator as part of the test set up will have a positive effect on the measurements and the measurements with bigger packet size will also succeed at a temporal resolution above [us] level.

8. Acknowledgements

Joachim Mende, Marc Wieland, Ralf Widera and Jens Wyduba helped to implement and operate the LDP PMS in our research network. In memoriam of Holger Zarwel, who gave our project unconditional support.

9. IANA Considerations

This memo includes no request to IANA.

10. Security Considerations

A PMS monitoring packet should never leave the domain where it originated. It therefore should never use stale MPLS or IGP routing information. If the Label Switch Path is broken, a packet with the destination address 127.0.0.0/26 should not be routed, it should be discarded. The PMS must be configured with a measurement interval (or sum of all measurement stream intervals) that does not overload the network. Too many measurement streams with a big packet size could overload a link.

11. References

11.1. Normative References

[RFC2679]	Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679, September 1999.
[RFC4379]	Kompella, K. and G. Swallow, "Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, DOI 10.17487/RFC4379, February 2006.
[RFC6576]	Geib, R., Morton, A., Fardid, R. and A. Steinmitz, "IP Performance Metrics (IPPM) Standard Advancement Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March 2012.
[RFC6808]	Ciavattone, L., Geib, R., Morton, A. and M. Wieser, "Test Plan and Results Supporting Advancement of RFC 2679 on the Standards Track", RFC 6808, DOI 10.17487/RFC6808, December 2012.

11.2. Informative References

[BCP-TX]	NANOG, "Best Practices for Determining Traffic Matrices in IP Networks V 4.0", 2008.
[I-D.ietf-spring-oam-usecase]	Geib, R., Filsfils, C., Pignataro, C. and N. Kumar, "A Scalable and Topology-Aware MPLS Dataplane Monitoring System", Internet-Draft draft-ietf-spring-oam-usecase-03, April 2016.
[LDP-TE]	VDE-Verlag, "Traffic Matrices for MPLS Networks with LDP Traffic Statistics", 2004.

Appendix A. ADK2 Test Source Code

The following C++ source code is a modified version of the Code at [RFC6576]. This version allows to test two files containing values with the ADK2. It is not necessary that the values are sorted, because in the first step the values get sorted.

/*
Copyright (c) 2012 IETF Trust and the persons identified
as authors of the code.  All rights reserved.

Redistribution and use in source and binary forms, with
or without modification, is permitted pursuant to, and subject
to the license terms contained in, the Simplified BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents (http://trustee.ietf.org/license-info).
*/

/* Routines for computing the Anderson-Darling 2 sample
* test statistic.
*
* Implemented based on the description in
* "Anderson-Darling K Sample Test" Heckert, Alan and
* Filliben, James, editors, Dataplot Reference Manual,
* Chapter 15 Auxiliary, NIST, 2004.
* Official Reference by 2010
* Heckert, N. A. (2001).  Dataplot website at the
* National Institute of Standards and Technology:
* http://www.itl.nist.gov/div898/software/dataplot.html/
* June 2001.
*/

// this code is a modified version of the code in RFC6576

// use '-std=c++11' for compiling

#include <iostream>
#include <fstream>
#include <vector>
#include <sstream>
#include <iterator>

#include <algorithm>


using namespace std;

/* This function reads the values and sorts this in an ascending 
* order.
* The format is: one value per line followed by a line break.
* A blank line at the end of the file will crash the program.
*/
vector<double> read_file_sort (string filename) {
    vector<double> vec;
    // variable for one line of the file and the value
    string line;
    double tmp;
    
    ifstream file;
    file.open(filename, ios::in);
    if (!file) {
        cout << "Error in file " << filename << endl;
    }
    else {
        // read file in a vector
        while(!file.eof()) {
            getline (file, line);
            tmp = stod (line);
            vec.push_back(tmp);
        }
        // sort the vector ascending
        sort(vec.begin(), vec.end());
    }
    file.close();
    return vec;
}

 int main(int argn, char *argv[]) {
    
    if (argn != 1 && argn != 3) {
        cout << "wrong invocation" << endl;
        cout << "start with " << argv[0] << " file1 file2" << endl;
        cout << "start with " << argv[0] << " without parameter, if \
        the files are named file1.csv and file2.csv" << endl;
        return 1;
    }

    vector<double> vec1, vec2;
    double adk_result;
    static int k, val_st_z_samp1, val_st_z_samp2,
               val_eq_z_samp1, val_eq_z_samp2,
               j, n_total, n_sample1, n_sample2, L,
               max_number_samples, line, maxnumber_z;
    static int column_1, column_2;
    static double adk, n_value, z, sum_adk_samp1,
                  sum_adk_samp2, z_aux;
    static double H_j, F1j, hj, F2j, denom_1_aux, denom_2_aux;
    static bool next_z_sample2, equal_z_both_samples;
    static int stop_loop1, stop_loop2, stop_loop3,old_eq_line2,
               old_eq_line1;

    static double adk_criterium = 1.993;
    
    string filename1 = "file1.csv";
    string filename2 = "file2.csv";

    // if called with filenames
    if (argn == 3) {
        filename1 = argv[1];
        filename2 = argv[2];
    }
    
    // sort the two files i a vector
    vec1 = read_file_sort(filename1);
    vec2 = read_file_sort(filename2);  

    k = 2;
    n_sample1 = vec1.size() - 1;
    n_sample2 = vec2.size() - 1;

    // -1 because vec[0] is a dummy value
    n_total = n_sample1 + n_sample2;

    /* value equal to the line with a value = zj in sample 1.
     * Here j=1, so the line is 1.
     */
    val_eq_z_samp1 = 1;

    /* value equal to the line with a value = zj in sample 2.
     * Here j=1, so the line is 1.
     */
    val_eq_z_samp2 = 1;

    /* value equal to the last line with a value < zj
     * in sample 1.  Here j=1, so the line is 0.
     */
    val_st_z_samp1 = 0;

    /* value equal to the last line with a value < zj
     * in sample 1.  Here j=1, so the line is 0.
     */
    val_st_z_samp2 = 0;

    sum_adk_samp1 = 0;
    sum_adk_samp2 = 0;
    j = 1;

    // as mentioned above, j=1
    equal_z_both_samples = false;

    next_z_sample2 = false;

    // assuming the next z to be of sample 1
    stop_loop1 = n_sample1 + 1;

    // + 1 because vec[0] is a dummy, see n_sample1 declaration
    stop_loop2 = n_sample2 + 1;
    stop_loop3 = n_total + 1;

    /* The required z values are calculated until all values
     * of both samples have been taken into account.  See the
     * lines above for the stoploop values.  Construct required
     * to avoid a mathematical operation in the while condition.
    */
    while (((stop_loop1 > val_eq_z_samp1)
    
           || (stop_loop2 > val_eq_z_samp2)) && stop_loop3 > j) {
        if (val_eq_z_samp1 < n_sample1+1) {
            /* here, a preliminary zj value is set.
             * See below how to calculate the actual zj.
             */
            z = vec1[val_eq_z_samp1];

            /* this while sequence calculates the number of values
             * equal to z.
             */
            while ((val_eq_z_samp1+1 < n_sample1)
                && z == vec1[val_eq_z_samp1+1] ) {
                val_eq_z_samp1++;
                }
        }
        else {
        val_eq_z_samp1 = 0;
        val_st_z_samp1 = n_sample1;

        // this should be val_eq_z_samp1 - 1 = n_sample1
        }

        if (val_eq_z_samp2 < n_sample2+1) {
            z_aux = vec2[val_eq_z_samp2];

            /* this while sequence calculates the number of values
             * equal to z_aux
             */

            while ((val_eq_z_samp2+1 < n_sample2)
                    && z_aux == vec2[val_eq_z_samp2+1] ) {
                val_eq_z_samp2++;
            }

            /* the smaller of the two actual data values is picked
             * as the next zj.
             */

            if(z > z_aux) {
                z = z_aux;
                next_z_sample2 = true;
            }
            else {
                if (z == z_aux) {
                equal_z_both_samples = true;
                }

                /* This is the case if the last value of column1 is
                 * smaller than the remaining values of column2.
                 */
               if (val_eq_z_samp1 == 0) {
                z = z_aux;
                next_z_sample2 = true;
                }
            }
        }
        else {
            val_eq_z_samp2 = 0;
            val_st_z_samp2 = n_sample2;

            // this should be val_eq_z_samp2 - 1 = n_sample2
        }

         /* in the following, sum j = 1 to L is calculated for
          * sample 1 and sample 2.
          */
        if (equal_z_both_samples) {

            /* hj is the number of values in the combined sample
             * equal to zj
             */
            hj = val_eq_z_samp1 - val_st_z_samp1
              + val_eq_z_samp2 - val_st_z_samp2;

            /* H_j is the number of values in the combined sample
             * smaller than zj plus one half the number of
             * values in the combined sample equal to zj
             * (that's hj/2).
             */
            H_j = val_st_z_samp1 + val_st_z_samp2 + hj / 2;

            /* F1j is the number of values in the 1st sample
             * that are less than zj plus one half the number
             * of values in this sample that are equal to zj.
             */

            F1j = val_st_z_samp1 + (double)
                  (val_eq_z_samp1 - val_st_z_samp1) / 2;

            /* F2j is the number of values in the 1st sample
             * that are less than zj plus one half the number
             * of values in this sample that are equal to zj.
             */
            F2j = val_st_z_samp2 + (double)
                 (val_eq_z_samp2 - val_st_z_samp2) / 2;

            /* set the line of values equal to zj to the
             * actual line of the last value picked for zj.
             */
            val_st_z_samp1 = val_eq_z_samp1;

            /* Set the line of values equal to zj to the actual
             * line of the last value picked for zj of each
             * sample.  This is required as data smaller than zj
             * is accounted differently than values equal to zj.
             */
            val_st_z_samp2 = val_eq_z_samp2;

            /* next the lines of the next values z, i.e., zj+1
             * are addressed.
             */
            val_eq_z_samp1++;

            /* next the lines of the next values z, i.e.,
             * zj+1 are addressed
             */
            val_eq_z_samp2++;
        }
        else {
            
            /* the smaller z value was contained in sample 2;
             * hence, this value is the zj to base the following
             * calculations on.
             */
            if (next_z_sample2){
                /* hj is the number of values in the combined
                 * sample equal to zj; in this case, these are
                 * within sample 2 only.
                 */
                hj = val_eq_z_samp2 - val_st_z_samp2;

                /* H_j is the number of values in the combined sample
                 * smaller than zj plus one half the number of
                 * values in the combined sample equal to zj
                 * (that's hj/2).
                 */
                H_j = val_st_z_samp1 + val_st_z_samp2 + hj / 2;

              /* F1j is the number of values in the 1st sample that
             * are less than zj plus one half the number of values in
               * this sample that are equal to zj.
               * As val_eq_z_samp2 < val_eq_z_samp1, these are the
               * val_st_z_samp1 only.
               */
                F1j = val_st_z_samp1;

              /* F2j is the number of values in the 1st sample that
             * are less than zj plus one half the number of values in
             * this sample that are equal to zj.  The latter are from
               * sample 2 only in this case.
               */

                F2j = val_st_z_samp2 + (double)
                     (val_eq_z_samp2 - val_st_z_samp2) / 2;

             /* Set the line of values equal to zj to the actual line
               * of the last value picked for zj of sample 2 only in
               * this case.
               */
               val_st_z_samp2 = val_eq_z_samp2;

              /* next the line of the next value z, i.e., zj+1 is
               * addressed.  Here, only sample 2 must be addressed.
               */

                val_eq_z_samp2++;
                if (val_eq_z_samp1 == 0) {
                    val_eq_z_samp1 = stop_loop1;
                }
            }
           /* the smaller z value was contained in sample 2;
            * hence, this value is the zj to base the following
            * calculations on.
            */

            else {

                /* hj is the number of values in the combined
                 * sample equal to zj; in this case, these are
                 * within sample 1 only.
                 */
                hj = val_eq_z_samp1 - val_st_z_samp1;

                /* H_j is the number of values in the combined
                 * sample smaller than zj plus one half the number
                 * of values in the combined sample equal to zj
                 * (that's hj/2).
                 */

                H_j = val_st_z_samp1 + val_st_z_samp2 + hj / 2;

            /* F1j is the number of values in the 1st sample that
             * are less than zj plus; in this case, these are within
             * sample 1 only one half the number of values in this
             * sample that are equal to zj.  The latter are from
             * sample 1 only in this case.
             */

                F1j = val_st_z_samp1 + (double)
                    (val_eq_z_samp1 - val_st_z_samp1) / 2;

            /* F2j is the number of values in the 1st sample that
             * are less than zj plus one half the number of values
             * in this sample that are equal to zj.  As
             * val_eq_z_samp1 < val_eq_z_samp2, these are the
             * val_st_z_samp2 only.
             */

                F2j = val_st_z_samp2;

            /* Set the line of values equal to zj to the actual line
             * of the last value picked for zj of sample 1 only in
             * this case.
             */

                val_st_z_samp1 = val_eq_z_samp1;
                /* next the line of the next value z, i.e., zj+1 is
                 * addressed.  Here, only sample 1 must be addressed.
                 */
                val_eq_z_samp1++;

                if (val_eq_z_samp2 == 0) {
                    val_eq_z_samp2 = stop_loop2;
                }
            }
        }

        denom_1_aux = n_total * F1j - n_sample1 * H_j;
        denom_2_aux = n_total * F2j - n_sample2 * H_j;

        sum_adk_samp1 = sum_adk_samp1 + hj
                        * (denom_1_aux * denom_1_aux) /
                        (H_j * (n_total - H_j)
                        - n_total * hj / 4);
        sum_adk_samp2 = sum_adk_samp2 + hj
                        * (denom_2_aux * denom_2_aux) /
                        (H_j * (n_total - H_j)
                        - n_total * hj / 4);

        next_z_sample2 = false;
        equal_z_both_samples = false;

        /* index to count the z.  It is only required to prevent
         * the while slope to execute endless
         */
        j++;
    }

    // calculating the adk value is the final step.
    adk_result = (double) (n_total - 1) / (n_total
           * n_total * (k - 1))
            * (sum_adk_samp1 / n_sample1
            + sum_adk_samp2 / n_sample2);


    /* if(adk_result <= adk_criterium)
     * adk_2_sample test is passed
     */
    //return adk_result <= adk_criterium;
    cout << "Result: " << adk_result << endl;
}

         

Authors' Addresses