Internet DRAFT - draft-elkins-ippm-6man-pdm-option
draft-elkins-ippm-6man-pdm-option
INTERNET-DRAFT N. Elkins
Inside Products
R. Hamilton
Chemical Abstracts Service
M. Ackermann
Intended Status: Proposed Standard BCBS Michigan
Expires: October 2, 2015 March 31, 2015
IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
draft-elkins-ippm-6man-pdm-option-00
Table of Contents
1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 End User Quality of Service (QoS) . . . . . . . . . . . . . 4
1.3 Need for a Packet Sequence Number . . . . . . . . . . . . . 5
1.4 Rationale for proposed solution . . . . . . . . . . . . . . 5
1.5 PDM Works in Collaboration with Other Headers . . . . . . . 6
2 Measurement Information Derived from PDM . . . . . . . . . . . . 6
2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Performance and Diagnostic Metrics Destination Option Layout . . 7
3.1 Destination Options Header . . . . . . . . . . . . . . . . . 7
3.2 Performance and Diagnostic Metrics Destination Option . . . 7
3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 10
3.4 Implementation Considerations . . . . . . . . . . . . . . . 11
3.5 Dynamic Configuration Options . . . . . . . . . . . . . . . 11
3.6 5-tuple Aging . . . . . . . . . . . . . . . . . . . . . . . 12
4 Considerations of Timing Representation . . . . . . . . . . . . 12
4.1 Encoding the Delta-Time Values . . . . . . . . . . . . . . . 12
4.2 Timer registers are different on different hardware . . . . 12
4.3 Timer Units on Other Systems . . . . . . . . . . . . . . . . 13
4.4 Time Base . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.5 Timer-value scaling . . . . . . . . . . . . . . . . . . . . 14
4.6 Limitations with this encoding method . . . . . . . . . . . 15
4.7 Lack of precision induced by timer value truncation . . . . 15
5 PDM Flow - Simple Client Server . . . . . . . . . . . . . . . . 16
5.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
6 Other Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . . . 22
Elkins Expires October 2, 2015 [Page 1]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
6.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . . . 23
6.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . . . 24
7 Potential Overhead Considerations . . . . . . . . . . . . . . . 25
8 Security Considerations . . . . . . . . . . . . . . . . . . . . 26
9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 27
10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 27
10.1 Normative References . . . . . . . . . . . . . . . . . . . 27
10.2 Informative References . . . . . . . . . . . . . . . . . . 27
11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27
Abstract
To assess performance problems, measurements based on optional
sequence numbers and timing may be embedded in each packet. Such
measurements may be interpreted in real-time or after the fact. An
implementation of the existing IPv6 Destination Options extension
header, the Performance and Diagnostic Metrics (PDM) Destination
Options extension header as well as the field limits, calculations,
and usage of the PDM in measurement are included in this document.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Elkins Expires October 2, 2015 [Page 2]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Copyright and License Notice
Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License
Elkins Expires October 2, 2015 [Page 3]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
1 Background
To assess performance problems, measurements based on optional
sequence numbers and timing may be embedded in each packet. Such
measurements may be interpreted in real-time or after the fact. An
implementation of the existing IPv6 Destination Options extension
header, the Performance and Diagnostic Metrics (PDM) Destination
Options extension header has been proposed in a companion document.
This document specifies the layout, field limits, calculations, and
usage of the PDM in measurement.
As defined in RFC2460 [RFC2460], destination options are carried by
the IPv6 Destination Options extension header. Destination options
include optional information that need be examined only by the IPv6
node given as the destination address in the IPv6 header, not by
routers or other "middle boxes". This document specifies a new
destination option, the Performance and Diagnostic Metrics (PDM)
destination option.
1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
1.2 End User Quality of Service (QoS)
The difference between timing values in the PDM traveling along with
the packet will be used to estimate QoS as experienced by an end user
device.
For many applications, the key user performance indicator is response
time. When the end user is an individual, he is generally
indifferent to what is happening along the network; what he really
cares about is how long it takes to get a response back. But this is
not just a matter of individuals' personal convenience. In many
cases, rapid response is critical to the business being conducted.
When the end user is a device (e.g. with the Internet of Things),
what matters is the speed with which requested data can be
transferred -- specifically, whether the requested data can be
transferred in time to accomplish the desired actions. This can be
important when the relevant external conditions are subject to rapid
change.
Response time and consistency are not just "nice to have". On many
networks, the impact can be financial hardship or endanger human
life. In some cities, the emergency police contact system operates
Elkins Expires October 2, 2015 [Page 4]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
over IP, law enforcement uses TCP/IP networks, transactions on our
stock exchanges are settled using IP networks. The critical nature
of such activities to our daily lives and financial well-being demand
a simple solution to support measurements.
1.3 Need for a Packet Sequence Number
While performing network diagnostics of an end-to-end connection, it
often becomes necessary to find the device along the network path
creating problems. Diagnostic data may be collected at multiple
places along the path (if possible), or at the source and
destination. Then, in post-collection processing, the diagnostic
data corresponding to each packet at different observation points
must be matched for proper measurements. A sequence number in each
packet provides sufficient basis for the matching process. If need
be, the timing fields may be used along with the sequence number to
ensure uniqueness.
This method of data collection along the path is of special use to
determine where packet loss or packet corruption is happening.
The packet sequence number needs to be unique in the context of the
session (5-tuple). See section 2 for a definition of 5-tuple.
1.4 Rationale for proposed solution
The current IPv6 specification does not provide timing nor a similar
field in the IPv6 main header or in any extension header. So, we
propose the IPv6 Performance and Diagnostic Metrics destination
option (PDM).
Advantages include:
1. Real measure of actual transactions.
2. Independence from transport layer protocols.
3. Ability to span organizational boundaries with consistent
instrumentation
4. No time synchronization needed between session partners
The PDM provides the ability to quickly determine if the (latency)
problem is in the network or in the server (application). More
intermediate measurements may be needed if the host or network
discrimination is not sufficient. At the client, TCP/IP stack time
vs. applications time may still need to be broken out by client
software.
Elkins Expires October 2, 2015 [Page 5]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
1.5 PDM Works in Collaboration with Other Headers
The purpose of the PDM is not to supplant all the variables present
in all other headers but to provide data which is not available or
very difficult to get. The way PDM would be used is by a technician
(or tool) looking at a packet capture. Within the packet capture,
they would have available to them the layer 2 header, IP header (v6
or v4), TCP, UCP, ICMP, SCTP or other headers. All information
would be looked at together to make sense of the packet flow. The
technician or processing tool could analyze, report or ignore the
data from PDM, as necessary.
For an example of how PDM can help with TCP retransmit problems,
please look at section 8.
2 Measurement Information Derived from PDM
Each packet contains information about the sender and receiver. In IP
protocol, the identifying information is called a "5-tuple".
The 5-tuple consists of:
SADDR : IP address of the sender
SPORT : Port for sender
DADDR : IP address of the destination
DPORT : Port for destination
PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.)
The PDM contains the following base fields:
PSNTP : Packet Sequence Number This Packet
PSNLR : Packet Sequence Number Last Received
DELTATLR : Delta Time Last Received
DELTATLS : Delta Time Last Sent
Other fields for scaling and time base are also in the PDM and will
be described in section 3.
This information, combined with the 5-tuple, allows the measurement
of the following metrics:
1. Round-trip delay
2. Server delay
2.1 Round-Trip Delay
Round-trip *Network* delay is the delay for packet transfer from a
Elkins Expires October 2, 2015 [Page 6]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
source host to a destination host and then back to the source host.
This measurement has been defined, and the advantages and
disadvantages discussed in "A Round-trip Delay Metric for IPPM"
[RFC2681].
2.2 Server Delay
Server delay is the interval between when a packet is received by a
device and the first corresponding packet is sent back in response.
This may be "Server Processing Time". It may also be a delay caused
by acknowledgements. Server processing time includes the time taken
by the combination of the stack and application to return the
response. The stack delay may be related to network performance. If
this aggregate time is seen as a problem, and there is a need to make
a clear distinction between application processing time and stack
delay, including that caused by the network, then more client based
measurements are needed.
3 Performance and Diagnostic Metrics Destination Option Layout
3.1 Destination Options Header
The IPv6 Destination Options Header is used to carry optional
information that need be examined only by a packet's destination
node(s). The Destination Options Header is identified by a Next
Header value of 60 in the immediately preceding header and is defined
in RFC2460 [RFC2460]. The IPv6 Performance and Diagnostic Metrics
Destination Option (PDM) is an implementation of the Destination
Options Header (Next Header value = 60). The PDM does not require
time synchronization.
3.2 Performance and Diagnostic Metrics Destination Option
The IPv6 Performance and Diagnostic Metrics Destination Option (PDM)
contains the following fields:
TIMEBASE : Base timer unit
SCALEDTLR: Scale for Delta Time Last Received
SCALEDTLS: Scale for Delta Time Last Sent
PSNTP : Packet Sequence Number This Packet
PSNLR : Packet Sequence Number Last Received
DELTATLR : Delta Time Last Received
DELTATLS : Delta Time Last Sent
Elkins Expires October 2, 2015 [Page 7]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
The PDM destination option is encoded in type-length-value (TLV)
format as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Option Length |TB |ScaleDTLR | ScaleDTLS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PSN This Packet | PSN Last Received |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time Last Received | Delta Time Last Sent |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Option Type
TBD = 0xXX (TBD) [To be assigned by IANA] [RFC2780]
Option Length
8-bit unsigned integer. Length of the option, in octets, excluding
the Option Type and Option Length fields. This field MUST be set to
16.
Time Base
2-bit unsigned integer. It will indicate the lowest granularity
possible for this device. That is, for a value of 00 in the Time
Base field, a value of 1 in the DELTA fields indicates 1
microsecond.
This field is being included so that a device may choose the
granularity which most suits its timer ticks. That is, so that it
does not have to do more work than needed to convert values required
for the PDM.
The possible values of Time Base are as follows:
00 - milliseconds
01 - microseconds
10 - nanoseconds
11 - picoseconds
Scale Delta Time Last Received (SCALEDTLR)
7-bit signed integer. This is the scaling value for the Delta Time
Last Received (DELTATLR) field. The possible values are from -128 to
+127. See Section 4 for further discussion on Timing Considerations
and formatting of the scaling values.
Elkins Expires October 2, 2015 [Page 8]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Scale Delta Time Last Sent (SCALEDTLS)
7-bit signed integer. This is the scaling value for the Delta Time
Last Sent (DELTATLS) field. The possible values are from -128 to
+127.
Packet Sequence Number This Packet (PSNTP)
16-bit unsigned integer. This field will wrap. It is intended for
human use. That is, while to be used while analyzing packet traces.
Initialized at a random number and monotonically incremented for each
packet on the 5-tuple. The 5-tuple consists of the source and
destination IP addresses, the source and destination ports, and the
upper layer protocol (ex. TCP, ICMP, etc). The random number
initialization is to make it harder to spoof and insert such packets.
Operating systems MUST implement a separate packet sequence number
counter per 5-tuple. Operating systems MUST NOT implement a single
counter for all connections.
Packet Sequence Number Last Received (PSNLR)
16-bit unsigned integer. This is the PSN of the packet last received
on the 5-tuple.
Delta Time Last Received (DELTATLR)
A 16-bit unsigned integer field. The value is according to the scale
in SCALEDTLR.
DELTATLR = Send time packet 2 - Receive time packet 1
Delta TimeLast Sent (DELTATLS)
A 16-bit unsigned integer field. The value is according to the
scale in SCALEDTLS.
Delta Time Last Sent = Receive time packet 2 - Send time packet 1
Option Type
The two highest-order bits of the Option Type field are encoded to
Elkins Expires October 2, 2015 [Page 9]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
indicate specific processing of the option; for the PDM destination
option, these two bits MUST be set to 00. This indicates the
following processing requirements:
00 - skip over this option and continue processing the header.
RFC2460 [RFC2460] defines other values for the Option Type field.
These MUST NOT be used in the PDM. The other values are as follows:
01 - discard the packet.
10 - discard the packet and, regardless of whether or not the
packet's Destination Address was a multicast address, send an ICMP
Parameter Problem, Code 2, message to the packet's Source Address,
pointing to the unrecognized Option Type.
11 - discard the packet and, only if the packet's Destination Address
was not a multicast address, send an ICMP Parameter Problem, Code 2,
message to the packet's Source Address, pointing to the unrecognized
Option Type.
In keeping with RFC2460 [RFC2460], the third-highest-order bit of the
Option Type specifies whether or not the Option Data of that option
can change en-route to the packet's final destination.
In the PDM, the value of the third-highest-order bit MUST be 0. The
possible values are as follows:
0 - Option Data does not change en-route
1 - Option Data may change en-route
The three high-order bits described above are to be treated as part
of the Option Type, not independent of the Option Type. That is, a
particular option is identified by a full 8-bit Option Type, not just
the low-order 5 bits of an Option Type.
3.3 Header Placement
The PDM destination option MUST be placed as follows:
- Before the upper-layer header. That is, this is the last
extension header.
This follows the order defined in RFC2460 [RFC2460]
IPv6 header
Elkins Expires October 2, 2015 [Page 10]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Hop-by-Hop Options header
Destination Options header
Routing header
Fragment header
Authentication header
Encapsulating Security Payload header
Destination Options header
upper-layer header
For each IPv6 packet header, the PDM MUST NOT appear more than once.
However, an encapsulated packet MAY contain a separate PDM associated
with each encapsulated IPv6 header.
3.4 Implementation Considerations
The PDM destination options extension header SHOULD be turned on by
each stack on a host node. It MAY also be turned on only in case of
diagnostics needed for problem resolution.
3.5 Dynamic Configuration Options
If implemented, each operating system MUST have a default
configuration parameter, e.g. diag_header_sys_default_value=yes/no.
The operating system MAY also have a dynamic configuration option to
change the configuration setting as needed.
If the PDM destination options extension header is used, then it MAY
be turned on for all packets flowing through the host, applied to an
upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP
address only. These are at the discretion of the implementation.
The PDM MUST NOT be changed dynamically via packet flow as this may
create potential security violation or DoS attack by numerous packets
turning the header on and off.
As with all other destination options extension headers, the PDM is
for destination nodes only. As specified above, intermediate devices
MUST neither set nor modify this field.
Elkins Expires October 2, 2015 [Page 11]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
3.6 5-tuple Aging
Within the operating system, metrics must be kept on a 5-tuple basis.
The 5-tuple is:
SADDR : IP address of the sender SPORT : Port for sender DADDR : IP
address of the destination DPORT : Port for destination PROTC :
Protocol for upper layer (ex. TCP, UDP, ICMP)
The question comes of when to stop keeping data or restarting the
numbering for a 5-tuple. For example, in the case of TCP, at some
point, the connection will terminate. Keeping data in control blocks
forever, will have unfortunate consequences for the operating system.
So, the recommendation is to use a known aging parameter such as Max
Segment Lifetime (MSL) as defined in Transmission Control Protocol
[RFC0793] to reuse or drop the control block. The choice of aging
parameter is left up to the implementation.
4 Considerations of Timing Representation
4.1 Encoding the Delta-Time Values
This section makes reference to and expands on the document "Encoding
of Time Intervals for the TCP Timestamp Option" [TRAM-TCPM].
4.2 Timer registers are different on different hardware
One of the problems with timestamp recording is the variety of
hardware that generates the time value to be used. Different CPUs
track the time in registers of different sizes, and the most-
frequently-iterated bit could be the first on the left or the first
on the right. In order to generate some examples here it is necessary
to indicate the type of timer register being used.
As described in the "IBM z/Architecture Principles of Operation"
[IBM-POPS], the Time-Of-Day clock in a zSeries CPU is a 104-bit
register, where bit 51 is incremented approximately every
microsecond:
1
0 1 2 3 4 5 6 0
+--------+---------+---------+---------+---------+---------+--+...+
| | | | | |* | |
+--------+---------+---------+---------+---------+---------+--+...+
^ ^ ^
0 51 = 1 usec 103
Elkins Expires October 2, 2015 [Page 12]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
To represent these values concisely a hexadecimal representation will
be used, where each digit represents 4 binary bits. Thus:
0000 0000 0000 0001 = 1 timer unit (2**-12 usec, or about 244 psec)
0000 0000 0000 1000 = 1 microsecond
0000 0000 003E 8000 = 1 millisecond
0000 0000 F424 0000 = 1 second
0000 0039 3870 0000 = 1 minute
0000 0D69 3A40 0000 = 1 hour
0001 41DD 7600 0000 = 1 day
Note that only the first 64 bits of the register are commonly
represented, as that represents a count of timer units on this
hardware. Commonly the first 52 bits are all that are displayed, as
that represents a count of microseconds.
4.3 Timer Units on Other Systems
This encoding method works the same with other hardware clock
formats. The method uses a microsecond as the basic value and allows
for large time differentials.
4.4 Time Base
This specification allows for the fact that different CPU TOD clocks
use different binary points. For some clocks, a value of 1 could
indicate 1 microsecond, whereas other clocks could use the value 1 to
indicate 1 millisecond. In the former case, the binary digits to the
right of that binary point measure 2**(-n) microseconds, and in the
latter case, 2**(-n) milliseconds.
The Time Base allows us to ensure we have a common reference, at the
very least, common knowledge of what the binary point is for the
transmitted values.
We propose a base unit for the time. This is a 2-bit integer
indicating the lowest granularity possible for this device. That is,
for a value of 00 in the Time Base field, a value of 1 in the DELTA
fields indicates 1 picosecond.
The possible values of Time Base are as follows:
00 - milliseconds
01 - microseconds
10 - nanoseconds
11 - picoseconds
Elkins Expires October 2, 2015 [Page 13]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Time base is not necessarily equivalent to length of one timer tick.
That is, on many, if not all, systems, the timer tick value will not
be in complete units of nanoseconds, milliseconds, etc. For example,
on an IBM zSeries machine, one timer tick (or clock unit) is 2 to the
-12th microseconds.
Therefore, some amount of conversion may be needed to approximate
Time Base units.
4.5 Timer-value scaling
As discussed in [TRAM-TCPM] we propose storing not an entire time-
interval value, but just the most significant bits of that value,
along with a scaling factor to indicate the magnitude of the time-
interval value. In our case, we will use the high-order 16 bits. The
scaling value will be the number of bits in the timer register to the
right of the 16th significant bit. That is, if the timer register
contains this binary value:
1110100011010100101001010001000000000000
<-16 bits -><-24 bits ->
then, the values stored would be 1110 1000 1101 0100 in binary (E8D4
hexadecimal) for the time value and 24 for the scaling value. Note
that the displayed value is the binary equivalent of 1 second
expressed in picoseconds.
The below table represents a device which has a TimeBase of
picosecond (or 00). The smallest and simplest value to represent is
1 picosecond; the time value stored is 1, and the scaling value is 0.
Using values from the table below, we have:
Time value in Encoded Scaling
Delta time picoseconds value decimal
--------------------------------------------------------
1 picosecond 1 1 0
1 nanosecond 3E8 3E8 0
1 microsecond F4240 F424 4
1 millisecond 3B9ACA00 3B9A 16
1 second E8D4A51000 E8D4 24
1 minute 3691D6AFC000 3691 32
1 hour cca2e51310000 CCA2 36
1 day 132f4579c980000 132F 44
365 days 1b5a660ea44b80000 1B5A 52
Elkins Expires October 2, 2015 [Page 14]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Sample binary values (high order 16 bits taken)
1 psec 1 0001
1 nsec 3E8 0011 1110 1000
1 usec F4240 1111 0100 0010 0100 0000
1 msec 3B9ACA00 0011 1011 1001 1010 1100 1010 0000 0000
1 sec E8D4A51000 1110 1000 1101 0100 1010 0101 0001 0000 0000 0000
4.6 Limitations with this encoding method
If we follow the specification in [TRAM-TCPM], the size of one of
these time-interval fields is limited to this 11-bit value and five-
bit scale, so that they fit into a 16-bit space. With that
limitation, the maximum value that could be stored in 16 bits is:
11-bit value Scale
============= ======
1111 1111 111 1 1111
or an encoded value of 3FF and a scale value of 31. This value
corresponds to any time differential between:
|<Count of zeroes is the Scale value>|
11 1111 1111 1000 0000 0000 0000 0000 0000 0000 0000 (binary)
3 F F 8 0 0 0 0 0 0 0 (hexadecimal)
and
11 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 (binary)
3 F F F F F F F F F F (hexadecimal)
This time value, 3FFFFFFFFFF, converts to 50 days, 21 hours, 40
minutes and 46.511103 seconds. A time differential 1 microsecond
longer won't fit into 16 bits using this encoding method.
4.7 Lack of precision induced by timer value truncation
When the bit values following the first 11 significant bits are
truncated, obviously loss of precision in the value. The range of
values that will be truncated to the same encoded value is
2**(Scale)-1 microseconds.
Elkins Expires October 2, 2015 [Page 15]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
The smallest time differential value that will be truncated is
1000 0000 0000 = 2.048 msec
The value
1000 0000 0001 = 2.049 msec
will be truncated to the same encoded value, which is 400 in hex,
with a scale value of 1. With the scale value of 1, the value range
is calculated as 2**1 - 1, or 1 usec, which you can see is the
difference between these minimum and maximum values.
With that in mind, let's look at that table of delta time values
again, where the Precision is the range from the smallest value
corresponding to this encoded value to the largest:
Time value in Encoded
Delta time microseconds value Scale Precision
1 microsecond 1 1 0 0:00.000000
1 millisecond 38E 38E 0 0:00.000000
1 second F4240 7A1 9 0:00.000511
1 minute 3938700 727 15 0:00.032767
1 hour D693A400 6B4 21 0:02.097151
1 day 141DD76000 507 26 1:07.108863
Maximum value 3FFFFFFFFFF 7FF 31 35:47.483647
So, when measuring the delay between transmission of two packets, or
between the reception of two packets, any delay shorter than 50 days
21 hours and change can be stored in this encoded fashion within 16
bits. When you encode, for example, a DTN response time delay of 50
days, 21 hours and 40 minutes, you can be assured of accuracy within
35 minutes.
5 PDM Flow - Simple Client Server
Following is a sample simple flow for the PDM with one packet sent
from Host A and one packet received by Host B. The PDM does not
require time synchronization between Host A and Host B. The
calculations to derive meaningful metrics for network diagnostics are
shown below each packet sent or received.
Elkins Expires October 2, 2015 [Page 16]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Each packet, in addition to the PDM contains information on the
sender and receiver. As discussed before, a 5-tuple consists of:
SADDR : IP address of the sender
SPORT : Port for sender
DADDR : IP address of the destination
DPORT : Port for destination
PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP)
It should be understood that the packet identification information is
in each packet. We will not repeat that in each of the following
steps.
5.1 Step 1
Packet 1 is sent from Host A to Host B. The time for Host A is set
initially to 10:00AM.
The time and packet sequence number are saved by the sender
internally. The packet sequence number and delta times are sent in
the packet.
Packet 1
+----------+ +----------+
| | | |
| Host | ----------> | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 25
PSNLR : Packet Sequence Number Last Received: -
DELTATLR : Delta Time Last Received: -
SCALEDTLR: Scale of Delta Time Last Received: 0
DELTATLS : Delta Time Last Sent: -
SCALEDTLS: Scale of Delta Time Last Sent: 0
TIMEBASE : Granularity of Time: 00 (Milliseconds)
Internally, within the sender, Host A, it must keep:
Packet Sequence Number of the last packet sent: 25
Time the last packet was sent: 10:00:00
Note, the initial PSNTP from Host A starts at a random number. In
Elkins Expires October 2, 2015 [Page 17]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
this case, 25. The time in these examples is shown in seconds for
the sake of simplicity.
5.2 Step 2
Packet 1 is received at Host B. Its time is set to one hour later
than Host A. In this case, 11:00AM
Internally, within the receiver, Host B, it must note:
Packet Sequence Number of the last packet received: 25
Time the last packet was received : 11:00:03
Note, this timestamp is in Host B time. It has nothing whatsoever to
do with Host A time. The Packet Sequence Number of the last packet
received will become PSNLR which will be sent out in the packet sent
by Host B in the next step. The time last received will be used to
calculate the DELTALR value to be sent out in the packet sent by Host
B in the next step.
5.3 Step 3
Packet 2 is sent by Host B to Host A. Note, the initial packet
sequence number (PSNTP) from Host B starts at a random number. In
this case, 12. Before sending the packet, Host B does a calculation
of deltas. Since Host B knows when it is sending the packet, and it
knows when it received the previous packet, it can do the following
calculation:
Sending time (packet 2) - receive time (packet 1)
We will call the result of this calculation: Delta Time Last
Received
That is:
DELTATLR = Sending time (packet 2) - receive time (packet 1)
Note, both sending time and receive time are saved internally in Host
B. They do not travel in the packet. Only the Delta is in the
packet.
Assume that within Host B is the following:
Packet Sequence Number of the last packet received: 25
Time the last packet was received: 11:00:03
Packet Sequence Number of this packet: 12
Elkins Expires October 2, 2015 [Page 18]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Time this packet is being sent: 11:00:07
We can now calculate a delta value to be sent out in the packet.
DELTATLR becomes:
4 seconds = 11:00:07 - 11:00:03
This is the derived metric: Server Delay. The time and scaling
factor must be calculated. Then, this value, along with the packet
sequence numbers will be sent to Host A as follows:
Packet 2
+----------+ +----------+
| | | |
| Host | <---------- | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 12
PSNLR : Packet Sequence Number Last Received: 25
DELTATLR : Delta Time Last Received: 3A35 (4 seconds)
SCALEDTLR: Scale of Delta Time Last Received: 25
DELTATLS : Delta Time Last Sent: -
SCALEDTLS: Scale of Delta Time Last Sent: 0
TIMEBASE : Granularity of Time: 00 (Milliseconds)
The metric left to be calculated is the Round-Trip Delay. This will
be calculated by Host A when it receives Packet 2.
5.4 Step 4
Packet 2 is received at Host A. Remember, its time is set to one
hour earlier than Host B. Internally, it must note:
Packet Sequence Number of the last packet received: 12
Time the last packet was received : 10:00:12
Note, this timestamp is in Host A time. It has nothing whatsoever to
do with Host B time.
So, now, Host A can calculate total end-to-end time. That is:
End-to-End Time = Time Last Received - Time Last Sent
Elkins Expires October 2, 2015 [Page 19]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was
received by Host A at 10:00:12 so:
End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT
delay combined). This time may also be called total Overall Round-
trip time (which includes Network RTT and Host Response Time).
This derived metric we will call DELTATLS or Delta Time Last Sent.
We can now also calculate round trip delay. The formula is:
Round trip delay = DELTATLS - DELTATLR
Or:
Round trip delay = 12 - 4 or 8
Now, the only problem is that at this point all metrics are in Host A
only and not exposed in a packet. To do that, we need a third packet.
Note: this simple example assumes one send and one receive. That
is done only for purposes of explaining the function of the PDM. In
cases where there are multiple packets returned, one would take the
time in the last packet in the sequence. The calculations of such
timings and intelligent processing is the function of post-processing
of the data.
5.5 Step 5
Packet 3 is sent from Host A to Host B.
+----------+ +----------+
| | | |
| Host | ----------> | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 26
PSNLR : Packet Sequence Number Last Received: 12
DELTATLR : Delta Time Last Received: 0
SCALEDTLS: Scale of Delta Time Last Received 0
DELTATLS : Delta Time Last Sent: 105e (12 seconds)
SCALEDTLR: Scale of Delta Time Last Received: 26
TIMEBASE : Granularity of Time: 00 (Milliseconds)
Elkins Expires October 2, 2015 [Page 20]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
To calculate Two-Way Delay, any packet capture device may look at
these packets and do what is necessary.
6 Other Flows
What we have discussed so far is a simple flow with one packet sent
and one returned. Let's look at how PDM may be useful in other
types of flows.
Elkins Expires October 2, 2015 [Page 21]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
6.1 PDM Flow - One Way Traffic
The flow on a particular session may not be a send-receive paradigm.
Let us consider some other situations. In the case of a one-way
flow, one might see the following:
Packet Sender PSN PSN Delta Time Delta Time
This Packet Last Recvd Last Recvd Last Sent
=====================================================================
1 Server 1 0 0 0
2 Server 2 0 0 5
3 Server 3 0 0 12
4 Server 4 0 0 20
What does this mean and how is it useful?
In a one-way flow, only the Delta Time Last Sent will be seen as
used. Recall, Delta Time Last Sent is the difference between the
send of one packet from a device and the next. This is a measure of
throughput for the sender - according to the sender's point of view.
That is, it is a measure of how fast is the application itself (with
stack time included) able to send packets.
How might this be useful? If one is having a performance issue at
the client and sees that packet 2, for example, is sent after 5
microseconds from the server but takes 3 minutes to arrive at the
destination, then one may safely conclude that there are delays in
the path other than at the server which may be causing the delivery
issue of that packet. Such delays may include the network links,
middle-boxes, etc.
Now, true one-way traffic is quite rare. What people often mean by
"one-way" traffic is an application such as FTP where a group of
packets (for example, a TCP window size worth) is sent, then the
sender waits for acknowledgment. This type of flow would actually
fall into the "multiple-send" traffic model.
Elkins Expires October 2, 2015 [Page 22]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
6.2 PDM Flow - Multiple Send Traffic
Assume that two packets are sent for each ACK from the server.
Packet Sender PSN PSN Delta Time Delta Time
This Packet Last Recvd Last Recvd Last Sent
=====================================================================
1 Server 1 0 0 0
2 Server 2 0 0 5
3 Client 1 2 20 0
4 Server 3 1 10 15
How might this be used?
Notice that in packet 3, the client has a value of Delta Time Last
received of 20. Recall that Delta Time Last Received is the Send
time of packet 3 - receive time of packet 2. So, what does one know
now? In this case, Delta Time Last Received is the processing time
for the Client to send the next packet.
How to interpret this depends on what is actually being sent.
Remember, PDM is not being used in isolation, but to supplement the
fields found in other headers. Let's take some examples:
1. Client is sending a standalone TCP ACK. One would find this by
looking at the payload length in the IPv6 header and the TCP
Acknowledgement field in the TCP header. So, in this case, the
client is taking 20 units to send back the ACK. This may or may not
be interesting.
2. Client is sending data with the packet. Again, one would find
this by looking at the payload length in the IPv6 header and the TCP
Acknowledgement field in the TCP header. So, in this case, the
client is taking 20 units to send back data. This may represent
"User Think Time". Again, this may or may not be interesting, in
isolation. But, if there is a performance problem receiving data at
the server, then taken in conjunction with RTT or other packet timing
information, this information may be quite interesting.
Of course, one also needs to look at the PSN Last Received field to
make sure of the interpretation of this data. That is, to make
sure that the Delta Last Received corresponds to the packet of
interest.
The benefits of PDM are that we have such information available in a
uniform manner for all applications and all protocols without
extensive changes required to applications.
Elkins Expires October 2, 2015 [Page 23]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
6.3 PDM Flow - Multiple Send with Errors
One might wonder if all of the functions of PDM might be better
suited to TCP or a TCP option. Let us take the case of how PDM may
help in a case of TCP retransmissions in a way that TCP options or
TCP ACK / SEQ would not.
Assume that three packets are sent with each send from the server.
From the server, this is what is seen.
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt LastRecvd LastRecvd LastSent SEQ Bytes
=====================================================================
1 Server 1 0 0 0 123 100
2 Server 2 0 0 5 223 100
3 Server 3 0 0 5 333 100
The client however, does not get all the packets. From the client,
this is what is seen for the packets sent from the server.
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt LastRecvd LastRecvd LastSent SEQ Bytes
=====================================================================
1 Server 1 0 0 0 123 100
2 Server 3 0 0 5 333 100
Let's assume that the server now retransmits the packet.
(Obviously, a duplicate acknowledgment sequence for fast retransmit
or a retransmit timeout would occur. To illustrate the point, these
packets are being left out.)
So, then if a TCP retransmission is done, then from the client, this
is what is seen for the packets sent from the server.
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt LastRecvd LastRecvd LastSent SEQ Bytes
=====================================================================
1 Server 4 0 0 30 223 100
The server has resent the old packet 2 with TCP sequence number of
223. The retransmitted packet now has a PSN This Packet value of 4.
The Delta Last Sent is 30 - the time between sending the packet with
PSN of 3 and this current packet.
Let's say that packet 4 STILL does not make it. Then, after some
amount of time (RTO) then the packet with TCP sequence number of 223
Elkins Expires October 2, 2015 [Page 24]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
is resent.
From the client, this is what is seen for the packets sent from the
server.
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt LastRecvd LastRecvd LastSent SEQ Bytes
=====================================================================
1 Server 5 0 0 60 223 100
If now, this packet makes it, one has a very good idea that packets
exist which are being sent from the server as retransmissions and not
making it to the client. This is because the PSN of the resent
packet from the server is 5 rather than 4. If we had used TCP
sequence number alone, we would never have seen this situation.
Because the TCP sequence number in all situations is 223.
This situation would be experienced by the user of the application
(the human being actually sitting somewhere) as a "hangs" or long
delay between packets. On large networks, to diagnose problems such
as these where packets are lost somewhere on the network, one has to
take multiple traces to find out exactly where.
The first thing is to start with doing a trace at the client and the
server. So, we can see if the server sent a particular packet and
the client received it. If the client did not receive it, then we
start tracking back to trace points at the router right after the
server and the router right before the client. Did they get these
packets which the server has sent? This is a time consuming
activity.
With PDM, we can speed up the diagnostic time because we may be able
to use only the trace taken at the client to see what the server is
sending.
7 Potential Overhead Considerations
Questions have been posed as to the potential overhead of PDM.
First, PDM is entirely optional. That is, a site may choose to
implement PDM or not as they wish. If they are happy with the costs
of PDM vs. the benefits, then the choice should be theirs.
Below is a table outlining the potential overhead in terms of
additional time to deliver the response to the end user for various
assumed RTTs.
Elkins Expires October 2, 2015 [Page 25]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Packet
Bytes RTT Bytes Bytes New Overhead
in Packet Per Milli in PDM RTT
=====================================================================
1000 1000 milli 1 16 1016.000 16.000 milli
1000 100 milli 10 16 101.600 1.600 milli
1000 10 milli 100 16 10.160 .160 milli
1000 1 milli 1000 16 1.016 .016 milli
Below are some examples of actual RTTs for packets traversing large
enterprise networks. The first example is for packets going to
multiple business partners.
Packet
Bytes RTT Bytes Bytes New Overhead
in Packet Per Milli in PDM RTT
=====================================================================
1000 17 milli 58 16 17.360 .360 milli
The second example is for packets at a large enterprise customer
within a data center. Notice that the scale is now in microseconds
rather than milliseconds.
Packet
Bytes RTT Bytes Bytes New Overhead
in Packet Per Micro in PDM RTT
=====================================================================
1000 20 micro 50 16 20.320 .320 micro
8 Security Considerations
The PDM MUST NOT be changed dynamically via packet flow as this
creates a possibility for potential security violations or DoS
attacks by numerous packets turning the header on and off.
Attackers may also send many packets from multiple ports, for example
by doing a port scan. This will cause the stack to create many
control blocks. This is the same problem as seen for SYN flood
attacks. Similar protections should be implemented by the stack to
preserve the integrity of memory.
Elkins Expires October 2, 2015 [Page 26]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
9 IANA Considerations
Option Type to be assigned by IANA [RFC2780].
10 References
10.1 Normative References
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, September 1999.
[RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For
Values In the Internet Protocol and Related Headers", BCP 37, RFC
2780, March 2000.
10.2 Informative References
[TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP
Timestamp Option-01", Internet Draft, July 2013. [Work in Progress]
[IBM-POPS] IBM Corporation, "IBM z/Architecture Principles of
Operation", SA22-7832, 1990-2012
11 Acknowledgments
The authors would like to thank Keven Haining, Al Morton, Brian
Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick
Troth for their comments and assistance.
Authors' Addresses
Nalini Elkins
Inside Products, Inc.
36A Upper Circle
Carmel Valley, CA 93924
United States
Phone: +1 831 659 8360
Email: nalini.elkins@insidethestack.com
http://www.insidethestack.com
Elkins Expires October 2, 2015 [Page 27]
INTERNET DRAFT elkins-ippm-6man-pdm-option-00 March 31, 2015
Robert Hamilton
Chemical Abstracts Service
A Division of the American Chemical Society
2540 Olentangy River Road
Columbus, Ohio 43202
United States
Phone: +1 614 447 3600 x2517
Email: rhamilton@cas.org
http://www.cas.org
Michael S. Ackermann
Blue Cross Blue Shield of Michigan
P.O. Box 2888
Detroit, Michigan 48231
United States
Phone: +1 310 460 4080
Email: mackermann@bcbsmi.com
http://www.bcbsmi.com
Elkins Expires October 2, 2015 [Page 28]
