| TCPM Working Group | G. Fairhurst |
| Internet-Draft | A. Sathiaseelan |
| Obsoletes: 2861 (if approved) | R. Secchi |
| Updates: 5681 (if approved) | University of Aberdeen |
| Intended status: Standards Track | February 05, 2014 |
| Expires: August 09, 2014 |
Updating TCP to support Rate-Limited Traffic
draft-ietf-tcpm-newcwv-05
This document proposes an update to RFC 5681 to address issues that arise when TCP is used to support traffic that exhibits periods where the sending rate is limited by the application rather than the congestion window. It updates TCP to allow a TCP sender to restart quickly following either an idle or rate-limited interval. This method is expected to benefit applications that send rate-limited traffic using TCP, while also providing an appropriate response if congestion is experienced.
It also evaluates the Experimental specification of TCP Congestion Window Validation, CWV, defined in RFC 2861, and concludes that RFC 2861 sought to address important issues, but failed to deliver a widely used solution. This document therefore recommends that the status of RFC 2861 is moved from Experimental to Historic, and that it is replaced by the current specification.
NOTE: The standards status of this WG document is under review for consideration as either Experimental (EXP) or Proposed Standard (PS). This decision will be made later as the document is finalised.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 09, 2014.
Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
TCP is used to support a range of application behaviours. The TCP congestion window (cwnd) controls the number of unacknowledged packets/bytes that a TCP flow may have in the network at any time, a value known as the FlightSize [RFC5681]. A bulk application will always have data available to transmit. The rate at which it sends is therefore limited by the maximum permitted by the receiver advertised window and the sender congestion window (cwnd). In contrast, a rate-limited application will experience periods when the sender is either idle or is unable to send at the maximum rate permitted by the cwnd. This latter case is called rate-limited. The focus of this document is on the operation of TCP in such an idle or rate-limited case.
Standard TCP [RFC5681] requires the cwnd to be reset to the restart window (RW) when an application becomes idle. [RFC2861] noted that this TCP behaviour was not always observed in current implementations. Recent experiments [Bis08] confirm this to still be the case.
Standard TCP does not impose additional restrictions on the growth of the cwnd when a TCP sender is rate-limited. A rate-limited sender may therefore grow a cwnd far beyond that corresponding to the current transmit rate, resulting in a value that does not reflect current information about the state of the network path the flow is using. Use of such an invalid cwnd may result in reduced application performance and/or could significantly contribute to network congestion.
[RFC2861] proposed a solution to these issues in an experimental method known as Congestion Window Validation (CWV). CWV was intended to help reduce cases where TCP accumulated an invalid cwnd. The use and drawbacks of using the CWV algorithm in RFC 2861 with an application are discussed in Section 2.
Section 3 defines relevant terminology.
Section 4 specifies an alternative to CWV that seeks to address the same issues, but does this in a way that is expected to mitigate the impact on an application that varies its sending rate. The method described applies to both a rate-limited and an idle condition. Section 5 describes the rationale for selecting the safe period to preserve the cwnd.
RFC 2861 described a simple modification to the TCP congestion control algorithm that decayed the cwnd after the transition to a “sufficiently-long” idle period. This used the slow-start threshold (ssthresh) to save information about the previous value of the congestion window. The approach relaxed the standard TCP behaviour [RFC5681] for an idle session, intended to improve application performance. CWV also modified the behaviour for a rate-limited session where a sender transmitted at a rate less than allowed by cwnd.
RFC 2861 has been implemented in some mainstream operating systems as the default behaviour [Bis08]. Analysis (e.g. [Bis10] [Fai12]) has shown that a TCP sender using CWV is able to use available capacity on a shared path after an idle period. This can benefit some applications, especially over long delay paths, when compared to the slow-start restart specified by standard TCP. However, CWV would only benefit an application if the idle period were less than several Retransmission Time Out (RTO) intervals [RFC6298], since the behaviour would otherwise be the same as for standard TCP, which resets the cwnd to the RTCP Restart Window (RW) after this period.
Experience with RFC 2861 suggests that although the CWV method benefited the network in a rate-limited scenario (reducing the probability of network congestion), the behaviour was too conservative for many common rate-limited applications. This mechanism did not therefore offer the desirable increase in application performance for rate-limited applications and it is unclear whether applications actually use this mechanism in the general Internet.
It is therefore concluded that CWV, as defined in RFC2681, was often a poor solution for many rate-limited applications. It had the correct motivation, but had the wrong approach to solving this problem.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
The document assumes familiarity with the terminology of TCP congestion control [RFC5681].
The following new terminology is introduced:
cwnd-limited: A TCP flow that has sent the maximum number of segments permitted by the cwnd, where the application utilises the allowed sending rate (see section 4.5.2).
pipeACK sample: A measure of the volume of data acknowledged by the network within an RTT.
pipeACK variable: A variable that measures the available capacity using the set of pipeACK samples.
pipeACK Sampling Period: The maximum period that a measured pipeACK sample may influence the pipeACK variable.
Non-validated phase: The phase where the cwnd reflects a previous measurement of the available path capacity.
Non-validated period, NVP: The maximum period for which cwnd is preserved in the non-validated phase.
Rate-limited: A TCP flow that does not consume more than one half of cwnd, and hence operates in the non-validated phase.
Validated phase: The phase where the cwnd reflects a current estimate of the available path capacity.
This section proposes an update to the TCP congestion control behaviour during a rate-limited period. The new method permits a TCP sender to preserve the cwnd when an application becomes idle or when the sender is unable to send at the maximum rate permitted by the cwnd (the non-validated period, NVP, see section 5). The period where actual usage is less than allowed by cwnd, is named as the non-validated phase. This method allows an application to resume transmission at a previous rate without incurring the delay of slow-start. However, if the TCP sender experiences congestion using the preserved cwnd, it is required to immediately reset the cwnd to an appropriate value specified by the method. If a sender does not take advantage of the preserved cwnd within the NVP, the value of cwnd is reduced, ensuring the value better reflects the capacity that was recently actually used.
It is expected that this update will satisfy the requirements of many rate-limited applications and at the same time provide an appropriate method for use in the Internet. It also reduces the incentive for an application to send data simply to keep transport congestion state. (This is sometimes known as "padding").
The new method does not differentiate between times when the sender has become idle or rate-limited. This is partly a response to recognition that some applications wish to transmit at a rate less than allowed by the sender cwnd, and that it can be hard to make a distinction between rate-limited and idle behaviour. This is expected to encourage applications and TCP stacks to use standards-based congestion control methods. It may also encourage the use of long-lived connections where this offers benefit (such as persistent http).
The method is specified in following subsections.
[RFC5681] defines a variable, FlightSize, that indicates the amount of outstanding data in the network. This is assumed to be equal to the value of Pipe calculated based on the pipe algorithm [RFC3517]. In RFC5681 this value is used during loss recovery, whereas in this method a new variable "pipeACK" is introduced to measure the acknowledged size of the pipe, which is used to determine if the sender has validated the cwnd.
A sender determines a pipeACK sample by measuring the volume of data that was acknowledged by the network over the period of a measured Round Trip Time (RTT). Using the variables defined in [RFC3517], a value could be measured by caching the value of HighACK and after one RTT measuring the difference between the cached HighACK value and the current HighACK value. Other equivalent methods may be used.
A sender is not required to continuously update the pipeACK variable after each received ACK, but SHOULD perform a pipeACK sample at least once per RTT when it has sent unacknowledged segments.
The pipeACK variable MAY consider multiple pipeACK samples over the pipeACK Sampling Period. The value of the pipeACK variable MUST NOT exceed the maximum (highest value) within the sampling period. This specification defines the pipeACK Sampling Period as Max(3*RTT, 1 second). This period enables a sender to compensate for large fluctuations in the sending rate, where there may be pauses in transmission, and allows the pipeACK variable to reflect the largest recently measured pipeACK sample.
When no measurements are available, the pipeACK variable is set to the "undefined value". This value is used to inhibit entering the nonvalidated phase until the first new measurement of a pipeACK sample.
The method RECOMMENDS that the TCP SACK option [RFC3517] is enabled. This allows the sender to more accurately determine the number of missing bytes during the loss recovery phase, and using this method will result in a higher cwnd following loss.
A sender starts a TCP connection in the Validated phase and initialises the pipeACK variable to the "undefined" value. This value inhibits use of the value in cwv calculations.
The updated method creates a new TCP sender phase that captures whether the cwnd reflects a validated or non-validated value. The phases are defined as:
The value 1/2 was selected to reduce the effects of variations in the pipeACK variable, and to allow the sender some flexibility in when it sends data.
A TCP sender MUST enter the non-validated phase when the pipeACK is less than (1/2)*cwnd.
A TCP sender that enters the non-validated phase will preserve the cwnd (i.e., this neither grows nor reduces while the sender remains in this phase). If the sender receives an indication of congestion (loss or Explicit Congestion Notification, ECN, mark [RFC3168]) it uses the method described below. The phase is concluded after a fixed period of time (the NVP, as explained in section 4.4.3) or when the sender transmits sufficient data so that pipeACK > (1/2)*cwnd (i.e. it is no longer rate-limited).
The behaviour in the non-validated phase is specified as:
The cwnd-limited behaviour may be triggered during a transient condition that occurs when a sender is in the non-validated phase and receives an ACK that acknowledges received data, the cwnd was fully utilised, and more data is awaiting transmission than may be sent with the current cwnd. The sender is then allowed to use the standard method to increase the cwnd. (Note, if the sender suceeds in sending these new segments, the updated cwnd and pipeACK variables will eventually result in a transition to the validated phase.)
Reception of congestion feedback while in the non-validated phase is interpreted as an indication that it was inappropriate for the sender to use the preserved cwnd. The sender is therefore required to quickly reduce the rate to avoid further congestion. Since the cwnd does not have a validated value, a new cwnd value must be selected based on the utilised rate.
A sender that detects a packet-drop, or receives an indication of an ECN marked packet, MUST record the current FlightSize in the variable LossFlightSize and MUST calculate a safe cwnd for loss recovery using the method below:
cwnd = (Max(pipeACK,LossFlightSize))/2.
If there is a valid pipeACK value, the new cwnd is adjusted to reflect that a nonvalidated cwnd may be larger than the actual FlightSize, or recently used FlightSize (recorded in pipeACK). The updated cwnd therefore prevents overshoot by a sender significantly increasing its transmission rate during the recovery period.
At the end of the recovery phase, the TCP sender MUST reset the cwnd using the method below:
cwnd = (Max(pipeACK,LossFlightSize) - R)/2.
Where, R is the volume of data that was retransmitted during the recovery phase. This follows the method proposed for Jump Start [Liu07]. The inclusion of the term R makes an adjustment more conservative than standard TCP. (This is required, since the sender may have sent more segments than a standard TCP sender would have done. The additional reduction is beneficial when the LossFlightSize significantly overshoots the available path capacity incurring significant loss, for instance an intense traffic burst following a non-validated period.)
If the sender implements a method that allows it to identify the number of ECN-marked segments within a window that were observed by the receiver, the sender SHOULD use the method above, further reducing R by the number of marked segments.
The sender MUST also re-initialise the pipeACK variable to the "undefined" value. This ensures that standard TCP methods are used immediately after completing loss recovery until a new pipeACK value can be determined.
ssthresh is adjusted using the standard TCP method.
TCP congestion control allows a sender to accumulate a cwnd that would allow it to send a burst of segments with a total size up to the difference between the FlightsSize and cwnd. Such bursts can impact other flows that share a network bottleneck and/or may induce congestion when buffering is limited.
Various methods have been proposed to control the sender burstiness [Hug01], [All05]. For example, TCP can limit the number of new segments it sends per received ACK . This is effective when a flow of ACKs is received, but can not be used to control a sender that has not send appreciable data in the previous RTT [All05].
This document recommends using a method to avoid line-rate bursts after an idle or rate-limited period when there is less reliable information about the capacity of the network path: A TCP sender in the non-validated phase SHOULD control the maximum burst size, e.g. using a rate-based pacing algorithm in which a sender paces out the cwnd over its estimate of the RTT, or some other method, to prevent many segments being transmitted contiguously at line-rate. The most appropriate method(s) to implement pacing depend on the design of the TCP/IP stack, speed of interface and whether hardware support (such as TCP Segment Offload, TSO) is used. The present document does not recommend any specific method.
An application that remains in the non-validated phase for a period greater than the NVP is required to adjust its congestion control state. If the sender exits the non-validated phase after this period, it MUST update the ssthresh:
ssthresh = max(ssthresh, 3*cwnd/4).
(This adjustment of ssthresh ensures that the sender records that it has safely sustained the present rate. The change is beneficial to rate-limited flows that encounter occasional congestion, and could otherwise suffer an unwanted additional delay in recovering the sending rate.)
The sender MUST then update cwnd to be not greater than:
cwnd = max((1/2)*cwnd, IW).
Where IW is the appropriate TCP initial window, used by the TCP sender (e.g. [RFC5681]).
(This adjustment ensures that sender responds conservatively at the end of the non-validated phase by reducing the cwnd to better reflect the current rate of the sender. The cwnd update does not take into account FlightSize or pipeACK value because these values only reflect historical data and do not reflect the current sending rate.)
This section provides informative examples of implementation methods. Implementations may choose to use other methods that comply with the normative requirements.
A pipeACK sample may be measured once each RTT. This reduces the sender processing burden for calculating after each acknowledgement and also reduces storage requirements at the sender.
Since application behaviour can be bursty using CWV, it may be desirable to implement a maximum filter to accumulate the measured values so that the pipeACK variable records the largest pipeACK sample within the pipeACK Sampling Period. One simple way to implement this is to divide the pipeACK Sampling Period into several (e.g. 5) equal length measurement periods. The sender then records the start time for each measurement period and the highest measured pipeACK sample. At the end of the measurement period, any measurement(s) that are older than the pipeACK Sampling Period are discarded. The pipeACK variable is then assigned the largest of the set of the highest measured values.
+----------+----------+ +----------+---......
| Sample A | Sample B | No | Sample C | Sample D
| | | Sample | |
| |\ 5 | | | |
| | | | | | /\ 4 |
| | | | |\ 3 | | | \ |
| | \ | | \--- | | / \ | /| 2
|/ \------| - | | / \------/ \...
+----------+---------\+----/ /----+/---------+-------------> Time
<------------------------------------------------|
Sampling Period Current Time
Figure 1: Example of measuring pipeACK samples
Figure 1 shows an example of how measurement samples may be collected. At the time represented by the figure new samples are being accumulated into sample D. Three previous samples also fall within the pipeACK Sampling Period: A, B, and C. There was also a period of inactivity between samples B and C during which no measurements were taken. The current value of the pipeACK variable will be 5, the maximum across all samples.
After one further measurement period, Sample A will be discarded, since it then is older than the pipeACK Sampling Period and the pipeACK variable will be recalculated, Its value will be the larger of Sample C or the final value accumulated in Sample D.
Note that the pipeACK Sampling Period and the NVP period do not necessarily require a new timer to be implemented. An alternative is to record a timestamp when the sender enters the NVP. Each time a sender transmits a new segment, this timestamp may be used to determine if the NVP period has expired. If the period expires, the sender may take into account how many units of the NVP period have passed and make one reduction (as defined in section 4.4.3) for each NVP period.
A method is required to detect the cwnd-limited condition (see section 4.4). This is used to detect a condition where a sender in the non-validated phase receives an ACK, but the size of cwnd prevents sending more new data.
In simple terms this condition is true only when the TCP sender's FlightSize is equal to or larger than the cwnd. However, an implementation must consider other constraints on the way in which cwnd variable is used, for instance the need to support methods such as the Nagle Algorithm and TCP Segment Offload (TSO). This can result in a sender becoming cwnd-limited when the cwnd is nearly, rather than completely, equal to the FlightSize.
This section documents the rationale for selecting the maximum period that cwnd may be preserved, known as the non-validated period, NVP.
Limiting the period that cwnd may be preserved avoids undesirable side effects that would result if the cwnd were to be kept unnecessarily high for an arbitrary long period, which was a part of the problem that CWV originally attempted to address. The period a sender may safely preserve the cwnd, is a function of the period that a network path is expected to sustain the capacity reflected by cwnd. There is no ideal choice for this time.
A period of five minutes was chosen for this NVP. This is a compromise that was larger than the idle intervals of common applications, but not sufficiently larger than the period for which the capacity of an Internet path may commonly be regarded as stable. The capacity of wired networks is usually relatively stable for periods of several minutes and that load stability increases with the capacity. This suggests that cwnd may be preserved for at least a few minutes.
There are cases where the TCP throughput exhibits significant variability over a time less than five minutes. Examples could include wireless topologies, where TCP rate variations may fluctuate on the order of a few seconds as a consequence of medium access protocol instabilities. Mobility changes may also impact TCP performance over short time scales. Senders that observe such rapid changes in the path characteristic may also experience increased congestion with the new method, however such variation would likely also impact TCP’s behaviour when supporting interactive and bulk applications.
Routing algorithms may modify the network path, disrupting the RTT measurement and changing the capacity available to a TCP connection, however such changes do not often occur within a time frame of a few minutes.
The value of five minutes is therefore expected to be sufficient for most current applications. Simulation studies (e.g. [Bis11]) also suggest that for many practical applications, the performance using this value will not be significantly different to that observed using a non-standard method that does not reset the cwnd after idle.
Finally, other TCP sender mechanisms have used a 5 minute timer, and there could be simplifications in some implementations by reusing the same interval. TCP defines a default user timeout of 5 minutes [RFC0793] i.e. how long transmitted data may remain unacknowledged before a connection is forcefully closed.
General security considerations concerning TCP congestion control are discussed in [RFC5681]. This document describes an algorithm that updates one aspect of the congestion control procedures, and so the considerations described in RFC 5681 also apply to this algorithm.
There are no IANA considerations.
The authors acknowledge the contributions of Dr I Biswas, Mr Ziaul Hossain in supporting the evaluation of CWV and for their help in developing the mechanisms proposed in this draft. We also acknowledge comments received from the Internet Congestion Control Research Group, in particular Yuchung Cheng, Mirja Kuehlewind, and Joe Touch. This work was part-funded by the European Community under its Seventh Framework Programme through the Reducing Internet Transport Latency (RITE) project (ICT-317700).
RFC-Editor note: please remove this section prior to publication.
RFC-Editor note: please remove this section prior to publication.
There are several issues to be discussed more widely:
• There is potential performance loss in loss of a short burst (off list with M Allman)
• There is potential interaction with TCP Control Block Sharing(M Welzl)
RFC-Editor note: please remove this section prior to publication.
Draft 03 was submitted to ICCRG to receive comments and feedback.
Draft 04 contained the first set of clarifications after feedback:
Draft 05 contained various updates:
Draft 06 contained various updates:
WG draft 00 contained various updates:
WG draft 01 contained:
WG draft 02 contained:
WG draft 03 contained:
WG draft 04 contained:
WG draft 05 contained:
| [RFC0793] | Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. |
| [RFC2861] | Handley, M., Padhye, J. and S. Floyd, "TCP Congestion Window Validation", RFC 2861, June 2000. |
| [RFC6298] | Paxson, V., Allman, M., Chu, J. and M. Sargent, "Computing TCP's Retransmission Timer", RFC 6298, June 2011. |
| [RFC3168] | Ramakrishnan, K., Floyd, S. and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. |
| [RFC3517] | Blanton, E., Allman, M., Fall, K. and L. Wang, "A Conservative Selective Acknowledgment (SACK)-based Loss Recovery Algorithm for TCP", RFC 3517, April 2003. |
| [RFC5681] | Allman, M., Paxson, V. and E. Blanton, "TCP Congestion Control", RFC 5681, September 2009. |
| [RFC6928] | Chu, J., Dukkipati, N., Cheng, Y. and M. Mathis, "Increasing TCP's Initial Window", RFC 6928, April 2013. |