| TCPM Working Group | C. Gomez |
| Internet-Draft | UPC |
| Intended status: Informational | J. Crowcroft |
| Expires: September 8, 2020 | University of Cambridge |
| March 7, 2020 |
Delayed Acknowledgments Suppression: Problem Statement, Requirements and Analysis of Potential Solutions
draft-gomez-tcpm-delack-suppr-reqs-00
Delayed Acknowledgments (ACKs) allow reducing protocol overhead in many scenarios. However, in some cases, Delayed ACKs may significantly degrade network and device performance in terms of link utilization, latency, memory usage and/or energy consumption. This document presents the problem statement regarding Delayed ACKs suppression. The document discusses the scenarios and use cases in which Delayed ACKs suppression offers advantages. Then, requirements for a potential solution are derived. Finally, a number of potential solutions are discussed, based on the requirements, and also considering pros and cons in each case.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 8, 2020.
Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Delayed Acknowledgments (ACKs) were specified with the aim to reduce protocol overhead [RFC1122]. With Delayed ACKs, a TCP delays sending an ACK by up to 500 ms (often 200 ms, with lower values in recent implementations such as ~50 ms also reported), and typically sends an ACK for at least every second segment received in a stream of full-sized segments. This allows combining several segments into a single one (e.g. the application layer response to an application layer data message, and the corresponding ACK), and it also saves up to one of every two ACKs under many traffic patterns (e.g. bulk transfers). The "SHOULD" requirement level for implementing Delayed ACKs in RFC 1122, along with its expected benefits, has led to a widespread deployment of this mechanism.
However, there exist traffic patterns and scenarios for which Delayed ACKs can actually be detrimental to performance. When a segment carrying a message of a size up to one Maximum Segment Size (MSS) is transferred, if the message does not elicit an application-layer response, and a second data segment is not transferred earlier than the Delayed ACK timeout, the ACK is unnecessarily delayed, with a number of negative consequences. Furthermore, there may be reasons to allow a sender communicate the ACK ratio to be used in a TCP connection, and thus dynamically override (or restore) use of Delayed ACKs at the receiver.
This document presents the problem statement regarding Delayed ACKs suppression. The document discusses the scenarios and use cases in which Delayed ACKs suppression offers advantages. Then, requirements for a potential solution are derived. Finally, a number of potential solutions are discussed, based on the requirements, and also considering pros and cons in each case.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
This section provides scenarios and use cases where performance issues arise due to Delayed ACKs.
During slow start, the congestion window (cwnd) increases by up to Sender Maximum Segment Size (SMSS) upon receipt of an ACK covering new data [RFC5681]. However, use of Delayed ACKs reduces the amount of ACKs received by the sender, thus reducing the rate of cwnd growth, increasing transfer time and reducing throughput, when compared with sending an ACK for each incoming data segment. Furthermore, Delayed ACKs preclude using sender behaviors intended to 'more rapidly get up to speed during slow-start without overshoot' [I-D.kuehlewind-tcpm-accurate-ecn] (see -03 of the referenced document).
When the Nagle algorithm is used, in some cases the sender may be prevented from sending more data while awaiting a delayed ACK. In some high bit rate environment (e.g. Gigabit Ethernet) use cases, such a delay may be very large, and link utilitzation may be dramatically reduced, since the Delayed ACK timeout may be several orders of magnitude greater than the Round Trip Time (RTT) [RFC8490].
Delayed ACKs are also detrimental in Internet of Things (IoT) scenarios, where TCP is being increasingly used [I-D.ietf-lwig-tcp-constrained-node-networks]. Many IoT devices, such as sensors, transfer small messages (e.g. containing sensor readings) rather infrequently, therefore if the receiver uses Delayed ACKs, the ACK will often be unnecessarily delayed. The sender cannot release the memory resources associated to a transferred data segment until the ACK is received and processed. This may be a problem for many IoT devices, which are typically memory-constrained, and may even lead to subsequent packet drops if their scarce memory resources are blocked while awaiting an ACK. Moreover, if the IoT device uses a radio interface for communication, in some scenarios Delayed ACKs will lead to increased energy consumption (e.g. with the radio interface of the device staying in receive mode while awaiting the ACK). Since many IoT devices run on small batteries, the device lifetime may significantly decrease. Furthermore, the delay suffered by the ACK may interact negatively with layer two mechanisms, especially in wireless network technologies where devices remain in low-power states for long intervals [RFC8352], potentially leading to a further exacerbated delay (by even one or more orders of magnitude).
In some scenarios, it may be desirable to enable ACK transmission behaviors beyond the classic ones. For example, it may be beneficial to apply congestion control to ACKs [RFC5690]. For example, reducing the amount of ACKs on a congested reverse path may allow alleviating congestion on that path, with minimal impact on a relatively independent forward path.
This section provides the requirements for a potential solution to enable Delayed ACKs suppression.
An assumption is that the sender knows when Delayed ACKs operation should be overriden. For example, the sender may know in advance the pattern of the traffic it will generate, or it may know whether an application-layer response will be sent by the receiving endpoint upon reception of a given message. Therefore, Delayed ACKs suppression has to be sender-triggered.
One approach that cannot be recommended as a general solution for Delayed ACKs suppression is (permanently) disabling Delayed ACKs at the receiving TCP. In fact, the latter may interact with a wide variety of devices and many of those may still benefit from the advantages of Delayed ACKs. In addition, in some cases, a sender may offer a mixed traffic pattern comprising single data segments that will lead to unnecessarily delayed ACKs, with other data segments upon which Delayed ACKs will act as intended. Therefore, the solution has to be provided at a per-segment granularity.
Since the presented problem is about low performance in various scenarios, another requirement for the solution is to minimize incurring overhead in terms of header size increase or additional packets sent. For example, in IoT scenarios, every additional communicated byte consumes scarce resources (e.g. energy, bandwidth and computational resources).
Another benefit of keeping a low header/message overhead is alleviating the processing workload of a receiving TCP.
For many of the scenarios and use cases described in Section 2, an ACK ratio of 1 (i.e. a receiver sending one ACK per incoming data segment) would solve the mentioned issues. However, the ability of enforcing a generic ACK ratio (including values different from 1 and 2) allows to enable a wider range of ACK behaviors, which may support congestion control for ACKs, sender behaviors not based on ACK-clocking, etc.
Deployment of new functionality for TCP faces the risk of packets being discarded by existing middleboxes upon detection of unexpected or discouraged formats, header field values or even traffic patterns.
A solution for Delayed ACK suppression should offer relatively good middlebox traversal (to the extent possible).
A solution for Delayed ACKs suppression must ensure that normal Delayed ACKs operation is in force by default, and also once temporary suppression of Delayed ACKs operation needs to end.
A solution for Delayed ACKs suppression should not reduce the space of existing TCP functionality.
A solution for Delayed ACKs suppression should not pose significant risk of preventing future TCP development. If an available resource (e.g. a reserved bit of the TCP header, a new TCP option, etc.) is used by a solution, careful analysis must be carried out regarding the risks and benefits of using such resource.
Delayed ACKs suppression might be achieved by using workarounds, such as implementation techniques that may produce the desired effect. However, such approaches may be suboptimal regarding implementation cleanliness, and may entail other performance issues (see section 5.6).
This section enumerates and discusses potential solutions that might be considered to enable Delayed ACK suppression. The list of solutions is not necessarily comprehensive. This section intends to illustrate the trade-offs that arise when considering potential Delayed ACK suppression solutions. (Note: the analysis needs to be completed for many of the solutions below.)
In Acknowledgment Congestion Control (AckCC) [RFC5690], the sender tells the receiver the ACK ratio R to use, where the receiver sends one ACK per R data packets received. AckCC defines a 2-byte "TCP ACK Congestion Control Permitted Option" for negotiating use of AckCC, whereas it defines a 3-byte "ACK ratio TCP option" to communicate the ACK Ratio value from the sender to the receiver.
Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed).
Tail Loss Probe (TLP) [I-D.ietf-tcpm-rack] is intended to avoid RTO-expiration-based retransmission when tail loss occurs by inducing additional ACKs at the receiver. This is achieved by sending a probe segment after a probe time-out (PTO) when data have been sent but not confirmed. Of course, this means sending a whole new packet to trigger ACKs, which adds significant overhead.
This approach might offer good middlebox traversal (to be confirmed).
One solution that has been proposed for Delayed ACKs suppresion is called 'TCP ACK Pull' [I-D.gomez-tcpm-ack-pull].
TCP ACK Pull defines the AKP flag as bit number 6 of the 13th byte of the TCP header. When a TCP sender needs a data segment to be acknowledged by the receiving TCP without additional delay, the sender sets the AKP flag of the data segment TCP header. Upon reception of a segment with the AKP flag set, a conforming receiving TCP behaves accordingly by sending the corresponding ACK without additional delay.
This solution would entail zero header or message overhead. However, it would consume a TCP header bit, leaving only two available TCP header reserved bits. A question is thus whether one TCP header bit should be dedicated to this purpose or not.
Middlebox traversal characteristics of bit 6 of the TCP header need to be assessed.
Another approach relies on defining a new option-kind-only TCP option with the same semantics as the AKP flag, which might be called 'ACK Pull Option' or 'AKP Option'.
This solution would consume an available TCP Option Kind number. However, most of the 256 numbers in the TCP Option Kind number space are currently available. Therefore, consuming one such number does not appear to significantly limit future TCP development.
The header overhead of the AKP Option is one byte.
Middlebox traversal of a new TCP option is often regarded as 'bad' (to be confirmed).
Another approach that might be used to enable Delayed ACKs suppression is based on reusing existing TCP header fields. For example, use of the Urgent pointer has been suggested (e.g. by reserving 3 of its 16 bits to encode an ACK ratio exponent that may be communicated by the sender to the receiver), when URG=0. A problem with this approach is that the semantics of the reused TCP header field may become overloaded. Therefore, in some cases either the original intended use of the reused TCP header field may become limited, or if it prevails, then Delayed ACKs suppression might not always be available for use.
Middlebox traversal characteristics of this approach might be relatively good (to be confirmed).
One approach that allows eliciting an immediate ACK after sending a data segment is sending a subsequent segment carrying a previously acknowledged data byte. However, in addition to the inefficiency of sending a byte that has previously been sent, this approach may require the transmission of a new packet (even carrying a single byte of data payload) just for that purpose, which represents significant overhead. Furthermore, sending a previously sent byte is not a clean solution from an implementation perspective.
Another workaround intended to trigger an immediate ACK from the receiving TCP, which is used in the Contiki operating system (a popular operating system for constrained devices in IoT scenarios) is splitting the data to be sent into two segments of smaller size. A standard compliant TCP receiver will acknowledge the second MSS of data. However, this 'split hack' may not always work since a TCP receiver is required to acknowledge every second full-sized segment, but not two consecutive small segments. Furthermore, the overhead of sending two IP packets instead of one is another downside of the 'split hack'.
The next table summarizes whether the different solutions presented in Section 4 are able to satisfy the requirements stated in Section 4.
+-------+-------+-------+-------+-------+------+------+
| Per- | Over- |Generic|Middle-|Impact |Impact|Hack |
|segment| head |ACK rat|box tr.|current|future|Avoid.|
+------------+-------+-------+-------+-------+-------+------+------+
| ACKcc | Yes | Low | Yes | Bad? | No | Low | Yes |
+------------+-------+-------+-------+-------+-------+------+------+
| TLP | No | High | No | Good | No | No | Yes |
+------------+-------+-------+-------+-------+-------+------+------+
| AKP flag | Yes | No | Yes | ? | No |Med/Hi| Yes |
+------------+-------+-------+-------+-------+-------+------+------+
| AKP option | Yes | Low | Yes | Bad? | No | Low | Yes |
+------------+-------+-------+-------+-------+-------+------+------+
|Reuse fields| Yes | No | Yes | Good? | Yes | ? | Yes |
+------------+-------+-------+-------+-------+-------+------+------+
| Hacks | ? |Med/Hig| No | Good? | No | No | No |
+------------+-------+-------+-------+-------+-------+------+------+
Note: all considered potential solutions satisfy the following requirements:
i) sender-triggered Delayed ACKs suppression, and ii) safe return to normal
Delayed ACKs operation.
Figure 1: Summary of potential solutions for Delayed ACKs suppression.
TBD
Bob Briscoe, Jonathan Morton, Richard Scheffenegger, Michael Tuexen and Jana Iyengar provided useful input for this document.
Stuart Cheshire, Ted Lemon, Michael Scharf, and Christoph Paasch participated in a discussion that was seminal to the TCP ACK Pull proposal, which eventually led to this document.
Carles Gomez has been funded in part by the Spanish Government (Ministerio de Ciencia, Innovacion y Universidades) through Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya 2017 SGR 376.
| [RFC1122] | Braden, R., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, DOI 10.17487/RFC1122, October 1989. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC5681] | Allman, M., Paxson, V. and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009. |
| [RFC5690] | Floyd, S., Arcia, A., Ros, D. and J. Iyengar, "Adding Acknowledgement Congestion Control to TCP", RFC 5690, DOI 10.17487/RFC5690, February 2010. |
| [I-D.gomez-tcpm-ack-pull] | Gomez, C. and J. Crowcroft, "TCP ACK Pull", Internet-Draft draft-gomez-tcpm-ack-pull-01, November 2019. |
| [I-D.ietf-lwig-tcp-constrained-node-networks] | Gomez, C., Crowcroft, J. and M. Scharf, "TCP Usage Guidance in the Internet of Things (IoT)", Internet-Draft draft-ietf-lwig-tcp-constrained-node-networks-09, November 2019. |
| [I-D.ietf-tcpm-rack] | Cheng, Y., Cardwell, N., Dukkipati, N. and P. Jha, "RACK: a time-based fast loss detection algorithm for TCP", Internet-Draft draft-ietf-tcpm-rack-07, January 2020. |
| [I-D.kuehlewind-tcpm-accurate-ecn] | Briscoe, B., Kuehlewind, M. and R. Scheffenegger, "More Accurate ECN Feedback in TCP", Internet-Draft draft-kuehlewind-tcpm-accurate-ecn-05, October 2015. |
| [RFC8352] | Gomez, C., Kovatsch, M., Tian, H. and Z. Cao, "Energy-Efficient Features of Internet of Things Protocols", RFC 8352, DOI 10.17487/RFC8352, April 2018. |
| [RFC8490] | Bellis, R., Cheshire, S., Dickinson, J., Dickinson, S., Lemon, T. and T. Pusateri, "DNS Stateful Operations", RFC 8490, DOI 10.17487/RFC8490, March 2019. |