Internet-Draft sceonetwotests July 2019
Heist, et al. Expires 4 January 2020 [Page]
Workgroup:
Transport Working Group
Internet-Draft:
draft-heist-tsvwg-sce-one-and-two-flow-tests-00
Published:
Intended Status:
Informational
Expires:
Authors:
P. Heist
R.W. Grimes
J. Morton

Some Congestion Experienced One and Two-Flow Tests

Abstract

This note presents one and two-flow test results for the SCE (Some Congestion Experienced) reference implementation. These tests are not intended to be a comprehensive real-world evaluation of SCE, but an illustration of SCE's influence on basic TCP metrics in a controlled environment.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 4 January 2020.

▲

Table of Contents

1. Introduction

SCE provides early and proportional feedback to the CC (congestion control) algorithms for transport protocols, including but not limited to TCP. The [sce-repo] is a Linux kernel modified to support SCE, including:

In this note we run one and two-flow TCP tests across a range of simulated path bandwidths and RTTs. One-flow tests measure SCE's impact on TCP throughput and TCP RTT. Two-flow tests evaluate fairness between and among several SCE and non-SCE TCP implementations, while making several adjustments to Cake's SCE and fair queueing parameters.

It is recognized that these tests do not simulate real-world conditions, and will not be an indication of how SCE will perform in all situations. However, they serve as fundamental tests for the SCE reference implementation. Once the behavior in these tests is well understood and theory and experiment are in agreement, additional complexity can be added to the test procedures with the confidence that the reference implementation's fundamentals are sound.

2. Terminology

The following terminology is used in this document:

3. Test Tools and Environment

The [Flent] tool is used for all tests. Flent uses netperf for its TCP tests, and allows for test batches, plotting, the recording of results and the collection of metadata in JSON format [RFC8259]. Flent both captures the measured TCP throughput from netperf, and simultaneously uses the [ss] tool in Linux to passively monitor TCP RTT.

All tests are performed using a three node dumbbell topology:

  +--------+              +-----------+              +----------+
  | Sender |--------------| Middlebox |--------------| Receiver |
  +--------+              +-----------+              +----------+
Figure 1: Test topology

All nodes run the SCE reference implementation kernel as of commit 56915a82 (2019-06-20), and are connected directly via Gigabit Ethernet.

4. Tests

The tests are implemented with a Flent batch file to drive netperf and re-configure Cake and netem on the middlebox with various parameters. Scripts post-process the results and create csv and markdown tables for external use, including by this document.

Unless otherwise mentioned, measurements are obtained from TCP flows from start to finish, not at steady state. This allows for some discussion of the differences in TCP CC algorithm behavior during slow start and congestion avoidance. Typically, each test is run long enough to obtain a reasonable approximation of steady state throughput, but in a few high BDP cases slow start accounts for a significant portion of the test length. When relevant to the analysis of the results, this is stated in the text.

5. Results and Analysis

5.1. One-Flow Tests

The goal of the one-flow tests is to analyze the impact of SCE on the TCP throughput and TCP RTT of single TCP flows across a range of simulated path bandwidths and RTTs. What follows is an analysis of the results. See Section a.1 and Section a.2 for the raw results for TCP throughput and TCP RTT, respectively.

5.1.1. Reno-SCE TCP Throughput

The following table shows the difference in TCP throughput for Reno-SCE vs Reno across the tested range of simulated path bandwidths and RTTs:

Table 1: Difference in TCP Throughput (reno-sce - reno), normalized to Cake-limited Bandwidth; Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.000 0.000 0.000 0.000 0.000 -0.100 -0.180 -0.160
1 0.000 0.000 0.000 0.000 -0.110 -0.210 -0.280 -0.280
2 0.000 0.000 -0.010 -0.120 -0.210 -0.375 -0.370 -0.405
5 0.000 -0.052 -0.142 -0.228 -0.178 -0.032 -0.006 0.016
10 0.000 -0.071 -0.139 -0.143 -0.046 0.062 0.116 0.168
25 0.000 -0.003 -0.006 0.010 0.073 0.143 0.196 0.259
50 0.000 0.000 0.001 0.029 0.102 0.169 0.235 0.272
100 0.000 0.000 0.004 0.043 0.103 0.215 0.238 0.289

From the above TCP throughput differences we can observe:

  1. Improved utilization for SCE at sufficiently high BDPs. This is due to SCE's proportional congestion signals, which can significantly reduce the classic Reno throughput sawtooth by making drops or CE marks rare to non-existent. The utilization improvement increases with BDP largely because the TCP window recovery time after a drop or CE mark increases with BDP, deepening the sawtooth.

  2. Significant under-utilization at bandwidths <= 10Mbit, which tends to worsen as path RTT increases. Investigation is underway as to the source of this. These drops in utilization are however also accompanied by drops in TCP RTT (see below).

5.1.2. Reno-SCE TCP RTT

The following table shows the difference in TCP RTT for Reno-SCE vs Reno across the tested range of simulated path bandwidths and RTTs:

Table 2: Difference in TCP RTT (reno-sce - reno); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 -88.41 -87.51 -88.43 -88.59 -62.37 -52.56 -42.23 -31.93
1 -42.30 -41.94 -42.65 -33.03 -32.92 -27.92 -19.13 -13.20
2 -29.09 -27.67 -22.19 -19.23 -14.54 -10.12 -3.66 -1.50
5 -9.18 -9.47 -7.51 -8.37 -5.63 -3.49 -1.69 -1.24
10 -2.46 -2.75 -2.95 -3.91 -2.44 -1.42 -0.54 -0.41
25 -1.87 -2.09 -2.53 -1.74 -0.42 0.01 0.19 -0.01
50 -2.00 -2.18 -1.48 -0.31 0.59 0.98 0.86 0.57
100 -1.87 -1.67 -1.04 0.06 0.95 1.42 1.41 1.15

From the above TCP RTT differences we can observe:

  1. Significant reductions in TCP RTT for Reno-SCE at most BDPs. SCE's proportional congestion signals are aiding the sender in managing queue lengths.

  2. Greater reductions in TCP RTT at lower path bandwidths and RTTs. This is because Reno-linear growth becomes relatively larger as the path bandwidth decreases. This results in greater queue growth in the interval before AQM activates, and a subsequently longer drain time after congestion is signaled.

  3. A slight increase in TCP RTT for SCE at high BDPs. This has been observed to have a possible connection to the ESCE (Echo Some Congestion Experienced) feedback strategy implemented in on the TCP receive side. Research and optimization in this area is ongoing.

5.1.3. DCTCP-SCE TCP Throughput

The following table shows the difference in TCP throughput for DCTCP-SCE vs DCTCP across the tested range of simulated path bandwidths and RTTs:

Table 3: Difference in TCP Throughput (dctcp-sce - dctcp), normalized to Cake-limited Bandwidth; Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.000 0.000 0.000 0.000 0.000 -0.120 -0.220 -0.400
1 0.000 0.000 0.000 0.000 -0.120 -0.260 -0.420 -0.570
2 0.000 0.000 -0.015 -0.130 -0.285 -0.505 -0.565 -0.705
5 0.000 -0.038 -0.150 -0.270 -0.374 -0.588 -0.652 -0.620
10 0.000 -0.087 -0.197 -0.265 -0.266 -0.257 -0.271 -0.134
25 -0.006 -0.060 -0.131 -0.167 -0.154 -0.133 -0.067 0.110
50 0.000 -0.001 -0.006 -0.008 0.005 0.012 0.143 0.180
100 0.000 0.007 0.018 0.031 0.040 0.087 0.151 0.246

From the above TCP throughput differences we can observe:

  1. Improved utilization for SCE at higher BDPs. At first glance we might assume that this is due to SCE's feedback signals improving upon DCTCP's window recovery time, as with Reno, but another significant part of this increase is due to DCTCP-SCE's steeper ramp during slow start, and a test length that is short relative to the time spent in slow start. The test is 300 seconds long, and at a path bandwidth of 100Mbit and RTT of 160ms, DCTCP takes a full 225 seconds to ramp up to the BDP, while DCTCP-SCE, with its steeper ramp, takes only 80 seconds. That said, steady state throughput at this BDP is around 87Mbit for DCTCP, and around 96Mbit for DCTCP-SCE, an increase of around 10%, so the proportional congestion control signals also play a part in increasing utilization.

  2. Significant under-utilization at bandwidths <= 25Mbit, which tends to worsen as path RTT increases. Investigation is underway as to the source of this. These drops in utilization are however also accompanied by drops in TCP RTT (see below).

5.1.4. DCTCP-SCE TCP RTT

The following table shows the difference in TCP RTT for DCTCP-SCE vs DCTCP across the tested range of simulated path bandwidths and RTTs:

Table 4: Difference in TCP RTT (dctcp-sce - dctcp); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 -82.57 -81.21 -81.37 -81.53 -60.53 -70.29 -74.76 -54.33
1 -41.78 -42.31 -40.43 -30.98 -35.26 -39.27 -29.95 -15.62
2 -21.94 -20.29 -20.86 -17.58 -19.93 -12.70 -7.37 -3.97
5 -6.65 -7.27 -7.08 -6.45 -5.57 -3.59 -2.19 -1.31
10 -1.85 -2.11 -2.85 -2.25 -2.16 -1.51 -1.28 -1.12
25 -2.47 -2.52 -2.94 -3.13 -2.93 -1.63 -0.98 -0.78
50 -2.63 -3.15 -2.99 -3.12 -3.37 -3.57 -0.81 -0.63
100 -2.79 -2.81 -2.82 -3.01 -3.26 -2.52 -0.14 0.02

From the above TCP RTT differences we can observe:

  1. A nearly across the board reduction in TCP RTT for DCTCP-SCE. SCE's proportional congestion signals are aiding the sender in managing queue lengths.

  2. Greater reductions in TCP RTT at lower bandwidths and path RTTs. See Section 5.1.2 for an explanation of this.

  3. A slight increase in TCP RTT for DCTCP-SCE at 100Mbit / 160ms. See Section 5.1.2 for a likely explanation of this.

5.2. Two-Flow Tests

The goal of the two-flow tests is to measure fairness between and among SCE and non-SCE TCP flows, through either a single queue or with fair queueing. What follows is a partial analysis of the results, and see Section a.3 through Section a.8 for the raw results tables.

5.2.1. Single Queue (Cake "flowblind")

Cake's flowblind parameter disables fair queueing, so that Cake uses only a single queue. This can be used to evaluate single-queue throughput fairness between SCE and non-SCE flows.

5.2.1.1. Reno vs Reno

It might be useful to remind ourselves that competition between two (or more) flows of the same TCP CC algorithm is usually designed to yield a high degree of throughput fairness in a single queue. Looking at Jain's fairness index [RFC5166] for two flows across a range of simulated path bandwidths and RTTs shows this to be true for two TCP Reno flows, for example:

Table 5: reno vs reno Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 0.993 1.000 1.000 0.998
5 1.000 1.000 0.998 0.998
10 1.000 0.999 0.998 1.000
50 1.000 0.998 0.999 0.999
100 0.998 0.999 0.999 1.000
5.2.1.2. Reno vs Reno-SCE

Now, we do a comparison of Reno vs Reno-SCE in a single queue, while still using Cake's default SCE signaling ramp:

Table 6: reno vs reno-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 0.969 0.969 0.915 0.821
5 0.958 0.824 0.714 0.632
10 0.959 0.719 0.669 0.698
50 0.608 0.621 0.673 0.907
100 0.560 0.627 0.668 0.782

From the above Jain's fairness index numbers we can observe that while there can be reasonable fairness at very low BDPs, and while there is no starvation, fairness degrades quickly at higher throughputs and RTTs. This is due to the Cake's default SCE signaling ramp being tuned to provide an early signal of congestion, to avoid CE marks and packet drops. As a result, SCE enabled flows back off in the face of competition, whereas non-SCE flows fill the queue until a drop or CE mark occurs.

5.2.1.3. Cubic vs DCTCP-SCE

Of particular interest to the congestion control community is competition between the commonly used TCP Cubic algorithm and DCTCP-SCE. It is a well-established fact that classic DCTCP will typically out-compete Cubic in a single queue. It would be valuable if there were a way to improve that fairness with SCE.

When comparing Cubic vs DCTCP-SCE using Cake's default SCE signaling ramp, we can see that while there is no starvation, fairness does degrade fairly rapidly as BDP increases:

Table 7: cubic vs dctcp-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 0.979 0.976 0.922 0.821
5 0.977 0.797 0.708 0.604
10 0.979 0.693 0.645 0.627
50 0.599 0.570 0.586 0.607
100 0.547 0.552 0.563 0.599

As of today, SCE by default does not lead to fairness at all BDPs between SCE and non-SCE flows. However, efforts are ongoing to improve this, and as we can see in Section 5.2.2, Cake's signaling ramp can be tuned to improve this fairness.

5.2.2. Single Queue (Cake "sce-single")

As we saw in Section 5.2.1, there is room to improve SCE vs non-SCE fairness in a single queue. One way to do this is to change the SCE signaling ramp to reduce or delay SCE signals until closer to the point where CE signals occur. This is the motivation behind the sce-single and sce-thresh Cake parameters.

By default, Cake begins proportionally signaling SCE when a packet's sojourn time in the queue is greater than half the CoDel target, and reaches 100% SCE signaling when the sojourn time is greater than or equal to the CoDel target. The sce-single parameter delays the start of the ramp until the sojourn reaches the CoDel target itself, while keeping the ramp slope the same. The sce-thresh parameter, while not evaluated here, allows an intermediate ramp between the default and sce-single, using values from 2-1024, with sce-thresh set to 8 yielding a ramp that's halfway in-between.

It is expected that the result of signaling SCE later is a subsequent increase in TCP RTT, including for single flows. Thus, the adjustment of the SCE signaling ramp is a tradeoff between the increased utilization and reduced TCP RTT possible with SCE, and fairness with non-SCE flows in a single queue. By using sce-single, we can show the maximum fairness that can be achieved by tuning the SCE signaling ramp in this way.

5.2.2.1. Reno vs Reno-SCE

Revisiting the Reno vs Reno-SCE comparison that we did in Section 5.2.1.2, we now run the same test with Cake's sce-single parameter set:

Table 8: reno vs reno-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 1.000 0.998 1.000 0.987
5 0.997 0.952 0.785 0.819
10 0.997 0.950 0.790 0.994
50 1.000 0.986 0.980 0.977
100 0.988 0.901 0.932 0.987

We can see that single queue fairness has improved considerably. While there are results at a few bandwidth-delay combinations that are still under investigation, single queue fairness between Reno and Reno-SCE has largely been achieved.

5.2.2.2. Cubic vs DCTCP-SCE

Revisiting the Cubic vs DCTCP-SCE comparison that we did in Section 5.2.1.3, we now run the same test with Cake's sce-single parameter set:

Table 9: cubic vs dctcp-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 0.993 0.999 0.995 0.963
5 1.000 0.898 0.765 0.741
10 0.975 0.893 0.733 0.885
50 0.999 0.752 0.799 0.923
100 0.944 0.714 0.661 0.650

While it can be seen that there is an almost across the board improvement vs Cake's default SCE ramp, there are still bandwidth-delay combinations that do not yield a sufficient level of fairness in a single queue. Work is ongoing to improve this.

5.2.3. Fair Queue (Cake "triple-isolate")

Cake's default triple-isolate fairness mode provides fairness among flows and a combination of source and destination IP addresses. For our purposes, this will effectively serve as fair queueing among flows, as we are only using one IP on each of the source and destination hosts.

With fair queueing, we expect to achieve a high level of throughput fairness at most bandwidth-delay combinations.

5.2.3.1. Cubic vs DCTCP-SCE

Revisiting the Cubic vs DCTCP-SCE comparisons that we did in Section 5.2.1.3 and Section 5.2.2.2, we now run the same test with Cake's fair queueing enabled via triple-isolate:

Table 10: cubic vs dctcp-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 1.000 1.000 1.000 0.916
5 1.000 0.980 0.931 0.698
10 1.000 0.946 0.896 0.722
50 1.000 0.978 0.977 1.000
100 1.000 0.999 1.000 1.000

With fair queueing, fairness is achieved for a broad range of bandwidth-delay combinations. We do see a consistent and narrow deviation at around 160ms between 5 and 10Mbit, which happens in competition between SCE and non-SCE flows, regardless of the exact algorithms in use. The cause of this will be investigated.

5.2.3.2. DCTCP vs DCTCP-SCE

As another example of SCE vs non-SCE competition with fair queueing enabled, here we compare DCTCP vs DCTCP-SCE:

Table 11: dctcp vs dctcp-sce Jain's fairness index; Columns: netem bi-dir Delay (ms); Rows: Cake Bandwidth (Mbit)
0 10 20 80
1 1.000 1.000 1.000 0.900
5 1.000 0.973 0.925 0.661
10 1.000 0.950 0.901 0.739
50 1.000 0.976 0.978 1.000
100 1.000 0.999 0.999 0.999

As with Cubic vs DCTCP-SCE, a high degree of fairness is achieved for most bandwidth-delay combinations, yet we do see the same small gap in fairness at around 160ms between 5 and 10Mbit as we see with Cubic vs DCTCP-SCE and other SCE vs non-SCE flow tests with fair queueing. It is expected that further investigation into this will lead to a solution.

6. Security Considerations

There are no known security considerations introduced by this note.

7. IANA Considerations

This document has no IANA actions.

8. Acknowledgments

Many thanks go out to Toke Hoiland-Jorgensen for making several key changes to the Flent tool.

9. Informative References

[Flent]
"The FLExible Network Tester Home Page", , <https://flent.org/>.
[RFC5166]
Floyd, S., Ed., "Metrics for the Evaluation of Congestion Control Mechanisms", RFC 5166, DOI 10.17487/RFC5166, , <https://www.rfc-editor.org/info/rfc5166>.
[RFC8259]
Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, , <https://www.rfc-editor.org/info/rfc8259>.
[sce-repo]
"Some Congestion Experienced Reference Implementation GitHub Repository", , <https://github.com/chromi/sce/>.
[ss]
"ss man page", , <https://linux.die.net/man/8/ss>.

Appendix A. Appendix (Raw Results Tables)

A.1. One-Flow TCP Throughput

Table 12: cubic Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.48 0.48 0.48 0.48 0.48 0.47 0.48 0.44
1 0.96 0.96 0.96 0.96 0.95 0.94 0.87 0.82
2 1.91 1.91 1.91 1.91 1.91 1.83 1.82 1.52
5 4.78 4.78 4.78 4.78 4.76 4.57 4.32 3.28
10 9.56 9.56 9.56 9.55 9.43 8.85 8.23 6.51
25 23.91 23.91 23.89 23.82 23.17 22.13 20.83 16.07
50 47.81 47.80 47.71 47.43 45.87 44.12 40.44 31.64
100 95.64 95.50 95.17 94.39 92.52 87.73 77.25 82.38

Table 13: reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.48 0.48 0.48 0.48 0.48 0.47 0.45 0.40
1 0.96 0.96 0.96 0.96 0.95 0.94 0.89 0.82
2 1.91 1.91 1.91 1.90 1.84 1.85 1.64 1.54
5 4.78 4.78 4.78 4.78 4.64 3.91 3.88 3.29
10 9.56 9.56 9.56 9.52 8.90 8.04 7.44 6.43
25 23.91 23.91 23.89 23.38 21.65 19.51 17.63 14.50
50 47.82 47.81 47.74 46.21 42.17 38.31 31.97 28.85
100 95.63 95.53 94.91 90.65 83.51 69.92 63.70 53.89

Table 14: reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.48 0.48 0.48 0.48 0.48 0.42 0.36 0.32
1 0.96 0.96 0.96 0.96 0.84 0.73 0.61 0.54
2 1.91 1.91 1.89 1.66 1.42 1.10 0.90 0.73
5 4.78 4.52 4.07 3.64 3.75 3.75 3.85 3.37
10 9.56 8.85 8.17 8.09 8.44 8.66 8.60 8.11
25 23.90 23.84 23.74 23.62 23.47 23.09 22.54 20.97
50 47.81 47.81 47.78 47.66 47.25 46.74 43.70 42.44
100 95.64 95.58 95.30 94.98 93.82 91.37 87.46 82.83

Table 15: dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.48 0.48 0.48 0.48 0.48 0.48 0.47 0.47
1 0.96 0.96 0.96 0.96 0.96 0.95 0.93 0.91
2 1.91 1.91 1.91 1.91 1.91 1.88 1.78 1.78
5 4.78 4.78 4.78 4.77 4.74 4.68 4.54 4.10
10 9.56 9.56 9.55 9.52 9.41 9.23 8.86 7.23
25 23.91 23.90 23.84 23.68 23.25 22.34 20.33 15.86
50 47.82 47.75 47.55 46.96 45.33 43.93 34.63 31.54
100 95.62 94.89 93.59 91.78 90.26 81.84 70.42 57.89

Table 16: dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 0.48 0.48 0.48 0.48 0.48 0.42 0.36 0.27
1 0.96 0.96 0.96 0.96 0.84 0.69 0.51 0.34
2 1.91 1.91 1.88 1.65 1.34 0.87 0.65 0.37
5 4.78 4.59 4.03 3.42 2.87 1.74 1.28 1.00
10 9.56 8.69 7.58 6.87 6.75 6.66 6.15 5.89
25 23.75 22.39 20.56 19.51 19.39 19.02 18.66 18.62
50 47.81 47.68 47.27 46.54 45.59 44.52 41.77 40.53
100 95.64 95.59 95.34 94.90 94.30 90.53 85.48 82.53

A.2. One-Flow TCP RTT

Table 17: cubic Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 121.38 120.07 119.91 121.37 108.48 128.70 189.50 230.34
1 68.40 68.16 68.10 69.33 72.97 88.11 116.66 191.18
2 34.80 35.01 40.33 40.17 52.74 62.05 100.71 175.00
5 13.79 15.92 18.43 23.83 31.04 49.07 87.79 167.49
10 7.08 8.95 12.58 17.39 26.12 44.73 84.60 164.49
25 6.36 8.29 11.31 15.33 24.06 43.37 82.68 162.72
50 5.88 8.16 10.70 14.76 23.51 42.82 82.10 161.81
100 5.99 8.08 10.38 14.42 23.69 42.90 82.16 162.01

Table 18: reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 138.77 137.84 138.91 138.92 132.94 140.51 170.48 235.32
1 66.82 66.68 68.93 69.02 77.11 91.69 123.15 196.59
2 41.00 41.25 40.81 42.24 47.24 65.51 99.34 176.74
5 16.25 18.37 19.18 25.13 32.06 49.61 87.61 167.39
10 7.68 9.86 13.14 18.75 26.74 45.54 84.50 164.45
25 6.27 8.26 11.46 15.22 23.65 42.84 82.59 162.69
50 5.99 8.08 10.34 14.11 22.86 42.22 81.97 162.08
100 5.87 7.64 9.85 13.62 22.64 41.96 81.73 161.76

Table 19: reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 50.36 50.33 50.48 50.33 70.57 87.95 128.25 203.39
1 24.52 24.74 26.28 35.99 44.19 63.77 104.02 183.39
2 11.91 13.58 18.62 23.01 32.70 55.39 95.68 175.24
5 7.07 8.90 11.67 16.76 26.43 46.12 85.92 166.15
10 5.22 7.11 10.19 14.84 24.30 44.12 83.96 164.04
25 4.40 6.17 8.93 13.48 23.23 42.85 82.78 162.68
50 3.99 5.90 8.86 13.80 23.45 43.20 82.83 162.65
100 4.00 5.97 8.81 13.68 23.59 43.38 83.14 162.91

Table 20: dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 132.20 131.17 131.14 130.92 130.17 157.56 199.65 260.24
1 66.23 66.69 66.65 66.60 79.38 103.86 135.31 203.32
2 33.84 34.07 39.03 40.24 52.76 69.25 104.09 183.85
5 13.67 15.87 18.50 23.05 32.73 53.12 89.63 168.33
10 6.99 9.06 12.82 17.26 26.75 45.79 85.43 165.10
25 6.57 8.26 11.25 16.07 25.65 44.11 83.45 163.30
50 6.22 8.39 10.98 15.83 25.65 45.56 82.73 162.51
100 6.31 8.27 11.16 16.12 25.88 44.90 82.43 162.05

Table 21: dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 2 5 10 20 40 80 160
0.5 49.63 49.96 49.77 49.39 69.64 87.27 124.89 205.91
1 24.45 24.38 26.22 35.62 44.12 64.59 105.36 187.70
2 11.90 13.78 18.17 22.66 32.83 56.55 96.72 179.88
5 7.02 8.60 11.42 16.60 27.16 49.53 90.11 170.64
10 5.14 6.95 9.97 15.01 24.59 44.28 84.15 163.98
25 4.10 5.74 8.31 12.94 22.72 42.48 82.47 162.52
50 3.59 5.24 7.99 12.71 22.28 41.99 81.92 161.88
100 3.52 5.46 8.34 13.11 22.62 42.38 82.29 162.07

A.3. Two-Flow TCP Throughput (Cake "flowblind")

Table 22: cubic-cubic Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 0.48 0.48 0.48 0.46 0.48 0.48 0.47 0.46
5 2.41 2.39 2.37 2.27 2.37 2.38 2.37 2.19
10 4.82 4.57 4.70 3.83 4.74 4.93 4.77 4.83
50 22.64 23.46 24.09 22.26 25.18 24.19 23.07 20.68
100 47.66 42.88 46.29 44.10 48.01 52.03 48.29 43.72

Table 23: cubic-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 0.43 0.46 0.45 0.41 0.53 0.50 0.51 0.51
5 2.10 2.04 2.46 2.30 2.68 2.71 2.22 2.04
10 4.23 4.61 4.50 4.16 5.34 4.81 4.83 4.17
50 22.79 24.60 23.55 26.18 25.03 23.05 23.32 14.70
100 46.82 51.06 48.73 55.96 48.83 43.85 44.96 29.02

Table 24: cubic-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 0.55 0.55 0.62 0.66 0.41 0.41 0.34 0.24
5 2.75 3.63 3.96 3.92 2.03 1.14 0.80 0.46
10 5.50 7.97 8.24 7.57 4.06 1.57 1.26 1.37
50 43.53 44.29 42.98 39.36 4.30 3.44 4.25 4.89
100 91.16 89.89 88.69 76.76 4.46 5.28 5.77 11.79

Table 25: cubic-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 0.48 0.47 0.40 0.33 0.48 0.48 0.56 0.60
5 2.30 1.53 1.49 1.70 2.49 3.25 3.26 2.87
10 3.69 2.77 2.81 2.99 5.88 6.77 6.68 5.96
50 14.99 16.93 15.25 18.99 32.85 30.62 31.72 24.58
100 31.33 30.67 33.63 32.23 64.35 63.57 59.81 55.50

Table 26: cubic-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 0.55 0.55 0.62 0.66 0.41 0.40 0.34 0.24
5 2.76 3.59 3.91 4.01 2.02 1.18 0.85 0.42
10 5.48 7.95 8.24 7.81 4.08 1.60 1.22 1.01
50 43.46 44.60 43.46 39.78 4.36 3.12 3.75 4.29
100 91.37 90.47 88.47 77.27 4.28 4.69 5.59 7.74

Table 27: reno-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 0.52 0.48 0.47 0.44 0.44 0.48 0.48 0.48
5 2.39 2.36 2.23 1.98 2.39 2.37 2.41 2.15
10 4.74 4.86 4.79 3.92 4.82 4.51 4.40 3.93
50 23.88 22.77 22.88 18.78 23.95 24.90 24.03 17.50
100 49.89 45.68 48.07 36.99 45.76 49.17 44.65 37.37

Table 28: reno-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 0.56 0.56 0.62 0.66 0.39 0.39 0.33 0.24
5 2.89 3.43 3.82 3.64 1.89 1.26 0.86 0.49
10 5.77 7.71 7.91 6.96 3.80 1.78 1.38 1.44
50 43.10 42.13 39.08 28.40 4.72 5.18 6.99 14.62
100 90.26 83.33 78.25 62.70 5.40 10.72 13.52 19.41

Table 29: reno-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 0.55 0.54 0.42 0.37 0.41 0.42 0.53 0.57
5 2.60 1.65 1.58 1.44 2.18 3.12 3.15 3.09
10 4.12 3.28 2.98 2.80 5.44 6.24 6.46 5.99
50 17.17 17.86 16.27 11.57 30.66 29.71 30.63 28.80
100 36.09 33.70 28.55 18.60 59.58 60.61 64.04 59.50

Table 30: reno-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 0.57 0.56 0.63 0.66 0.39 0.39 0.33 0.24
5 2.89 3.74 3.88 3.68 1.89 1.03 0.83 0.47
10 5.70 7.72 7.94 7.03 3.86 1.77 1.32 1.18
50 43.12 42.73 39.77 31.55 4.70 4.57 6.11 9.93
100 90.46 84.75 79.64 63.20 5.18 9.21 11.03 11.65

Table 31: reno-sce-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 0.48 0.48 0.48 0.32 0.48 0.48 0.48 0.34
5 2.39 1.99 1.75 1.77 2.39 2.04 1.75 1.52
10 4.78 3.80 3.82 4.23 4.78 4.00 3.79 4.06
50 24.24 21.17 24.08 24.69 23.60 26.11 22.91 20.34
100 42.25 47.10 37.69 44.31 53.39 48.22 57.14 40.92

Table 32: reno-sce-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 0.46 0.46 0.31 0.19 0.50 0.50 0.65 0.75
5 2.16 1.12 0.78 0.38 2.63 3.66 4.00 4.20
10 3.62 1.57 1.15 0.90 5.94 7.98 8.35 8.33
50 4.03 2.33 2.98 10.00 43.81 45.32 44.04 32.95
100 4.38 4.10 3.77 15.34 91.30 89.54 88.99 67.81

Table 33: reno-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 0.48 0.48 0.48 0.34 0.48 0.48 0.48 0.33
5 2.39 1.97 1.65 1.78 2.39 2.05 1.73 0.96
10 4.78 3.97 3.93 5.01 4.78 3.73 3.29 2.60
50 31.92 39.92 40.62 39.76 15.90 7.24 5.80 4.70
100 62.33 82.41 85.50 82.05 33.30 12.61 9.49 6.82

Table 34: dctcp-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 0.48 0.48 0.50 0.47 0.48 0.48 0.45 0.47
5 2.41 2.40 2.39 2.32 2.37 2.38 2.37 2.27
10 4.89 4.88 4.77 4.66 4.68 4.66 4.70 4.38
50 23.87 23.68 25.27 19.63 23.95 23.72 21.30 21.54
100 49.20 45.78 47.85 41.79 46.43 47.44 45.23 39.41

Table 35: dctcp-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 0.50 0.50 0.63 0.75 0.45 0.46 0.33 0.19
5 2.62 3.67 4.00 4.26 2.16 1.12 0.76 0.34
10 5.98 7.98 8.38 8.47 3.59 1.56 1.12 0.69
50 43.80 45.41 44.22 34.01 4.02 2.21 2.72 9.50
100 91.58 89.85 89.10 69.89 4.07 3.82 3.61 8.74

Table 36: dctcp-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 0.48 0.48 0.48 0.34 0.48 0.48 0.48 0.33
5 2.39 1.99 1.70 1.02 2.39 2.04 1.66 1.01
10 4.78 3.78 3.58 3.04 4.78 3.71 3.23 3.33
50 24.16 19.48 19.33 18.15 23.61 21.42 19.76 20.18
100 48.08 43.17 54.14 19.58 47.56 49.83 36.85 19.25

A.4. Two-Flow TCP Throughput (Cake "flowblind sce-single")

Table 37: cubic-cubic Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 0.48 0.48 0.48 0.42 0.48 0.48 0.48 0.51
5 2.42 2.31 2.44 2.21 2.36 2.46 2.29 2.30
10 4.83 4.95 4.62 4.15 4.74 4.59 4.88 4.63
50 23.88 23.31 24.19 23.67 23.95 24.34 22.99 19.38
100 50.40 47.62 48.40 38.13 45.25 47.30 46.14 48.92

Table 38: cubic-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 0.55 0.46 0.44 0.40 0.41 0.50 0.51 0.52
5 2.14 2.06 2.23 2.08 2.64 2.68 2.46 2.18
10 4.39 4.63 4.92 4.28 5.18 4.78 4.44 4.06
50 22.61 24.49 25.41 29.22 25.22 23.18 21.59 12.14
100 46.37 48.43 44.51 52.13 49.30 46.45 48.88 31.90

Table 39: cubic-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 0.45 0.46 0.51 0.48 0.50 0.50 0.44 0.42
5 2.24 3.07 3.78 3.24 2.54 1.68 0.97 1.13
10 4.16 5.59 7.04 4.21 5.40 3.78 2.36 4.18
50 23.95 24.71 25.16 17.98 23.87 22.53 21.18 20.74
100 50.40 66.75 65.82 64.67 45.26 28.41 28.70 22.52

Table 40: cubic-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 0.40 0.39 0.37 0.30 0.56 0.57 0.59 0.65
5 1.87 1.38 1.25 1.57 2.91 3.40 3.50 3.03
10 3.37 1.88 1.56 2.00 6.20 7.66 7.89 7.04
50 6.74 5.79 6.91 11.76 41.11 39.67 37.24 30.38
100 11.73 12.49 11.51 22.96 83.95 71.89 69.95 60.58

Table 41: cubic-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 0.44 0.46 0.51 0.55 0.52 0.49 0.44 0.37
5 2.40 3.18 3.69 3.46 2.39 1.58 1.06 0.89
10 4.02 6.35 7.58 5.48 5.54 3.09 1.87 2.58
50 24.54 37.56 35.51 27.95 23.29 10.16 11.77 15.42
100 59.52 77.63 80.96 75.79 36.12 17.42 13.35 11.62

Table 42: reno-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 0.49 0.47 0.49 0.46 0.47 0.48 0.47 0.45
5 2.38 2.39 2.33 1.96 2.40 2.35 2.33 2.16
10 4.78 4.58 4.63 3.79 4.79 4.84 4.61 4.23
50 24.05 23.35 24.20 19.51 23.78 24.33 22.69 16.79
100 46.69 50.26 46.11 35.97 48.97 44.58 46.86 38.56

Table 43: reno-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 0.47 0.50 0.49 0.49 0.49 0.46 0.47 0.39
5 2.51 2.89 3.58 3.08 2.27 1.83 1.12 1.11
10 4.51 5.67 6.96 4.39 5.06 3.55 2.22 3.74
50 23.88 26.49 26.33 15.66 23.94 20.93 19.68 21.41
100 53.06 62.99 58.85 48.15 42.59 31.69 33.83 38.41

Table 44: reno-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 0.42 0.42 0.38 0.33 0.54 0.53 0.58 0.62
5 2.00 1.50 1.47 1.49 2.79 3.28 3.28 3.06
10 3.62 2.19 1.92 2.07 5.95 7.35 7.53 6.87
50 7.80 6.96 6.91 8.15 40.04 38.98 37.41 31.25
100 13.40 13.20 10.37 16.04 82.28 72.23 70.72 61.88

Table 45: reno-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 0.47 0.48 0.48 0.24 0.49 0.47 0.47 0.66
5 2.50 3.15 3.64 2.92 2.28 1.59 1.07 1.10
10 4.53 6.77 7.30 5.18 5.04 2.65 1.91 2.55
50 26.20 35.85 34.81 22.99 21.62 11.46 11.45 19.58
100 64.59 71.71 71.03 59.15 31.06 22.61 20.74 17.50

Table 46: reno-sce-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 0.47 0.48 0.50 0.41 0.49 0.48 0.45 0.41
5 2.39 2.19 2.04 1.81 2.39 2.15 1.85 1.97
10 4.77 4.13 3.90 3.65 4.79 4.21 4.05 4.16
50 23.90 23.79 21.48 17.40 23.92 23.94 25.97 21.52
100 48.57 54.33 48.58 47.14 47.10 40.97 46.49 41.08

Table 47: reno-sce-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 0.41 0.41 0.38 0.28 0.55 0.55 0.58 0.66
5 1.87 1.39 1.06 0.67 2.91 3.38 3.69 3.96
10 3.69 2.43 2.03 1.17 5.88 7.08 7.37 7.96
50 6.49 3.69 3.98 9.37 41.35 41.62 39.72 30.90
100 7.56 6.25 6.13 16.00 88.12 76.49 72.46 62.73

Table 48: reno-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 0.47 0.47 0.49 0.42 0.48 0.48 0.46 0.40
5 2.35 2.10 1.91 2.27 2.43 2.20 1.94 1.38
10 4.78 4.37 4.89 4.74 4.78 3.78 2.97 2.62
50 27.23 39.56 40.95 25.72 20.59 8.16 6.56 13.56
100 74.00 82.84 79.48 67.86 21.65 12.49 15.53 11.94

Table 49: dctcp-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 0.48 0.48 0.48 0.47 0.48 0.48 0.47 0.47
5 2.38 2.35 2.35 2.30 2.40 2.43 2.42 2.32
10 4.74 4.76 4.73 4.33 4.82 4.79 4.72 4.63
50 24.18 22.36 22.40 13.88 23.63 24.65 23.05 26.04
100 42.61 32.42 25.55 43.30 53.04 56.18 59.67 35.60

Table 50: dctcp-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 0.56 0.56 0.58 0.68 0.40 0.41 0.37 0.27
5 2.89 2.38 3.66 3.91 1.90 2.54 1.09 0.67
10 5.97 5.93 7.55 8.00 3.60 3.58 1.85 1.05
50 41.35 41.32 39.50 31.56 6.47 3.79 3.92 9.96
100 89.09 76.31 73.53 63.92 6.60 6.36 4.97 9.40

Table 51: dctcp-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 0.50 0.48 0.49 0.40 0.46 0.48 0.47 0.41
5 2.40 2.12 1.90 1.65 2.39 2.14 1.92 1.58
10 4.78 3.91 3.45 3.71 4.79 3.92 3.59 3.33
50 23.65 24.33 22.34 21.85 24.19 23.17 24.15 21.08
100 53.54 59.02 54.68 46.28 42.10 36.29 40.24 43.95

A.5. Two-Flow TCP Throughput (Cake "triple-isolate")

Table 52: cubic-cubic Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 0.48 0.48 0.48 0.46 0.48 0.48 0.48 0.46
5 2.39 2.39 2.39 2.20 2.39 2.39 2.36 2.21
10 4.78 4.74 4.80 4.42 4.78 4.79 4.70 4.19
50 23.91 23.82 23.59 20.89 23.91 23.83 23.56 22.55
100 47.83 47.40 46.89 43.30 47.82 47.40 47.48 43.60

Table 53: cubic-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 0.48 0.48 0.48 0.46 0.48 0.48 0.48 0.45
5 2.39 2.39 2.48 2.11 2.39 2.39 2.18 2.16
10 4.78 4.71 4.81 4.43 4.78 4.80 4.60 4.20
50 23.91 24.24 24.11 23.11 23.91 23.42 22.23 17.99
100 47.83 48.65 48.14 48.87 47.83 46.14 43.84 35.92

Table 54: cubic-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 0.48 0.48 0.48 0.57 0.48 0.48 0.48 0.32
5 2.39 2.70 2.87 3.40 2.39 2.00 1.75 1.00
10 4.78 5.60 5.38 4.56 4.78 3.75 3.79 4.09
50 23.92 24.11 23.51 20.30 23.90 23.56 23.72 24.27
100 47.83 47.45 46.41 42.39 47.81 47.71 48.22 48.49

Table 55: cubic-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 0.48 0.48 0.48 0.44 0.48 0.48 0.48 0.50
5 2.39 2.38 2.34 2.16 2.39 2.40 2.41 2.35
10 4.78 4.68 4.77 4.38 4.78 4.85 4.71 4.44
50 23.91 23.86 23.36 21.81 23.91 23.68 23.58 21.10
100 47.84 47.77 46.77 47.27 47.82 46.94 47.30 40.48

Table 56: cubic-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 0.48 0.48 0.48 0.58 0.48 0.48 0.48 0.31
5 2.39 2.69 2.94 3.58 2.39 2.02 1.68 0.74
10 4.78 5.81 6.15 6.93 4.78 3.57 3.03 1.62
50 24.02 27.16 26.07 20.95 23.80 20.14 19.16 21.20
100 47.85 49.02 47.67 42.50 47.80 46.02 45.94 42.97

Table 57: reno-reno Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 0.48 0.48 0.48 0.46 0.48 0.48 0.48 0.46
5 2.39 2.37 2.31 2.04 2.39 2.35 2.33 2.01
10 4.78 4.81 4.60 4.25 4.78 4.74 4.60 4.33
50 23.91 23.77 23.07 17.70 23.91 23.77 22.92 17.72
100 47.83 47.39 45.06 37.23 47.83 47.54 44.58 35.40

Table 58: reno-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 0.48 0.48 0.48 0.55 0.48 0.48 0.48 0.32
5 2.39 2.61 2.76 2.91 2.39 2.01 1.76 1.16
10 4.78 5.39 5.44 4.34 4.78 3.84 3.76 4.24
50 23.93 23.81 22.02 17.75 23.90 23.72 24.79 25.80
100 47.83 46.09 42.73 35.26 47.82 49.03 50.97 50.07

Table 59: reno-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 0.48 0.48 0.48 0.43 0.48 0.48 0.48 0.50
5 2.39 2.30 2.27 1.88 2.39 2.47 2.47 2.44
10 4.78 4.73 4.66 4.19 4.78 4.79 4.78 4.30
50 23.91 23.45 21.71 17.98 23.91 24.13 24.75 21.47
100 47.82 46.42 43.70 35.62 47.81 48.18 49.48 42.19

Table 60: reno-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 0.48 0.48 0.48 0.54 0.48 0.48 0.48 0.33
5 2.39 2.60 2.81 3.25 2.39 2.02 1.69 0.75
10 4.78 5.65 6.18 6.09 4.78 3.60 2.97 1.90
50 24.05 27.29 24.37 18.32 23.77 19.78 19.76 22.57
100 47.87 47.75 44.23 35.55 47.79 46.27 47.20 42.70

Table 61: reno-sce-reno-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 0.48 0.48 0.48 0.34 0.48 0.48 0.48 0.33
5 2.39 2.05 1.80 1.75 2.39 2.03 1.89 1.62
10 4.78 4.15 4.14 4.22 4.78 4.15 4.12 4.31
50 23.91 23.66 23.54 22.01 23.90 23.69 23.24 22.75
100 47.82 47.66 47.52 45.05 47.82 47.66 47.54 44.97

Table 62: reno-sce-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 0.48 0.48 0.48 0.32 0.48 0.48 0.48 0.61
5 2.39 1.98 1.71 0.91 2.39 2.78 2.95 3.60
10 4.78 3.75 3.85 3.99 4.78 5.64 5.41 4.90
50 23.91 23.71 23.51 22.56 23.91 23.99 23.57 20.79
100 47.83 48.07 47.63 46.93 47.83 47.03 46.40 39.48

Table 63: reno-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 0.48 0.48 0.48 0.33 0.48 0.48 0.48 0.34
5 2.39 2.01 1.86 1.82 2.39 2.05 1.73 1.03
10 4.78 4.24 4.60 6.33 4.78 3.95 3.36 1.97
50 24.01 26.91 27.21 23.75 23.81 18.74 16.94 18.75
100 47.85 49.17 49.32 46.19 47.81 45.83 44.78 40.59

Table 64: dctcp-dctcp Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 0.48 0.48 0.48 0.47 0.48 0.48 0.48 0.47
5 2.39 2.39 2.38 2.24 2.39 2.39 2.38 2.24
10 4.78 4.76 4.73 4.45 4.78 4.77 4.71 4.47
50 23.91 23.70 23.29 20.37 23.91 23.70 23.30 20.80
100 47.73 47.13 46.71 39.67 47.73 47.17 46.73 41.12

Table 65: dctcp-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 0.48 0.48 0.48 0.62 0.48 0.48 0.48 0.31
5 2.39 2.77 3.00 3.86 2.39 1.98 1.67 0.64
10 4.78 5.76 6.17 6.85 4.78 3.61 3.10 1.74
50 24.01 27.30 26.52 20.66 23.81 19.96 19.56 19.92
100 47.85 48.63 48.11 40.69 47.77 46.10 45.60 38.33

Table 66: dctcp-sce-dctcp-sce Mean TCP Throughput (Mbit); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 0.48 0.48 0.48 0.34 0.48 0.48 0.48 0.34
5 2.39 2.04 1.71 1.09 2.39 2.03 1.76 1.06
10 4.78 3.91 3.55 2.97 4.78 4.01 3.59 2.84
50 23.88 21.37 20.20 19.68 23.91 21.09 20.51 20.44
100 47.82 45.19 43.83 36.16 47.82 46.37 44.59 37.07

A.6. Two-Flow TCP RTT (Cake "flowblind")

Table 67: cubic-cubic Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 70.73 69.97 83.70 139.06 69.07 70.38 86.15 140.64
5 14.09 25.54 35.72 91.62 14.26 25.99 35.59 91.28
10 7.88 18.69 28.67 86.54 7.79 18.46 28.60 85.74
50 6.53 16.30 25.55 82.98 6.47 16.37 25.66 83.06
100 6.35 16.40 25.49 83.27 6.43 16.29 25.57 83.36

Table 68: cubic-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 65.97 71.14 83.63 139.14 66.25 72.11 87.70 140.12
5 14.00 25.94 35.16 91.73 14.87 25.03 35.98 92.22
10 7.60 18.82 28.77 86.37 7.99 19.39 29.11 86.39
50 6.43 16.26 25.45 82.72 6.41 16.38 25.46 83.06
100 6.31 16.20 25.09 83.07 6.47 16.09 25.11 83.19

Table 69: cubic-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 71.98 73.29 93.38 127.68 61.37 61.69 71.07 121.98
5 14.58 25.06 33.50 88.85 11.80 21.16 32.19 94.96
10 7.70 17.86 27.34 85.73 6.98 18.59 30.43 90.51
50 5.95 15.00 24.47 83.61 6.01 16.71 26.93 86.54
100 6.01 14.78 24.44 83.56 6.06 15.99 26.21 85.14

Table 70: cubic-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 61.63 62.74 79.63 144.41 81.28 81.99 88.30 142.69
5 12.75 27.02 38.94 94.68 18.98 24.23 33.58 92.59
10 7.19 20.98 30.61 87.70 8.12 17.94 27.88 86.25
50 6.84 16.43 26.24 83.90 6.38 16.05 26.01 84.39
100 6.49 16.34 26.42 85.08 6.46 16.42 26.19 84.83

Table 71: cubic-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 73.64 72.10 92.35 127.57 60.95 63.18 73.14 117.81
5 14.10 23.88 32.68 89.27 11.73 21.25 32.24 95.10
10 7.99 17.94 27.11 85.58 6.94 18.23 30.11 91.01
50 5.91 14.97 24.16 83.11 5.97 16.67 26.92 86.08
100 6.13 14.98 24.27 82.93 6.08 16.17 26.17 84.85

Table 72: reno-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 64.31 68.79 84.19 141.79 66.79 71.57 83.55 140.03
5 14.46 25.39 35.62 92.66 14.61 25.49 35.57 92.29
10 8.15 18.91 29.00 86.57 8.08 18.83 29.11 86.26
50 6.49 16.43 24.97 82.44 6.44 16.23 25.06 82.48
100 6.43 16.20 24.68 82.39 6.50 16.18 24.91 82.39

Table 73: reno-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 68.15 70.79 91.27 130.38 62.79 62.52 75.28 122.99
5 14.95 23.90 34.12 89.47 12.89 21.95 33.51 95.43
10 8.05 18.27 28.12 85.51 7.22 18.71 29.40 90.04
50 5.99 14.84 24.17 83.36 6.09 16.08 25.91 84.37
100 6.19 14.61 24.02 83.03 6.05 15.50 25.06 83.81

Table 74: reno-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 59.96 62.40 83.02 144.86 114.59 108.05 87.97 143.70
5 12.78 27.64 38.54 95.38 22.21 24.00 33.12 92.06
10 8.10 20.18 30.66 88.42 8.31 18.10 27.56 86.64
50 6.87 16.58 26.04 83.56 6.48 16.36 25.81 83.38
100 6.38 16.54 26.00 84.56 6.48 16.43 25.93 83.75

Table 75: reno-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 67.25 70.19 93.80 131.16 63.59 63.29 75.04 120.28
5 15.01 26.28 34.17 89.37 12.68 23.72 33.43 94.57
10 7.89 18.30 27.91 85.30 7.11 18.84 29.84 90.39
50 6.09 14.74 23.74 82.58 6.07 16.17 25.95 84.17
100 5.78 14.32 23.62 82.36 5.95 15.20 24.81 83.98

Table 76: reno-sce-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 49.45 50.66 50.71 114.04 50.74 50.87 50.54 113.68
5 9.58 19.47 30.20 88.97 9.66 19.40 30.39 89.81
10 6.55 16.99 27.00 85.35 6.55 16.61 26.91 85.29
50 4.46 13.52 23.27 82.93 4.54 13.35 23.26 83.11
100 4.40 14.06 23.83 82.74 4.21 14.07 23.74 82.71

Table 77: reno-sce-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 53.35 54.29 82.98 132.00 93.33 94.62 84.60 141.33
5 11.15 21.45 31.72 93.81 18.29 23.96 33.21 90.14
10 7.04 18.09 30.04 92.23 8.00 17.48 27.41 86.25
50 6.15 16.43 27.42 85.57 6.32 15.91 25.72 83.84
100 6.29 16.54 26.52 85.06 6.41 16.26 26.07 83.57

Table 78: reno-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 49.87 51.12 50.45 113.39 50.23 50.54 50.13 112.50
5 9.57 19.48 30.47 88.94 9.64 19.11 29.76 91.18
10 6.58 16.86 27.26 84.81 6.63 16.66 27.39 85.95
50 4.20 12.98 22.81 82.45 4.59 14.89 24.87 84.83
100 4.15 13.47 23.29 82.64 4.11 14.16 24.11 83.93

Table 79: dctcp-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 94.39 94.71 87.26 145.73 95.80 95.81 92.26 144.97
5 18.87 25.91 35.08 93.41 19.48 26.10 34.76 93.83
10 9.29 18.56 27.94 87.31 9.40 18.65 28.15 87.12
50 6.78 16.54 26.18 84.83 6.74 16.48 26.21 84.80
100 6.69 16.43 25.89 85.15 6.75 16.30 25.94 85.35

Table 80: dctcp-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 91.81 93.17 83.18 141.45 53.62 53.10 76.65 133.28
5 18.22 23.79 33.56 91.14 11.04 21.38 31.87 93.16
10 8.06 17.46 27.21 86.24 7.25 18.04 29.56 92.76
50 6.14 15.92 25.60 83.50 6.17 16.47 27.02 86.00
100 6.35 16.04 25.95 83.06 6.43 16.35 26.71 85.56

Table 81: dctcp-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 48.67 49.51 50.22 115.03 49.42 49.95 49.96 113.68
5 9.56 19.62 29.79 91.63 9.63 19.19 30.02 91.44
10 6.52 16.78 27.09 85.64 6.61 16.65 27.42 85.73
50 4.16 13.13 22.39 82.21 4.18 12.96 22.47 82.09
100 3.90 13.02 22.61 82.16 3.97 12.94 22.72 82.04

A.7. Two-Flow TCP RTT (Cake "flowblind sce-single")

Table 82: cubic-cubic Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 68.26 69.38 83.80 142.98 69.18 68.64 83.08 142.21
5 13.86 25.36 35.54 92.39 13.92 24.90 36.61 92.30
10 8.39 18.95 28.76 86.67 8.19 19.22 28.77 86.23
50 6.52 16.30 25.85 83.11 6.59 16.26 25.84 83.18
100 6.42 16.49 25.57 83.19 6.40 16.38 25.65 83.22

Table 83: cubic-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 65.10 69.82 83.36 144.19 64.96 69.20 85.01 144.48
5 13.62 25.89 35.86 92.45 14.26 25.25 36.40 93.45
10 7.82 18.99 28.74 86.24 8.23 19.64 29.36 86.42
50 6.57 16.15 25.30 82.60 6.53 16.17 25.47 83.21
100 6.46 16.14 24.89 83.04 6.54 16.23 24.88 83.28

Table 84: cubic-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 67.62 68.51 83.34 138.15 68.37 69.98 84.17 139.38
5 13.56 24.05 34.64 91.07 13.81 24.46 37.77 96.38
10 8.16 18.42 28.22 86.47 8.07 19.70 31.56 86.75
50 6.44 15.91 25.22 82.90 6.81 16.17 25.55 82.74
100 6.47 16.73 26.43 84.25 6.62 16.63 26.62 84.78

Table 85: cubic-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 64.14 64.56 77.80 145.02 65.75 64.93 80.94 142.91
5 13.23 25.90 38.90 96.18 13.84 23.58 33.08 93.08
10 7.73 21.30 34.10 90.52 7.86 18.08 27.45 87.45
50 7.93 18.01 28.17 86.63 7.02 16.02 26.69 85.82
100 7.66 17.36 27.50 86.24 6.87 16.47 26.59 85.79

Table 86: cubic-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 67.09 68.20 83.79 137.00 66.85 68.78 84.89 142.55
5 14.22 24.20 34.59 90.97 13.60 24.91 36.51 96.69
10 8.16 18.76 28.16 86.07 8.16 20.29 31.97 88.89
50 6.42 16.27 25.71 83.55 6.69 17.27 26.47 84.08
100 6.54 16.35 25.54 83.88 6.79 16.65 26.21 84.75

Table 87: reno-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 67.27 69.27 83.70 145.08 67.81 68.53 84.48 145.65
5 14.07 26.21 36.48 93.63 14.28 26.00 36.21 93.05
10 8.21 19.06 29.68 86.73 8.20 18.96 29.25 86.39
50 6.62 16.41 24.97 82.36 6.68 16.32 25.01 82.48
100 6.57 15.91 24.64 82.45 6.45 15.90 24.65 82.56

Table 88: reno-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 67.65 68.24 83.30 139.85 68.25 70.37 79.56 142.52
5 14.09 24.97 35.97 91.33 14.55 24.35 37.58 96.81
10 8.54 19.17 28.84 86.91 8.21 19.96 32.27 87.41
50 6.44 15.55 24.88 82.79 6.56 16.08 25.22 82.59
100 6.40 16.27 25.71 85.25 6.74 16.42 25.90 85.13

Table 89: reno-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 62.85 65.76 79.20 146.75 65.00 66.92 78.64 140.83
5 13.65 26.18 38.66 97.29 14.02 23.92 33.54 93.24
10 8.12 21.46 32.84 91.43 7.89 17.89 27.67 87.40
50 7.90 18.20 28.50 85.93 6.86 16.53 26.78 84.89
100 7.58 17.41 27.77 85.89 7.06 16.76 27.43 85.22

Table 90: reno-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 66.94 68.38 84.13 443.94 68.15 70.50 80.58 1036.03
5 14.59 25.96 35.41 91.34 14.17 25.50 36.79 96.14
10 8.76 19.46 28.55 86.17 8.35 21.14 31.51 88.70
50 6.32 15.99 25.40 83.48 6.63 17.15 26.42 83.55
100 6.33 15.98 25.24 83.42 6.67 16.22 25.74 84.21

Table 91: reno-sce-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 71.92 72.60 85.12 140.88 71.48 71.82 82.18 138.74
5 14.11 24.23 34.56 92.98 14.22 24.21 35.30 92.20
10 9.86 19.83 29.38 86.98 9.89 19.80 29.32 86.79
50 6.63 17.01 26.57 82.78 6.69 16.95 26.48 82.69
100 6.49 16.69 26.68 85.00 6.53 16.93 26.69 85.15

Table 92: reno-sce-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 69.23 66.60 78.75 146.62 67.72 67.64 79.32 139.43
5 13.35 24.63 36.42 98.92 13.90 23.35 32.95 92.69
10 8.00 19.99 30.77 94.50 7.85 18.12 27.44 87.54
50 8.06 19.71 31.42 88.87 7.06 16.79 26.70 85.95
100 8.34 19.77 32.99 91.65 7.55 16.70 27.21 87.35

Table 93: reno-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 71.87 72.66 83.24 139.33 72.10 73.13 86.11 136.98
5 15.51 24.77 34.98 91.65 15.39 23.89 34.00 93.84
10 9.89 19.46 28.37 87.12 9.73 19.48 30.06 89.37
50 6.53 16.33 25.97 82.70 6.66 17.52 27.41 83.22
100 6.81 16.30 26.07 83.48 6.96 16.88 26.35 84.50

Table 94: dctcp-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 68.18 68.95 80.54 143.93 69.62 69.18 80.52 141.76
5 15.31 24.50 34.27 94.45 15.27 24.16 33.93 95.11
10 8.30 18.85 28.20 87.75 8.44 18.78 28.46 87.50
50 7.47 16.58 26.42 85.35 7.49 16.75 26.67 85.34
100 7.64 16.55 26.77 86.26 7.52 17.00 26.75 86.42

Table 95: dctcp-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 66.02 66.25 78.29 139.85 67.98 65.82 77.89 145.81
5 13.88 207.36 32.89 92.04 13.55 380.51 35.59 98.73
10 7.81 80.88 27.60 87.58 7.95 154.57 30.74 93.93
50 6.98 16.12 26.46 85.87 8.00 19.22 32.18 88.74
100 7.72 16.92 27.67 86.88 8.67 19.60 32.95 91.79

Table 96: dctcp-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 69.76 71.34 84.74 139.62 72.85 71.92 86.45 137.08
5 15.58 23.92 33.94 92.72 15.65 23.80 33.77 93.02
10 9.78 19.14 29.55 90.22 9.73 19.36 29.58 89.44
50 6.75 15.21 24.86 82.92 6.80 15.27 24.83 83.08
100 6.41 15.71 25.16 83.67 6.59 15.95 25.31 83.71

A.8. Two-Flow TCP RTT (Cake "triple-isolate")

Table 97: cubic-cubic Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic cubic
1 92.48 88.69 88.22 131.91 92.05 90.01 87.87 131.57
5 18.67 28.80 36.43 90.88 18.74 28.85 36.72 90.74
10 8.48 18.56 28.64 86.24 8.53 18.63 28.72 86.10
50 6.51 15.85 24.13 82.81 6.60 15.85 24.00 82.69
100 6.04 15.21 23.73 82.62 5.92 15.29 23.71 82.59

Table 98: cubic-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno
1 90.34 89.98 87.87 133.32 68.49 66.53 73.84 138.30
5 17.62 29.12 36.16 90.80 14.13 30.47 37.59 92.04
10 8.49 18.41 28.65 86.04 8.90 20.00 30.17 86.63
50 6.48 15.92 23.71 82.53 6.56 15.44 23.98 82.33
100 6.15 15.14 23.53 82.86 6.22 14.57 23.50 82.20

Table 99: cubic-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic reno-sce
1 92.71 90.37 87.84 129.22 50.64 50.51 50.24 117.80
5 16.87 24.55 35.20 90.00 9.62 19.86 30.07 92.04
10 8.53 18.97 29.09 86.20 6.59 16.56 26.35 85.23
50 6.55 15.97 24.28 82.65 4.41 13.89 23.32 82.85
100 6.03 15.27 23.98 82.61 3.93 13.85 23.58 82.96

Table 100: cubic-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp
1 92.60 89.79 87.75 135.20 97.84 96.66 96.75 149.61
5 14.45 28.19 37.86 91.20 19.07 25.58 35.17 94.03
10 8.63 18.12 28.34 86.15 8.16 17.58 27.57 87.68
50 6.57 15.82 24.40 82.81 6.80 16.42 25.94 84.05
100 6.24 15.32 23.81 82.84 6.52 16.12 25.83 82.92

Table 101: cubic-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
cubic dctcp-sce
1 92.73 87.54 87.60 126.28 50.28 49.77 49.69 115.82
5 14.77 24.31 34.63 89.26 9.54 20.04 29.91 93.48
10 8.46 18.88 28.82 85.49 6.87 16.46 26.73 88.22
50 6.61 17.05 24.37 82.43 4.19 13.11 22.72 82.28
100 6.41 15.44 23.87 82.35 3.66 12.84 22.70 82.27

Table 102: reno-reno Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno
1 68.39 66.58 73.72 142.08 68.61 66.66 73.74 142.94
5 15.73 25.66 38.41 92.20 14.62 25.71 37.82 92.07
10 8.48 20.48 29.68 86.62 8.86 20.37 29.82 86.93
50 6.58 15.60 23.71 82.41 6.54 15.64 23.77 82.36
100 6.32 14.10 23.44 82.17 6.36 14.13 23.13 81.99

Table 103: reno-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno reno-sce
1 68.77 66.59 73.78 137.04 50.65 50.32 50.23 116.09
5 16.12 24.59 37.04 90.48 9.41 19.69 30.14 91.04
10 8.46 19.67 31.33 86.72 6.76 16.53 26.33 85.41
50 6.59 15.65 23.98 82.46 4.43 13.54 23.21 82.73
100 6.35 14.57 23.37 82.16 4.11 13.62 23.45 82.85

Table 104: reno-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp
1 68.04 66.19 73.70 140.21 96.81 96.80 96.58 143.33
5 16.18 26.99 38.88 92.89 18.99 24.82 35.27 93.97
10 8.59 19.99 30.15 86.79 8.27 17.92 27.99 88.08
50 6.38 15.57 23.94 82.44 6.78 16.46 25.77 83.82
100 6.24 14.72 23.45 82.17 6.30 16.11 24.98 82.96

Table 105: reno-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno dctcp-sce
1 69.06 67.33 65.57 133.89 50.17 49.67 49.59 115.33
5 16.04 24.23 36.62 89.94 9.57 19.90 30.12 93.42
10 8.51 19.93 29.67 85.60 6.65 16.53 26.99 87.11
50 6.47 16.89 24.02 82.35 4.19 13.04 22.72 82.35
100 6.35 14.57 23.39 81.90 3.74 12.59 22.53 82.02

Table 106: reno-sce-reno-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce reno-sce
1 50.10 50.27 50.22 117.12 50.62 50.44 50.23 116.67
5 9.54 19.29 29.77 88.86 9.63 19.25 29.76 89.43
10 6.89 16.55 26.16 85.44 6.62 16.47 26.14 85.52
50 4.50 13.31 22.99 82.65 4.47 13.28 22.93 82.60
100 4.25 13.89 23.70 83.01 4.14 13.74 23.59 82.88

Table 107: reno-sce-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp
1 49.93 50.27 50.22 119.15 97.30 97.00 96.39 135.73
5 9.45 19.45 30.20 92.32 19.13 23.40 33.39 91.37
10 6.71 16.62 26.26 85.57 8.16 18.21 28.41 87.40
50 4.42 13.69 23.24 82.83 6.75 16.30 26.05 84.00
100 4.07 13.79 23.61 82.79 6.36 16.43 26.21 83.42

Table 108: reno-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
reno-sce dctcp-sce
1 50.11 50.26 50.22 115.48 50.26 49.68 49.61 114.34
5 9.52 19.16 29.74 88.60 9.61 19.32 29.60 91.03
10 6.49 16.48 25.83 84.54 6.77 16.38 26.33 87.03
50 4.41 13.17 22.98 82.60 4.12 13.08 22.99 82.38
100 4.16 13.48 23.17 82.59 3.78 12.83 22.57 82.04

Table 109: dctcp-dctcp Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp
1 97.66 96.66 96.60 151.58 97.03 96.71 96.65 150.04
5 18.76 25.73 35.30 93.14 19.22 25.75 35.17 93.24
10 8.15 18.04 27.99 87.51 8.36 17.57 27.57 87.54
50 6.62 16.44 25.70 83.54 6.56 16.35 25.91 84.12
100 6.46 16.25 26.01 82.89 6.56 16.07 25.86 82.93

Table 110: dctcp-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp dctcp-sce
1 97.80 96.84 96.75 137.02 50.27 49.76 49.67 114.01
5 18.89 23.68 33.54 91.13 9.53 19.80 29.74 93.86
10 8.09 17.86 28.37 85.94 6.63 16.42 26.54 87.48
50 6.74 16.51 25.50 83.04 4.06 13.08 22.68 82.42
100 6.44 16.27 26.02 83.05 3.74 12.88 22.68 82.08

Table 111: dctcp-sce-dctcp-sce Mean TCP RTT (ms); Columns: netem bi-directional Delay (ms); Rows: Cake-limited Bandwidth (Mbit)
0 10 20 80 0 10 20 80
dctcp-sce dctcp-sce
1 50.89 49.74 49.64 116.00 50.28 49.72 49.68 113.76
5 9.44 19.41 30.11 90.46 9.53 19.19 29.78 91.12
10 6.66 16.46 26.39 86.01 6.77 16.44 26.44 86.28
50 4.14 12.85 22.68 82.30 4.19 13.06 22.55 82.25
100 3.78 12.57 22.34 81.86 3.80 12.56 22.26 81.96

Authors' Addresses

Peter G. Heist
Redacted
463 11 Liberec 30
Czech Republic
Rodney W. Grimes
Redacted
Portland, OR 97217
United States
Jonathan Morton
Kokkonranta 21
FI-31520 Pitkajarvi
Finland
Phone: +358 44 927 2377