Internet DRAFT - draft-an-multipath-quic
draft-an-multipath-quic
QUIC Q. An
Internet-Draft Y. Liu
Intended status: Standards Track Y. Ma
Expires: April 25, 2021 Alibaba Inc.
Z. Li
ICT-CAS
October 22, 2020
Multipath Extension for QUIC
draft-an-multipath-quic-00
Abstract
This document specifies multipath extension for the QUIC protocol to
enable the simultaneous usage of multiple paths for a single
connection.
The extension is compliant with the single-path QUIC design. The
design principle is to support multipath by adding limited extension
to QUIC-Transport [I-D.ietf-quic-transport].
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 25, 2021.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
An, et al. Expires April 25, 2021 [Page 1]
Internet-Draft multipath-quic October 2020
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Notational Conventions . . . . . . . . . . . . . . . . . . . 3
3. Sub-Connection . . . . . . . . . . . . . . . . . . . . . . . 4
4. Enable Multipath QUIC - Handshake . . . . . . . . . . . . . . 5
5. Sub-connection Management . . . . . . . . . . . . . . . . . . 5
5.1. Multipath QUIC Interaction . . . . . . . . . . . . . . . 5
5.2. Path validation and sub-connection ID negotiation . . . . 8
5.3. New sub-connection establishment . . . . . . . . . . . . 9
5.4. Close sub-connection . . . . . . . . . . . . . . . . . . 9
5.5. Sub-connection Lookup . . . . . . . . . . . . . . . . . . 10
5.6. Sub-connection migration . . . . . . . . . . . . . . . . 10
5.7. Sub-connection state machine management . . . . . . . . . 10
5.8. Sender and Receiver Connection (Sub-connection) States . 11
5.9. Use load balancer in Multipath QUIC . . . . . . . . . . . 11
6. Packet scheduling . . . . . . . . . . . . . . . . . . . . . . 11
6.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 11
6.2. Basic Static Scheduling Strategy . . . . . . . . . . . . 12
6.3. Dynamic (feedback-based) Scheduling Strategy . . . . . . 13
6.4. Application Policy-Awareness . . . . . . . . . . . . . . 14
6.4.1. Per-connection Policy . . . . . . . . . . . . . . . . 15
6.4.2. Per-stream Policy . . . . . . . . . . . . . . . . . . 17
7. Congestion control and loss detect . . . . . . . . . . . . . 18
7.1. Congestion control . . . . . . . . . . . . . . . . . . . 18
7.2. Packet number space and acknowledgements . . . . . . . . 18
7.3. Flow control . . . . . . . . . . . . . . . . . . . . . . 18
8. New frames . . . . . . . . . . . . . . . . . . . . . . . . . 18
8.1. MP_SUB_CONN_NEW frame . . . . . . . . . . . . . . . . . . 19
8.2. MP_SUB_CONN_ACCEPT frame . . . . . . . . . . . . . . . . 19
8.3. MP_SUB_CONN_CLOSE frame . . . . . . . . . . . . . . . . . 19
8.4. MP_ACK frame . . . . . . . . . . . . . . . . . . . . . . 20
8.5. MP_ADD_ADDRESS frame . . . . . . . . . . . . . . . . . . 21
8.6. MP_REMOVE_ADDRESS frame . . . . . . . . . . . . . . . . . 21
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
10. Informative References . . . . . . . . . . . . . . . . . . . 21
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
An, et al. Expires April 25, 2021 [Page 2]
Internet-Draft multipath-quic October 2020
1. Introduction
In this document, we propose an extension to the current QUIC design
to enable the simultaneous usage of multiple paths for a single
connection.
This proposal differs from past proposals
[I-D.deconinck-quic-multipath] in two fundamental perspectives:
o The multi-path QUIC is built on top of the concept of the
bidirectional sub-connection, which readily fits into the nature
of both cellular and wifi links that cover the majority of multi-
path applications in QUIC while keeping the design simple and easy
to implement. In doing so, we are able to re-use most of the
current QUIC transport design with the sole addition of six new
frames.
o The multi-path QUIC design enables feedback-based dynamic
scheduling strategy. As the major goal of multi-path QUIC is to
enhance performance in mobile applications, where the sender and
receiver may have different viewpoints about the fast-changing
wireless connectivity, especially in high-mobility scenarios, the
proposed design allows the sender and receiver to synchronize
their viewpoints via message exchange in ACK packet in order to
maximize performance.
This document is organized as follows. It first provides definition
of sub-connection in Section 3. It then specifies how to enable
multipath QUIC during handshake in Section 4, and sub-connection
management in Section 5. It discusses packet scheduling in
Section 6, and congestion control in Section 7. It specifies the new
frames in Section 8.
2. Notational Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
We assume that the reader is familiar with the terminology used in
[I-D.ietf-quic-transport]. In addition, we define the following
terms:
o Path: A sequence of links between a sender and a receiver, defined
in this context by a 4-tuple of source and destination address/
port pairs[RFC8684].
An, et al. Expires April 25, 2021 [Page 3]
Internet-Draft multipath-quic October 2020
o Sub-Connection: Sub-connection is bidirectional and provides
reliable transmission between client and server. A connection can
contain one or multiple sub-connections. A sub-connection is
identified by an internal identifier, called Sub-Connection Index
(SCI). Each sub-connection has its unique Source Connection ID
and Destination Connection ID. The Connection ID is mapped with
the 2-tuple of IP address and port.
o (Multipath QUIC) Connection: A set of one or more sub-connections,
over which an application can communicate between two host.
3. Sub-Connection
A connection can contain one or multiple sub-connections which are
bidirectional and provides reliable transmission between client and
server. Sub-connection is identified by Sub-Connection Index (SCI).
If a connection contains at least 2 sub-connections, then the first
established sub-connection is called Initial sub-connection. The
rest sub-connections are called supplementary sub-connections.
Every sub-connection has its own unique CID pair that is associated
with the 4-tuple (source IP, source port, destination IP, destination
port) of the underlying network path currently used by the sub-
connection. The Connection ID negotiation process is specified in
Section 5.1. In case of sub-connection migration, the CID pair will
be renegotiated following the connection migration procedure
specified in [I-D.ietf-quic-transport].
Endpoints can find which sub-connection a received packet belongs to
according to the CID pair of the packet. Endpoints can find the
context of a sub-connection by its' CID pair or SCI. In the context
of a sub-connection, a reference pointer MUST be provided to access
the context of the multipath QUIC connection that the sub-connection
belongs to.
Each sub-connection has its independent Packet Number Space. And all
sub-connections in the same connection share the same 1-RTT
encryption key which is generated during the connection's
cryptographic handshake.
Note: The reason of using SCI to identify a Sub-connection:
acknowledgements may not be transferred via the same sub-connection
where the packets were sent, therefore the MP_ACK frame SHOULD
contain field that can uniquely identify the sub-connection, and the
same logic applies to other new MP frames. If we use Connection ID
to identify a sub-connection in MP frames, the length of Connection
ID is too long and will add more overhead in the frames.
An, et al. Expires April 25, 2021 [Page 4]
Internet-Draft multipath-quic October 2020
4. Enable Multipath QUIC - Handshake
The connection handshake flow follows QUIC-Transport
[I-D.ietf-quic-transport], using the transport parameter to negotiate
multipath feature. The negotiation mechanism is similar to the
negotiation in [I-D.deconinck-quic-multipath] Section 5.1, while the
semantic of the transport parameter is different.
A new transport parameter is defined:
o name: max_sub_conn_index (0x40)
o value: a variable-length integer (1 to 8 bytes)
The value range and definition:
o 0: MP feature disabled
o [1, 2^31-1]: the maximum number of sub-connections
The value of SCI(sub-connection index) starts from 1 and increases by
1 when a new sub-connection is created. The value range of SCI is
[1, max_sub_conn_index]. The SCI of initial sub-connection is 1. A
multipath QUIC connection MUST NOT reuse any used SCI for new sub-
connections in its' lifetime.
If the peer does not carry the max_sub_conn_index(0x40) transport
parameter, which means the peer does NOT support multipath, endpoint
MUST fallback to QUIC-Transport [I-D.ietf-quic-transport] with single
path, and MUST NOT send any MP frames in the following packets.
5. Sub-connection Management
This section describes the details of sub-connection management.
5.1. Multipath QUIC Interaction
Figure 1 illustrates the Multipath QUIC interaction process.
Server Client Server
Path 2 Path 1
Initial (DCID=A, SCID=B,
ClientHello(max_sub_conn_index=4)) ->
Handshake(DCID=B, SCID=C,
<- EE(max_sub_conn_index=4))
An, et al. Expires April 25, 2021 [Page 5]
Internet-Draft multipath-quic October 2020
Pkt(DCID=B, SCID=C,
<- frames=[New_connection_id(E)])
Pkt(DCID=C, SCID=B,
frames=[New_connection_id(E)]) ->
(Detect new path 2)
Pkt(DCID=E,
frames=[MP_SUB_CONN_NEW(
<- SCI=2, X)])
Pkt(DCID=F,
frames=[MP_SUB_CONN_ACCEPT(
SCI=2, X)],
PATH_CHALLENGE(Y)) ->
Pkt(DCID=E,
<- frames=[PATH_CHALLENGE(Y)])
(Data transmission)
Pkt(DCID=E, pktnum=N1,
<- frames=[STREAM("Request 2")])
Pkt(DCID=F, pktnum=N2,
frames=[MP_ACK(SCI=2, N1),
STREAM("Response 2")]) ->
Pkt(DCID=C, pktnum=M1,
frames=[STREAM("Request 1")]) ->
Pkt(DCOD=E, pktnum=N3,
frames=[MP_ACK(SCI=2, N2),
<- STREAM("Request 3")])
Pkt(DCID=B, pktnum=M2,
frames=[MP_ACK(SCI=1, M1),
<- STREAM("Response 1")])
Pkt(DCID=F, pktnum=N4,
frames=[MP_ACK(SCI=2, N3),
STREAM("Response 3")]) ->
Pkt(DCID=C, pktnum=M3,
frames=[MP_ACK(SCI=1, M2),
MP_ACK(SCI=2, N4)]) ->
An, et al. Expires April 25, 2021 [Page 6]
Internet-Draft multipath-quic October 2020
Figure 1: Multipath QUIC interaction process
The process is composed of four phases.
A. Handshake negotiation
During the QUIC-Transport [I-D.ietf-quic-transport] handshake,
endpoints negotiate whether multipath feature is supported. The
negotiation parameter (see Section 4) is carried within the transport
parameters of TLS cryptographic handshake. After the handshake
finished, the connection contains the initial sub-connection with SCI
equals 1. In Figure 1, the maximum sub-connection index is four.
B. Exchange unused Connection ID in advance
After the two endpoints complete the connection establishment, they
can exchange unused Connection IDs by NEW_CONNECTION_ID frame.
Before an endpoint starts to create a new sub-connection, it SHOULD
check if there are unused Connection IDs for both endpoints.
Note: QUIC-Transport [I-D.ietf-quic-transport] requires Connection ID
is uniquely mapped with 2-tuple of IP address and port. If client
attempts to use a new 2-tuple as source address to establish a new
sub-connection, a new Connection ID is required for client, and also
a new Connection ID is required for server.
C. New sub-connection establishment
During this phase, a new sub-connection is established between client
and server, and address validation is needed.
When client detects a new network path, it MAY attempts to establish
a new sub-connection by sending MP_NEW_SUB_CONN frame which carries a
64-bit random value and claims the new sub-connection's SCI (which is
2 in the example flow in Figure 1). The establishment of sub-
connection is always initiated by client.
After the server receives the MP_NEW_SUB_CONN frame from the client,
it responds with MP_SUB_CONN_ACCEPT frame which carries the identical
64-bit random value from the received MP_NEW_SUB_CONN frame and
agrees with the sub-connection's SCI (2 in the example). Server MUST
also perform path validation following the procedure specified in
QUIC-Transport [I-D.ietf-quic-transport]. Once the server
successfully validates its' peers' address, the new sub-connection is
established.
D. Data transmission on new sub-connection
An, et al. Expires April 25, 2021 [Page 7]
Internet-Draft multipath-quic October 2020
As soon as sub-connections are established, endpoints can communicate
with each others over the newly established sub-connections. All
valid short header packets defined in QUIC-Transport
[I-D.ietf-quic-transport] can be carried on these sub-connections.
Every sub-connection has its' independent PNS. Thus, standard QUIC
ACK frames defined in QUIC-Transport [I-D.ietf-quic-transport] only
acknowledge packets that belong to the same PNS of the sub-connection
on which the ACK frames were received.
To enable endpoints reply acknowledgements on different sub-
connections rather than the sub-connection where the corresponding
packets were received, a new type of frame, MP_ACK, is defined.
MP_ACK frames can also be replied over the same sub-connection on
which data packets were received. In this case, MP_ACK frames serves
very similar purposes as QUIC ACK frames do.
MP_ACK frame contains the sub-connection index of the packets to be
acknowledged. For example, in Figure 1, the packet (packet number is
N4) is sent via the second sub-connection (SCI is 2), and its
corresponding acknowledgement MP_ACK(Sub-Connection Index=2, N4) is
sent via the initial sub-connection.
5.2. Path validation and sub-connection ID negotiation
Before clients initiate new sub-connections by sending
MP_SUB_CONN_NEW frames to servers through their additional network
addresses, they MAY want to validate the reachability between their
new network addresses and servers' addresses. In this case, clients
can initiate a path validation procedure as specified in QUIC-
Transport [I-D.ietf-quic-transport] per address pair.
Path validation uses the PATH_CHALLENGE and PATH_RESPONSE frame
defined in QUIC-Transport [I-D.ietf-quic-transport].
Each sub-connection MUST has a unique pair of SCID and DCID within a
multipath QUIC connection. Thus, endpoints MUST NOT initiate or
accept new sub-connections if they currently have no free CIDs
supplied by their peers. In this case, endpoints SHOULD announce new
free CIDs to their peers by exchanging NEW_CONNECTION_ID frames.
To ensure that endpoints have free CIDs to create new sub-connections
as soon as they get new network addresses, an endpoint SHOULD
announce a least one free CID to its peer by sending
NEW_CONNECTION_ID frame [I-D.ietf-quic-transport] over its initial
sub-connection as soon as the handshake on the initial sub-connection
is completed. Endpoints MAY also track the number of free CIDs that
their peers can use and announce more free CIDs if needed.
An, et al. Expires April 25, 2021 [Page 8]
Internet-Draft multipath-quic October 2020
Sub-connection ID negotiation follows the Connection ID negotiation
method in Connection Migration defined in QUIC-Transport
[I-D.ietf-quic-transport], which is to let client and server claim
its own unused Connection ID in advance by NEW_CONNECTION_ID frame.
If there is no available unused Connection ID, then establishment of
new sub-connection is not allowed.
5.3. New sub-connection establishment
New sub-connection establishment is always initiated by client, by
sending MP_NEW_SUB_CONN frame.
Because source address(2-tuple of IP address and port) is usually
different in the new network path, client needs to generate and claim
new Source Connection IDs prior to the new sub-connection
establishment.
Client that sends the MP_SUB_CONN_NEW frame in 1-RTT packets with
short headers, MUST use the unused Connection ID claimed in advance
by server as Destination Connection ID. MP_SUB_CONN_NEW frame
carries a 64-bit random value, and a SCI (increased progressively).
After receiving the MP_SUB_CONN_NEW frame, server responds with
MP_SUB_CONN_ACCEPT frame carrying the identical SCI and identical
64-bit random value from the received MP_NEW_SUB_CONN frame. Then,
server sends PATH_CHALLENGE to verify the client address.
After client receives the PATH_CHALLENGE frame, it replies with
PATH_RESPONSE frame In the following 1-RTT packet (short header) to
complete the address validation. After the address validation is
completed, client and server can send and receive data unrestrictedly
on the established sub-connection.
Before the client's address validation is completed, server needs to
limit the cumulative size of packets it sends to an unvalidated
address to three times the size of packets it receives from that
address in the new sub-connection (to prevent amplification attack).
5.4. Close sub-connection
Both client and server can terminate a sub-connection, by sending
MP_SUB_CONN_CLOSE frame that carries a SCI. In scenarios such as
client detects the network environment change (client's 4G/Wi-Fi is
turned off, Wi-Fi signal is fading to a threshold), or endpoints
detect that the quality of RTT or loss rate is becoming worse, client
or server can terminate a sub-connection immediately.
An, et al. Expires April 25, 2021 [Page 9]
Internet-Draft multipath-quic October 2020
MP_SUB_CONN_CLOSE frame can be sent via a different sub-connection
instead of the sub-connection to be closed.
5.5. Sub-connection Lookup
Endpoints use Connection IDs to find the context of a connection.
Figure 2 illustrates the Connection context. Each sub-connection's
Connection IDs can be mapped to the connection.
Connection
+--------------------------------------------------------------+
| |
| Sub-Conn 1 |
| |
+------+ IP 1, Port 1 SCI 1 - SCID A, DCID B IP 3, Port 3 +------+
| |<----------------------------------------------------->| |
|Client| |Server|
| |<----------------------------------------------------->| |
+------+ IP 2, Port 2 SCI 2 - SCID C, DCID D IP 3, Port 3 +------+
| |
| Sub-Conn 2 |
| |
+--------------------------------------------------------------+
Figure 2: Connection context
In the connection context, client and server can use SCI or
Connection ID to find a sub-connection. Note that if sub-connection
migration happens, sub-connection's Connection ID need to be
renegotiated (See Section 5.6), while the SCI of sub-connection could
remain unchanged.
5.6. Sub-connection migration
Sub-connection migration follows the Connection Migration defined in
QUIC-Transport [I-D.ietf-quic-transport]. When client experiences
NAT rebinding (source address is changed), server needs to revalidate
the client address.
5.7. Sub-connection state machine management
TODO
An, et al. Expires April 25, 2021 [Page 10]
Internet-Draft multipath-quic October 2020
5.8. Sender and Receiver Connection (Sub-connection) States
For each sender and receiver, the sub-connection states include:
+---------+----------------------+----------+--------------+--------+
| Sender | SubConnectionIndex(S | CIDs(SCI | 4-tuple(sIP, | packet |
| | CI) | D, DCID) | dIP, sPort, | number |
| | | | dPort) | space |
+---------+----------------------+----------+--------------+--------+
| Receive | SubConnectionIndex(S | CIDs(SCI | 4-tuple(sIP' | packet |
| r | CI) | D, DCID) | , dIP', | number |
| | | | sPort', | space |
| | | | dPort') | |
+---------+----------------------+----------+--------------+--------+
Table 1: Sender and Receiver Connection (Sub-connection) States
5.9. Use load balancer in Multipath QUIC
This specification follows the Connection ID negotiation defined in
QUIC-Transport [I-D.ietf-quic-transport]. For stateless or low-state
load balancers supporting Multipath QUIC, implementations SHOULD use
the specification of Connection ID generation and Load balancer
routing defined in QUIC-LB [I-D.ietf-quic-load-balancers], guarantee
that packets with Connection IDs belonging to the same connection,
can be routed to same server.
6. Packet scheduling
6.1. Overview
For an outgoing packet, the packet scheduler decides which sub-
connection the packet shall be transmitted. The concept of packet
scheduler in Multipath QUIC is similar to that in MPTCP. As long as
more than one path's congestion controller allows for a new packet
transmission, the packet scheduler is enabled. However, the proposed
packet scheduler in this draft differs from past MPTCP proposals in
the following aspects:
o We enable dynamic (feedback-based) scheduling strategy with
feedback in this proposal to further enhance quality of user
experience(QoE) and to facilitate the expression of the
application policy-awareness. Such a capability is made available
by adding the QoE control signal length field and QoE control
signal field in MP_ACK frame (see Section 8.4). With the help of
such extension, a receiver is able to interact with a sender's
scheduling strategy in real time.
An, et al. Expires April 25, 2021 [Page 11]
Internet-Draft multipath-quic October 2020
o Unlike MPTCP which send ACK packet over the same path, multipath
QUIC allows a packet to be acknowledged over a different path,
which allows multipath QUIC to better handle the uplink-downlink
heterogeneity in wireless networks.
o We support application policy-awareness in multipath QUIC. An
application can implement both per-connection policies and per-
stream policies. For example, a live streaming application is
allowed to choose a different policy from a web application. The
per-connection policy includes path mode, path preference,
scheduling algorithm and packet redundancy strategy. A per-stream
policy is associated with user-defined stream priorities to
express the applications's intent.
6.2. Basic Static Scheduling Strategy
A basic static scheduling strategy consists of four major components:
o Path mode: A scheduler may want to decide which sub-connections
shall be activated to transmit data. For instance, a scheduler
can choose to use only one of the two sub-connections and
completely ignore the other one. A scheduler marks the selected
sub-connections to be in the "active" state and the un-selected
ones in the "inactive" state.
o Path preference: Due to the fact that costs of transmitting data
over different sub-connections are not always equal. For example,
the energy (battery) cost over a 5G sub-connection and a Wi-Fi
sub-connection are very different, so a user may prefer the Wi-Fi
sub-connection when his/her cell phone's battery is low. In
another example, transmissions over a Wi-Fi sub-connection and a
cellular sub-connection may incur different service charges per
packet such that a user prefers to use the Wi-Fi sub-connection
over the LTE one. Note that a user's preference may change over
time. For instance, certain mobile carriers offer unlimited free
data for a particular streaming app. Therefore, the sub-
connection priority should be made available in the scheduler.
o Sub-connection selection algorithm:A selection algorithm
splits packets across different sub-connections and determines the
order of sub-connections to be selected. The selection algorithm
takes congestion controller states as inputs, such as smoothed
RTTs (sRTTs), estimated bandwidths (eBWs) and congestion window
sizes (CWNDs) as well as application-defined information such as
sub-connection priorities and path states. The outputs of the
algorithm is an ordered list of sub-connections to put a packet
on. To name a few, some of the commonly used algorithms are:
An, et al. Expires April 25, 2021 [Page 12]
Internet-Draft multipath-quic October 2020
o
* Round-Robin: There is no priority. it selects sub-connections
one by one in order to transmit data.
* Lowest-RTT: It first chooses the sub-connection with the lowest
RTT and feeds packets to it until that sub-connection's
congestion window is full. Then it chooses the sub-connection
with the second lowest RTT.
* Highest-Sending-Rate: It first chooses the sub-connection with
the highest bandwidth and feeds packets to it until that sub-
connection's congestion window is full. Then it chooses sub-
connection with the second largest bandwidth.
o Packet redundancy strategy: One major challenge in multi-path
transmission is that a packet loss on the slow sub-connection
might block the overall transmission when packets are split across
fast-changing sub-connections. As the sub-connection selection
algorithm takes inputs from congestion controllers which are
basically rough predictions of the network and may not be accurate
enough for fast-changing wireless channels, such an imprecise
estimation could lead to network overuse/underuse. A solution to
this problem is to implement packet redundancy strategy. A
redundancy strategy can be applied to only ACK packets (partial
redundancy) or all data packets (full redundancy). The multipath
QUIC in this draft is designed to enable such flexible redundancy
strategies. It is up to the application to determine whether,
when, and on which packets to activate transmission redundancy.
6.3. Dynamic (feedback-based) Scheduling Strategy
An important feature of this proposal is the capability of dynamic
(feedback-based) scheduling. In a dynamic scheduling strategy, a
receiver notifies its currently preferred scheduling strategy to a
sender. Such feedback information is carried by QoE control signal
in MP_ACK frames. The frequency of such feedback can be controlled
to limit the amount of extra information. To do so, four types of
MP_ACK frames are designed (Figure 8):
o Type(i) = 0x22 , with no ECN Counts and no QoE Control Signals
o Type(i) = 0x23 , with ECN Counts and no QoE Control Signals
o Type(i) = 0x24 , with no ECN Counts and QoE Control Signals
o Type(i) = 0x25 , with ECN Counts and QoE Control Signals
An, et al. Expires April 25, 2021 [Page 13]
Internet-Draft multipath-quic October 2020
The type 0x24 and 0x25 give the flexibility of carrying QoE control
signals. Given that the sender and the receiver may have different
views of the wireless environments, especially in high-mobility
scenarios, the QoE control signal allows a synchronization between
their viewpoints dynamically. It is up to the application to
determine the interpretation of QoE control signal and its encoding
method.
6.4. Application Policy-Awareness
Applications may have completely different QoE requirements---the
interactive applications are delay sensitive, while the video
streaming applications are more throughput sensitive. There is thus
a trend of cross-layer design that tries to take applications'
demands into account when managing paths or scheduling packets. The
static scheduling strategy and the dynamic scheduling strategy are
used together to fully support application policy-awareness in
multipath scheduling. To be more specifically, a 'control plane' is
separated from a `data plane' as in software-defined networking. The
'control plane' takes applications' high-level demands (a.k.a intent)
as input to generate the corresponding policies, which later are
deployed on the 'data plane'. The 'data plane' maps users policies
to the 'actions', which control the packet scheduler and other
functionalities that the transport implements. To allow maximum
design flexibility, the proposed multipath QUIC let applications to
access/change every single logic of the packet scheduling and path
management. The application policy consists of two layers: per-
connection policy and per-stream policy.
An, et al. Expires April 25, 2021 [Page 14]
Internet-Draft multipath-quic October 2020
per-stream intent per-conn policy
Control e.g. stream prioirity e.g. path preference, path mode,
Plane deadline-aware etc
| |
-----------|----------------------------------------|-------------
V V
Data +--------+ - +------------------------+
Plane | | | | |
+--------+ | | +-----+ |
| | +->| ... | |
+--------+ | | | +-----+ |
| | | | | +-----+ |
+--------+ | | +->| ... | |
| | | +-----+ |
+--------+ | +----------+ chunks |+----------+ | |
| | ->| stream |------->|| packet |-+ ... |
+--------+ | |scheduling| ||scheduling| | |
... | +----------+ |+----------+ | +-----+ |
+--------+ | | +->| ... | |
| | | | | +-----+ |
+--------+ | | | +-----+ |
| | +->| ... | |
+--------+ | | +-----+ |
| | | | path manager |
+--------+ - +------------------------+
streams
Figure 3: Application Policy Awareness in Multi-path QUIC framework
6.4.1. Per-connection Policy
An application imposes per-connection policy through the primitives
provided by the control plane.
As described above, the policy is translated into indications on sub-
connection states, sub-connection priorities, sub-connection
selection algorithms and packet redundant strategies. The packet
scheduler at the data plane will act based on these indications. We
assume the policies are 'soft'---the policies are not a must.
Instead, the data plane will follow the policies as much as possible.
An, et al. Expires April 25, 2021 [Page 15]
Internet-Draft multipath-quic October 2020
+-----+-----------------+-------------+------------+----------------+
| No. | Application | Application | Underlying | Underlying |
| | defined policy: | defined | action: | action: Path |
| | Path mode | policy: | Packet | mngm. |
| | | Path | Scheduling | |
| | | Preference | | |
+-----+-----------------+-------------+------------+----------------+
| 1 | Wi-Fi=full, | Wi-Fi=1, | full | / |
| | Cellular=full | Cellular=1 | redundant | |
| | | | | |
| 2 | Wi-Fi=full, | Wi-Fi=1, | full | activate |
| | Cellular=backup | Cellular=1 | redundant | backup |
| | | | | interfaces |
| | | | | when the |
| | | | | active one's |
| | | | | performance is |
| | | | | lower than X |
| | | | | for 5s |
| | | | | |
| 3 | Wi-Fi=full, | Wi-Fi=2, | partially | / |
| | Cellular=full | Cellular=1 | redundant | |
+-----+-----------------+-------------+------------+----------------+
Table 2: Example policies of a real-time interaction application
Let us take real-time interaction applications as an example to
illustrate the basic idea. The applications are indeed delay
sensitive but data volume is often low. 3 types of policies may be
used by different applications, as shown in Table 2 where we assume
only two paths are available (Wi-Fi and Cellular)
The first type of policies would like to use two paths equally, and
because the applications are delay sensitive, the actions will be to
active 'full redundancy' for the packet redundancy strategy---two
paths send the same data. The second type of policies, on the other
hand, would like to use the Wi-Fi interface (possibly because of data
charge) as much as possible, hence giving the Wi-Fi sub-connection a
higher priority. But if two paths have to be activated at the same
time due to the lower performance of Wi-Fi, then the two paths are
set with same the priority which can be configured dynamically
through QoE control signal in MP_ACK feedbacks. The third type of
policies would like to use the two interfaces at the same time, but
Wi-Fi is preferred twice as the cellular one. The actions will take
this into consideration, by implementing a weighed round-robin sub-
connection selection algorithm.
An, et al. Expires April 25, 2021 [Page 16]
Internet-Draft multipath-quic October 2020
Likewise, we can define a mapping between the policies of different
types of applications and the actions in the data plane. We leave
the design of such a mapping to the designers.
6.4.2. Per-stream Policy
Per-stream intent is a unique feature provided by (MP)QUIC---it is
implemented through the multiple streams in QUIC. Streams can be
associated with priorities to implement applications intent. For
instance, objects in a web page may be dependent on others and thus
have different priorities [MPQUIC-Scheduler]. A stream priority-
aware packet scheduling algorithm will improve the performance
notably.
High priority /\ +---------+
|| | |
|| +---------+
|| +---------+
|| | |
|| +---------+
|| ... User-defined priority
|| +---------+
Low priority || | |
|| +---------+
-----------------------------------------------------------
High priority /\ +---------+
|| | |
|| +---------+
|| +---------+
|| | |
|| +---------+
|| ... Default priority
|| +---------+
Low priority || | |
|| +---------+
Figure 4: Stream priority
We envision a priority management scheme of two separated priority
ranges (see Figure 4). The user-defined priority ranges are those
streams that the applications explicitly designate the priorities,
where the default priority ranges include the streams with no
priority values set by the applications. Only when the streams in
the user-defined ranges have no data sent, the data in the streams in
the default priority ranges can be sent. In the same range, one can
use the weighted round robin for scheduling---the higher-priority
An, et al. Expires April 25, 2021 [Page 17]
Internet-Draft multipath-quic October 2020
streams get more quantum for data sending in each round. One can
also dynamically set/change the priorities of the streams in the
default priority ranges to enable short stream first if needed.
7. Congestion control and loss detect
7.1. Congestion control
Implementations MAY support coupled congestion controllers such as
LIA [MPTCP-LIA], OLIA [MPTCP-OLIA]s, and etc., or support decoupled
congestion controllers in environments using disjoint network paths.
In decoupled congestion control, every sub-connection runs its own
congestion controller without interacting with the congestion
controllers of other sub-connections. That is to say, in the aspect
of congestion control, a sub-connection behaves exactly the same as a
normal QUIC connection over the same network path.
Each sub-connection MAY choose congestion control algorithm
independently.
7.2. Packet number space and acknowledgements
Every sub-connection has its' own packet number space for
transmitting 1-RTT packets.
ACK frame [I-D.ietf-quic-transport] MUST be returned via the same
sub-connection on which the corresponding packets were sent.
MP_ACK frame can be returned via either a different sub-connection,
or the same sub-connection, based on different strategies of sending
MP_ACK frames.
Note: Only MP_ACK frame returned via the same sub-connection can be
used to calculate RTT(round trip time).
7.3. Flow control
TODO
8. New frames
All the MP frames MUST be sent in 1-RTT packet, and MUST NOT use
other encryption levels.
If an endpoint receives MP frames from packets of other encryption
levels, it MAY return MP_PROTOCOL_VIOLATION as a connection error and
close the connection.
An, et al. Expires April 25, 2021 [Page 18]
Internet-Draft multipath-quic October 2020
8.1. MP_SUB_CONN_NEW frame
MP_SUB_CONN_NEW frame(type=0x2a) is used to establish a new sub-
connection. The MP_SUB_CONN_NEW frame will specify a SCI and include
a 64-bit random value.
MP_SUB_CONN_NEW frames are formatted as shown in Figure 5.
MP_SUB_CONN_NEW Frame {
Type (i) = 0x2a,
Sub_Connection_Index (i),
Data (64),
}
Figure 5: MP_SUB_CONN_NEW Frame Format
8.2. MP_SUB_CONN_ACCEPT frame
MP_SUB_CONN_ACCEPT frame (type=0x2b) is used by endto accept a new
sub-connection, as a response to MP_NEW_SUB_CONN frame.
MP_SUB_CONN_ACCEPT frames are formatted as shown in Figure 6, which
is identical to the MP_NEW_SUB_CONN frame (Section 8.1).
MP_SUB_CONN_ACCEPT Frame {
Type (i) = 0x2b,
Sub_Connection_Index (i),
Data (64),
}
Figure 6: MP_SUB_CONN_ACCEPT Frame Format
8.3. MP_SUB_CONN_CLOSE frame
MP_SUB_CONN_CLOSE frame(type=0x2c..0x2d) is used to close a sub-
connection, which is formatted by adding a SCI field to QUIC-
Transport [I-D.ietf-quic-transport] CONNECTION_CLOSE frame. The SCI
is used to distinguish sub-connections, so each sub-connection can be
closed independently.
MP_SUB_CONN_CLOSE frames are formatted as shown in Figure 7.
An, et al. Expires April 25, 2021 [Page 19]
Internet-Draft multipath-quic October 2020
MP_SUB_CONN_CLOSE Frame {
Type (i) = 0x2c..0x2d,
Sub_Connection_Index (i),
Error Code (i),
[Frame Type (i)],
Reason Phrase Length (i),
Reason Phrase (..),
}
Figure 7: MP_SUB_CONN_CLOSE Frame Format
8.4. MP_ACK frame
MP_ACK frame allows for acknowledgements on different sub-
connections.
MP_ACK frame is formatted by adding a SCI field and QoE signal fields
to QUIC-Transport [I-D.ietf-quic-transport] ACK frame.
MP_ACK frames are formatted as shown in Figure 8.
MP_ACK Frame {
Type (i) = 0x22..0x23..0x24..0x25,
Sub_Connection_Index(i),
Largest Acknowledged (i),
ACK Delay (i),
ACK Range Count (i),
First ACK Range (i),
ACK Range (..) ...,
[ECN Counts (..)],
[QoE Control Signals Length(8)],
[QoE Control Signals (..)],
}
Figure 8: MP_ACK Frame Format
Type(i) = 0x22 , with no ECN Counts and no QoE Control Signals
Type(i) = 0x23 , with ECN Counts and no QoE Control Signals
Type(i) = 0x24 , with no ECN Counts and QoE Control Signals
Type(i) = 0x25 , with ECN Counts and QoE Control Signals
An, et al. Expires April 25, 2021 [Page 20]
Internet-Draft multipath-quic October 2020
8.5. MP_ADD_ADDRESS frame
TODO
8.6. MP_REMOVE_ADDRESS frame
TODO
9. IANA Considerations
This document makes no request of IANA.
10. Informative References
[I-D.deconinck-quic-multipath]
De Coninck, Q. and O. Bonaventure, "Multipath Extensions
for QUIC (MP-QUIC)", draft-deconinck-quic-multipath-05
(work in progress), August 2020.
[I-D.ietf-quic-load-balancers]
Duke, M. and N. Banks, "QUIC-LB: Generating Routable QUIC
Connection IDs", draft-ietf-quic-load-balancers-04 (work
in progress), August 2020.
[I-D.ietf-quic-transport]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", draft-ietf-quic-transport-32 (work
in progress), October 2020.
[MPQUIC-Scheduler]
Wang, J., Gao, Y., and C. Xu, "A Multipath QUIC Scheduler
for Mobile HTTP/2", Proceedings of the 3rd Asia-Pacific
Workshop on Networking 2019 (APNet '19). Association for
Computing Machinery, New York, NY, USA, 43-49., 2019,
<https://doi.org/10.1145/3343180.3343185>.
[MPTCP-LIA]
Raiciu, C., Handly, M., and D. Wischik, "Coupled
Congestion Control for Multipath Transport Protocols",
RFC 6365, October 2011,
<https://tools.ietf.org/html/rfc6356>.
[MPTCP-OLIA]
Khalili, R., Gast, N., and J. Boudec, "Opportunistic
Linked-Increases Congestion Control Algorithm for MPTCP",
July 2014, <https://datatracker.ietf.org/doc/html/draft-
khalili-mptcp-congestion-control-05>.
An, et al. Expires April 25, 2021 [Page 21]
Internet-Draft multipath-quic October 2020
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8684] Ford, A., Raiciu, C., Handley, M., Bonaventure, O., and C.
Paasch, "TCP Extensions for Multipath Operation with
Multiple Addresses", RFC 8684, March 2020,
<https://tools.ietf.org/html/rfc8684>.
Authors' Addresses
Qing An
Alibaba Inc.
Email: anqing.aq@alibaba-inc.com
Yanmei Liu
Alibaba Inc.
Email: miaoji.lym@alibaba-inc.com
Yunfei Ma
Alibaba Inc.
Email: yunfei.ma@alibaba-inc.com
Zhenyu Li
ICT-CAS
Email: zyli@ict.ac.cn
An, et al. Expires April 25, 2021 [Page 22]