Internet DRAFT - draft-zmlk-quic-te
draft-zmlk-quic-te
TODO Z. Zheng
Internet-Draft Alibaba Inc.
Intended status: Experimental Y. Ma
Expires: 11 May 2024 Uber Technologies Inc.
Y. Liu
Alibaba Inc.
M. Kühlewind
Ericsson
8 November 2023
QUIC-enabled Service Differentiation for Traffic Engineering
draft-zmlk-quic-te-01
Abstract
This document defines a method for supporting QUIC-enabled service
differentiation for traffic engineering through multipath and QUIC
connection identifier (CID) encoding. This approach enables end-host
networking stacks and applications to select packet routing paths in
a wide area network (WAN), potentially improving end-to-end
performance, cost, and reliability. The proposed method can be used
in conjunction with segment-routing traffic engineering technologies,
such as SRv6 TE.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 11 May 2024.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
Zheng, et al. Expires 11 May 2024 [Page 1]
Internet-Draft quic-te November 2023
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Conventions and Definitions . . . . . . . . . . . . . . . 3
2. Alternatives to support service differentiation in traffic
engineering . . . . . . . . . . . . . . . . . . . . . . . 4
3. Using Multipath QUIC . . . . . . . . . . . . . . . . . . . . 4
3.1. New Transport Parameter . . . . . . . . . . . . . . . . . 4
3.2. QUIC Connection Setup . . . . . . . . . . . . . . . . . . 5
3.3. Paths Setup with More Priorities . . . . . . . . . . . . 5
3.4. Effect of CID rotation . . . . . . . . . . . . . . . . . 6
4. CID Generation and Parsing . . . . . . . . . . . . . . . . . 6
4.1. Overview of End-to-end Network . . . . . . . . . . . . . 6
4.2. CID Generation on the Server Side . . . . . . . . . . . . 7
4.3. CID Generation on the Client Side . . . . . . . . . . . . 7
4.4. CID Parsing . . . . . . . . . . . . . . . . . . . . . . . 8
5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 8
5.1. Packet Flow from Server to Client . . . . . . . . . . . . 8
5.2. Deployment of CID Parser . . . . . . . . . . . . . . . . 10
6. Security Considerations . . . . . . . . . . . . . . . . . . . 10
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
8. Normative References . . . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12
1. Introduction
This draft outlines a method that enables a WAN to manage packets
from a single QUIC connection with differentiated traffic engineering
policies at packet-level granularity. For instance, a WAN can offer
two priority routing paths for packets from the same QUIC connection:
(1) it directs first-time transmitted packets on a default-priority,
low-cost path, and (2) it routes retransmitted packets and handshake
packets on a high-priority (low-latency) but more expensive path.
Consequently, this approach achieves cost-efficiency and reduces tail
latency simultaneously.
While there are existing solutions to support Differentiated Service
(DiffServ), such as using Differentiated Services Code Point (DSCP)
or opening multiple ports to enable QoS control, they have several
Zheng, et al. Expires 11 May 2024 [Page 2]
Internet-Draft quic-te November 2023
limitations. For example, DSCP values may be modified by an ISP or a
network service provider, and opening multiple ports raises security
concerns (more details as discussed in Section 2).
The solution proposed in this draft can overcome these limitations,
as it is designed with the following properties: (1) The service
differentiation encoding is immune to modifications by middleboxes.
(2) It provides traffic engineering control at packet-level
granularity within a connection. (3) A user-space application can
directly control each packet's priority without the kernel's special
support. (4) It does not change the deployment method, as current
QUIC applications have.
In a nutshell, this proposal establishes multiple routing paths at
different priority levels inside a single QUIC connection and encodes
a packet's routing priority in its CID. Specifically, this proposal
reuses mechanisms from Multipath QUIC [Multipath-QUIC], but makes two
major changes: (1) Multiple QUIC paths are allowed to be created
between a single-homed client and server (e.g., the client and server
can both have only one (IP, port) address). (2) This proposal
leverages different CIDs to describe various priority levels for QUIC
packets that belong to the same connection.
In this draft, QUIC paths can share the same perceived 4-tuple, but
as long as their underlying physical routing paths are not the same,
they are treated as different paths. This proposal shares many key
similarities to Multipath QUIC: (1) Senders manage per-path
congestion status; (2) RTT measurements are per-path. (3) Each path
uses a separate packet number space (PNS); (4) A path ID is the
sequence number of the CID used to send packets. (5) Once multipath
is negotiated, ACK_MP frame is used to acknowledge packets in
multiple packet number spaces.
The proposed solution is intended to work with edge routers that have
segment routing capabilities, such as SRv6 [RFC8986]. Note that the
QUIC CID is not intended to be directly used for routing in Cloud
WAN, but SHOULD be translated to SRv6 SID or an MPLS label by an edge
router for traffic engineering purposes.
1.1. Conventions and Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document uses the following terms:
Zheng, et al. Expires 11 May 2024 [Page 3]
Internet-Draft quic-te November 2023
* Cloud WAN: The backbone network provided and controlled by a Cloud
provider, or the backbone network operated by an application
service provider.
* Edge router: Edge router of a backbone network.
* QUIC path: The path described in Multipath QUIC protocol, which is
a logical end-to-end transmission path created by the client.
* Routing path: The network path that a packet physically traverses,
also specified as the Cloud WAN path in this document.
2. Alternatives to support service differentiation in traffic
engineering
This section describes existing solutions that offer service
differentiation in traffic engineering and their limitations when
applied to the internet.
The most commonly used solution is to use the DSCP packet marking
[RFC8837] for traffic engineering. However, such a solution faces
several limitations: (1) A DSCP marker is subject to modifications by
an ISP or a network service provider. (2) Marking DSCP on mobile
devices may require kernel changes. (3) DSCP is marked at the
granularity of connection or session, which means packets with the
same 4-tuple are treated with the same priority, and hence, it cannot
support packet-level QoS control.
The second solution is to use multiple ports, either source or
destination ports, to differentiate QoS between two endpoints. When
this solution is used, packets with different priorities are sent to
sockets bound to different ports. However, such a solution also
faces several drawbacks. Source ports may be modified by a NAT in
the network, which changes the priority. Using destination ports to
support QoS raises security concerns, as opening many ports increases
the potential attack surface for cyber threats. Destination ports
other than some well-known ports (e.g., 80, 433) may also be subject
to blocking by firewalls.
3. Using Multipath QUIC
3.1. New Transport Parameter
This draft defines a new transport parameter to claim the adoption of
priority paths. It is negotiated during QUIC handshake as described
in [RFC9000]. The new parameter is defined as PRIORITY_PATHS. This
parameter is used to indicate the number of priority levels:
Zheng, et al. Expires 11 May 2024 [Page 4]
Internet-Draft quic-te November 2023
* 0, 1: Only one path is available, i.e., multiple priority levels
are not considered.
* n (n > 1): n paths are available, i.e., n priority levels are
available for use.
When the client supports PRIORITY_PATHS, it sends a value greater
than 1 to the server side. After receiving it, the server returns
the configured value of PRIORITY_PATHS to the client. The
PRIORITY_PATHS value that a client uses MUST be consistent with the
returned value from the server, i.e., the server determines how many
priority paths can be used.
The server and CID parser share the same PRIORITY_PATHS values, which
are configured by the control plane. After negotiation, the client
obtains the same value as the server.
3.2. QUIC Connection Setup
When a QUIC client starts a new connection, it uses a CID generated
using the default priority (defined as 0). If PRIORITY_PATHS is
supported, a server SHOULD respond with a CID that represents the
default priority and indicate to the client that it supports
PRIORITY_PATHS.
3.3. Paths Setup with More Priorities
After QUIC connection negotiation is completed, endpoints need to
exchange CIDs with each other. These CIDs are generated using the
method described in Section 4. Figure 1 illustrates an example of
this process. The client sends CIDs with priority 1 and priority 2
(C-CID1 and C-CID2), and the server also sends CIDs representing
these two priority levels (S-CID1 and S-CID2).
Client Server
(CID exchange)
NEW_CONNECTION_ID[C-CID1]
NEW_CONNECTION_ID[C-CID2] --->
NEW_CONNECTION_ID[S-CID1]
<--- NEW_CONNECTION_ID[S-CID2]
(New path init with priority 1)
PATH_CHALLENGE[S-CID1] --->
<--- PATH_RESPONSE,PATH_CHALLENGE[C-CID1]
PATH_RESPONSE[S_CID1] --->
Zheng, et al. Expires 11 May 2024 [Page 5]
Internet-Draft quic-te November 2023
Figure 1: An example of CID exchange and path setup
After the CID exchange is completed, the client selects S-CID1 to
establish a new path of priority 1 via PATH_CHALLENGE frame. Even
though the packet that uses S-CID1 comes from the same 4-tuple as the
default path (i.e., priority 0), the server SHOULD treat it as a new
path establishment request and respond with PATH_RESPONSE and
PATH_CHALLENGE frames using a CID of the same path priority, i.e.,
C-CID1.
3.4. Effect of CID rotation
When an endpoint rotates to a new CID on an existing path, it will
not send PATH_CHALLENGE frame and the new CID does not change the
path priority, so the peer will not treat this new CID as an attempt
to create a new path.
4. CID Generation and Parsing
4.1. Overview of End-to-end Network
+--------+
| Server |
+--------+
|
| Cloud DCN
|
+------+------+
| CID parser2 |
+------+------+
|
| Cloud WAN
|
+------+------+
| CID parser2 |
+------+------+
|
| Internet access
| network
|
+----+---+
| Client |
+--------+
Figure 2: End-to-end network
Zheng, et al. Expires 11 May 2024 [Page 6]
Internet-Draft quic-te November 2023
QUIC packets traverse the end-to-end network as described in
Figure 2. This network includes three parts: internet access
network, Cloud WAN, and Cloud DCN.
The Cloud WAN has various routing paths with different performances,
thus priority scheduling is applied. The client and server assign
different priorities to the CIDs they generate and send. At the
Cloud WAN edge, two CID parsers are responsible for parsing the
priority of incoming packets and forwarding them to the appropriate
routing path.
4.2. CID Generation on the Server Side
There are two purposes for encoding the CID: 1. The client and
server can understand the priority represented by the CID; 2. There
MUST NOT be a correlation between the CID and its priority, which is
important to prevent privacy leaks if an attacker observes.
Their CIDs are generated on the server side for packets from the
client to the server. The format of an encoded CID on the server
side is defined below:
CID {
Random Bits (32 .. 152),
Priority Bits (8),
}
To prevent the priority value from being observed by an attacker, the
server uses encryption (e.g., AES-128-ECS cipher) to obscure the CID,
which is similar to [QUIC-LB]. The encryption key and length are
pre-configured on the server.
Note that when the generated CID is sent to the client through
NEW_CONNECTION_ID frame, the Sequence Number in this frame needs to
satisfy priority = sequence_number % n (n is described in
Section 3.1). The client will use Sequence Number to calculate
priority.
4.3. CID Generation on the Client Side
Their CIDs are generated on the client side for packets from the
server to the client. Its format is the same as the definition in
[RFC9000], which is completely random.
When the server sends QUIC packets with these CIDs, an additional
byte is appended before the first byte of a CID to indicate the
priority. The CID parser 2 will remove this byte after receiving
these packets from the server. Therefore, CIDs within these packets
Zheng, et al. Expires 11 May 2024 [Page 7]
Internet-Draft quic-te November 2023
remain completely random as they were generated on the client side.
As they traverse through the Cloud WAN and internet access network,
the priority cannot be observed. Moreover, the DCN operates as a
private network that attackers typically cannot access.
4.4. CID Parsing
CID parser 1 parses QUIC packets sent from the client to the server.
These packets contain encrypted CIDs. Thus, parser 1 initially
decrypts the CIDs (using the same encryption key as the server, which
is also pre-configured). For each CID, it only needs to be decrypted
once. It then reads the last byte of the CID, which indicates the
priority and forwards the packets.
CID Parser 2 parses QUIC packets from the server to the client. It
extracts the first byte of the CID, which represents the priority,
then removes it and forwards the packets.
When forwarding a packet, its priority is translated to an SRv6 SID
or an MPLS label to map it to the appropriate routing path. Thus,
both parsers record the mapping of priority levels to specific
traffic engineering policies. The configuration file is issued by
the control plane and can be updated during runtime.
In the case of SRv6 TE, such a priority level is mapped to SRv6 SIDs
as shown below:
(priority 0） ==> (SRv6 SID 0)
(priority 1） ==> (SRv6 SID 1)
(priority 2） ==> (SRv6 SID 2)
Based on the parsed priority, the parser adds the corresponding SRv6
SID to the packet, so that the packet can be routed on the desired
priority path. As these operations only include matching, removing
bytes, and inserting new headers, they can be efficiently offloaded
to hardware.
5. Examples
5.1. Packet Flow from Server to Client
Figure 3 shows a typical example that how the packets from a server
to a client are routed on different Cloud WAN routing paths according
to different CIDs (priorities).
Zheng, et al. Expires 11 May 2024 [Page 8]
Internet-Draft quic-te November 2023
+--------------------+
| Cloud VM |
| +----------------+ |API: priority description
| | APP ser^er | |(retrans_pkts: high_priority)
| +----------------+ |(other_pkts: default_priority)
| | Multipath QUIC | |
| +----------------+ |
| |
+-------+----+-------+
| |
Path1[C+DCID0] | | Path2[C+DCID1]
(other_pkts) | | (retrans_pkts)
+------------------------------------------------------+
| | | Cloud WAN |
| +---v----v----+ |
| +-+ CID parser2 +-+ |
| | +-------------+ | |
| C+DCID0 | |C+DCID1 |
| | | |
| +--v-+ +--v-+ |
| | R1 | | R2 | |
| +--+-+ +--+-+ |
| Regular link | | Premium link |
| | | |
| | +-------------+ | |
| +-> CID parser1 <-+ |
| +-----+-------+ |
| | |
+------------------------------------------------------+
|
+----v-----+
| ISP edge |
+----------+
C+DCID0 || C|DCID1
+-----^--------^------+
| |
| +-----------------+ |
| | Multipath QUIC | |
| +-----------------+ |
| | APP client | |
| +-----------------+ |
| Mobile de^ice |
+---------------------+
Figure 3: An example of packet flow from server to client
Zheng, et al. Expires 11 May 2024 [Page 9]
Internet-Draft quic-te November 2023
The app first describes its priority requirements through the API to
the QUIC layer. As shown in this figure, it describes that
retransmitted packets should be routed on a high-priority path, while
other packets are routed on a default-priority path.
When these QUIC packets arrive at the edge of Cloud WAN, the CID
parser parses each packet and extracts the priority encoded in the
CIDs as described in Section 4.4. Based on the priority, packets are
sent on different routing paths via added SRv6 SIDs. For example,
low-priority packets will be routed on a default low-cost path, while
high-priority packets will routed on a preimum low-latency path.
Since packets using different CIDs belong to the same QUIC
connection, they will be combined into a complete and ordered stream
by QUIC and then submitted to the upper-layer application.
5.2. Deployment of CID Parser
This section discusses some experiences regarding the deployment of
CID parsers in a Cloud WAN. Two issues should be considered in the
actual deployment:
* Where to deploy?
* Translating QUIC CID to SRv6 segment identifiers or MPLS
labelling?
For the first one, the parser is usually deployed on the edge of
Cloud WAN. For uplink traffic, it is deployed at the intersection of
the ISP network; for downlink traffic, it is deployed at the edge
routers of a datacenter region.
For the second one, this document recommends using SRv6 because it
has better scalability and flexibility.
6. Security Considerations
TBD.
7. IANA Considerations
The following entry in Table Table 1 should be added to the "QUIC
Transport Parameters" registry under the "QUIC Protocol" heading:
Zheng, et al. Expires 11 May 2024 [Page 10]
Internet-Draft quic-te November 2023
+======================+=================+===============+
| Value | Parameter Name. | Specification |
+======================+=================+===============+
| TBD (experiments use | PRIORITY_PATHS | Section 3.1 |
| 0x5052494f52495459) | | |
+----------------------+-----------------+---------------+
Table 1: New Transport Parameter
8. Normative References
[Multipath-QUIC]
Liu, Y., Ma, Y., De Coninck, Q., Bonaventure, O., Huitema,
C., and M. Kühlewind, "Multipath Extension for QUIC", Work
in Progress, Internet-Draft, draft-ietf-quic-multipath-04,
13 March 2023, <https://datatracker.ietf.org/doc/html/
draft-ietf-quic-multipath-04>.
[QUIC-LB] Duke, M., Banks, N., and C. Huitema, "QUIC-LB: Generating
Routable QUIC Connection IDs", Work in Progress, Internet-
Draft, draft-ietf-quic-load-balancers-15, 24 October 2022,
<https://datatracker.ietf.org/doc/html/draft-ietf-quic-
load-balancers-15>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8837] Jones, P., Dhesikan, S., Jennings, C., and D. Druta,
"Differentiated Services Code Point (DSCP) Packet Markings
for WebRTC QoS", RFC 8837, DOI 10.17487/RFC8837, January
2021, <https://www.rfc-editor.org/rfc/rfc8837>.
[RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer,
D., Matsushima, S., and Z. Li, "Segment Routing over IPv6
(SRv6) Network Programming", RFC 8986,
DOI 10.17487/RFC8986, February 2021,
<https://www.rfc-editor.org/rfc/rfc8986>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
<https://www.rfc-editor.org/rfc/rfc9000>.
Zheng, et al. Expires 11 May 2024 [Page 11]
Internet-Draft quic-te November 2023
Authors' Addresses
Zhilong Zheng
Alibaba Inc.
Email: zhiyou.zzl@alibaba-inc.com
Yunfei Ma
Uber Technologies Inc.
Email: yunfei.ma@uber.com
Yanmei Liu
Alibaba Inc.
Email: miaoji.lym@alibaba-inc.com
Mirja Kühlewind
Ericsson
Email: mirja.kuehlewind@ericsson.com
Zheng, et al. Expires 11 May 2024 [Page 12]
