Internet DRAFT - draft-westerlund-masque-transport-issues
draft-westerlund-masque-transport-issues
MASQUE M. Westerlund
Internet-Draft M. Ihlar
Intended status: Informational Z. Sarker
Expires: 13 January 2022 M. Kuehlewind
Ericsson
12 July 2021
Transport Considerations for IP and UDP Proxying in MASQUE
draft-westerlund-masque-transport-issues-02
Abstract
The HTTP Connect method uses back-to-back TCP connections to and from
a proxy. Such a solution takes care of many transport aspects as
well as IP Flow related concerns. With UDP and IP proxying on the
other hand, there are several per-packet and per-flow aspects that
need consideration to preserve the properties of end- to-end IP/UDP
flows. The aim of this document is to highlight and provide
solutions for these issues related to UDP and IP proxying.
Discussion Venues
This note is to be removed before publishing as an RFC.
Discussion of this document takes place on the MASQUE Working Group
mailing list (masque@ietf.org), which is archived at
https://mailarchive.ietf.org/arch/browse/masque/
(https://mailarchive.ietf.org/arch/browse/masque/).
Source for this draft and an issue tracker can be found at
https://github.com/gloinul/draft-westerlund-masque-transport-issues
(https://github.com/gloinul/draft-westerlund-masque-transport-
issues).
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 13 January 2022.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD License text
as described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License.
Table of Contents
1. Introduction
1.1. Definitions
1.2. HTTP Connect
1.3. TURN
1.4. HTTP Connect-UDP
2. Review of IP Header
2.1. Base Header Fields
2.1.1. Version
2.1.2. DSCP
2.1.3. ECN
2.1.4. Identification, Flags, and Fragmentation offset (IPv4
Only)
2.1.5. Flow Label (IPv6 Only)
2.1.6. Total Length / Payload length
2.1.7. Protocol / Next Header
2.1.8. Time to Live / Hop Limit
2.1.9. Header Checksum (IPv4 Only)
2.1.10. Source and Destination Address
2.2. IPv4 Options Header
2.3. IPv6 Extension Headers
3. Review of UDP Header
3.1. Source and Destination Port
3.2. UDP Length
3.3. UDP Checksum
4. ICMP
5. Maximum Transmission Unit (MTU)
5.1. IPv6 Fragmentation
5.2. IPv4 Fragmentation
6. Summary
6.1. UDP Flow Information and configuration
6.2. Potential Per Packet Information
6.3. Event based Interactions
7. Conclusion
8. References
8.1. Normative References
8.2. Informative References
Authors' Addresses
1. Introduction
This document examines several aspects related to UDP [RFC0768] over
IP [RFC0791] [RFC8200] (IP/UDP) flows when they are proxied according
to the MASQUE proposal over QUIC and using HTTP CONNECT method for
flow establishment (Connect-UDP) [I-D.ietf-masque-connect-udp]. It
also looks at how transport protocols on top of UDP use this
information and contrast that with both the HTTP Connect method
[RFC7231] using either TCP [RFC0793] or QUIC [RFC9000] as well as the
methods used by the TURN protocol [RFC8656].
Aspects discussed include ECN [RFC3168], Differentiated Services
Field and its codepoint (DSCP) [RFC2474], Fragmentation and MTU, ICMP
[RFC0792], IPv6 FLOW ID [RFC8200], IPv6 Extension headers, and IPv4
Options [RFC0791]. This document also discusses the use of the UDP
checksum and the UDP Length field usage related to UDP Options
[I-D.ietf-tsvwg-udp-options].
1.1. Definitions
* UDP Flow: A sequence of UDP packets sharing a 5-tuple.
* ECN: Explicit Congestion Notification [RFC3168].
* DSCP: Differentiated Service Code Point [RFC2474].
* Proxy: This document uses proxy as synonym for the MASQUE Server
or an HTTP proxy, depending on context.
* Client: The endpoint initiating a MASQUE tunnel and UDP/IP
relaying with the proxy.
* Target: A remote endpoint the client wishes to establish bi-
directional communication with.
* UDP proxying: A proxy forwarding data to a target over an UDP
"connection". Data is decapsulate at the proxy and amended by a
UDP and IP header before forwarding to the target. Datagram
boundaries need to be preserved or signalled between the client
and proxy.
* IP proxying: A proxy forwarding data to a target over an IP
"connection". Data is decapsulate at the proxy and amended by a
IP header before forwarding to the target. Packet boundaries need
to be preserved or signalled between the client and proxy.
Address = IP address + UDP port
Target Address --+
\
+--------+ +--------+ \ +--------+
| | Path #1 | | Path #2 V| |
| Client |<--------->| Proxy |<--------->| Target |
| | ^| |^ | |
+--------+ / +--------+ \ +--------+
/ \
/ +-- Proxy's external address
/
+-- Proxy's service address
Figure 1: The nodes and their addresses
Figure 1 provides an overview figure of the involved nodes, i.e.
Client, Proxy, and Target, that are discussed below. We use the name
target for the node or endpoint the client intends to communicate
with via the proxy. There are also two network paths. Path #1 is
the client to proxy path, where the MASQUE protocol will be used over
an HTTP/3 session, usually over QUIC, to tunnel IP/UDP flow(s). Path
#2 is the path between the Proxy and the Target.
The client will use the proxy's service address to establish a
transport connection on which to communicate with the proxy using the
MASQUE protocol. The MASQUE protocol will be used to establish the
relaying of a IP/UDP flow from the client using as the source address
the proxy's external address and sending to the target address. In
addition, after establishment, the reverse is also configured on the
proxy; IP/UDP packets sent from the target address to the proxy's
external address will be relayed to the client.
1.2. HTTP Connect
The HTTP Connect method [RFC7231] is defined such that the HTTP proxy
that receives the request will set up a TCP connection towards a
provided (or resolved) target address. After the TCP connection has
been established, the proxy will connect the byte stream from the
client to the byte stream of the new TCP connection. On byte stream
level this is only tunneling. On the transport level on the other
hand, two distinct transport connections are established. If the
client to proxy connection is HTTP/3, i.e. over QUIC, the basic HTTP
Connect method will still lead to TCP connection establishment from
proxy to the target servers. In this case the client to proxy QUIC
stream's byte stream is connected to the proxy to server TCP
connection. For simplicity and clarify the rest of the document will
use back to back TCP sessions as a comparison.
Due to the byte stream semantics and the use of transport protocol
proxying, most of the transport implications of the header fields and
their values are handled on a per path basis, e.g. response to ECN
Congestion Experienced (CE) marks. Some information that may be end-
to-end related, such as DSCP values, can be copied or translated
between the connections if supported by the proxy. MTU is mostly
irrelevant and handled fully due to the byte stream nature of the
data flowing in the connection. If the MTU differs between the two
paths the number of packets required to send a particular data object
may differ as well. However, the impact of that is small and depends
on the amount of buffering between the two connections. There is no
requirement to have a one-to-one correspondence of packets between
the two TCP connections.
1.3. TURN
"Traversal Using Relays around NAT (TURN): Relay Extensions to
Session Traversal Utilities for NAT (STUN)" [RFC8656] is a solution
for relaying UDP datagrams. However, it is a hybrid between a purely
encapsulating tunnel and proxying. A somewhat simplified description
of the protocol follows below.
A client makes a TURN request over a TCP/TLS connection to a TURN
server to authenticate itself and acquire a long-term secret. Note
that subsequent requests are secured using keying material from the
long term secret exchange. Next, the client can send a request over
UDP to be allocated a UDP port number on one of the server's external
IP addresses. After that, the client knows the IP address and UDP
port number that will be used for the TURN server's side of any UDP
flow sent to or received from the remote target.
A TURN client can send UDP packets in two ways. The first solution
is to include a send indication for each UDP packet to be relayed.
The send indications contains the target destination address and
port. The other solution is to create a binding, where a channel ID
between the client and TURN server is bound to a specific 5-tuple
from the TURN Server to the target destination. The established
channel is bi-directional. When a channel has been established, UDP
packets can be relayed with a low overhead of 4 bytes.
To receive UDP packets on the allocated port, the client must specify
what source address and port (target address) is to be allowed, this
is called establishing a permission. Binding a channel creates a
permission at the same time. When the TURN server receives a UDP
packet from an external source where permission exists but no
channel, it will relay the data using a DATA indication, which
includes the source address.
A relevant aspect for the rest of the discussion in this document is
that TURN has a one to one mapping between IP/UDP/TURN messages
between client and TURN server and the IP/UDP packet received or sent
on the external side. Also, though TURN messages and indications
include header information in the UDP payload, there are distinct IP/
UDP headers on each side of the TURN server. Because of this the
TURN protocol is able to preserve the per flow and per packet
functionalities that exist in IP/UDP headers. For instance, ECN and
DSCP markings associated with specific packets are preserved by the
relay by copying or translating them from incoming packets to
outgoing packets.
1.4. HTTP Connect-UDP
The MASQUE WG is chartered to work on the tunneling of UDP datagrams
in a QUIC connection between client and a MASQUE server (We will
refer to the MASQUE server as a proxy in the rest of this document to
align with the basic HTTP Connect terminology). The proxy forwards
tunneled UDP payload to the correct target address. An HTTP Connect-
UDP request is used to establish the tunnel and provide the required
addressing information.
In the review performed in this document we make the assumption that
it is desirable to minimize the per-packet overhead. Specifically,
we assume that IP/UDP header information from packets exchanged
between the proxy and target will not be sent within the tunnel. The
initial relay exchange establishment needs to be performed for each
target the client wants to communicate with, i.e. one relay exchange
per IP/UDP flow on the proxy to target path. Keeping this state in
the proxy on a per UDP flow basis appears trivial. Therefore, the
review will determine what IP/UDP state is required per flow and what
is per per packet information.
In this review we also aim to identify the impact on the end-to-end
flows by using QUIC as a tunneling protocol, both in stream and/or
datagram mode. Properties of IP/UDP flows and higher level transport
protocols such as QUIC or RTP will be considered.
Two high level observations need to be made when comparing Connect-
UDP to TURN. The first is that QUIC is a proper transport protocol
with congestion control. This means that there might not be a one-
to-one mapping between events that impact the transport, such as
congestion indications, on either path relative to the proxy. The
second observation is that tunneled UDP datagrams can be coalesced in
a single UDP datagram with either multiple QUIC packets, or multiple
QUIC datagram frames in a single QUIC packet. If reliable streams
are used instead of datagram frames, then a UDP payload may even be
fragmented over multiple UDP packets containing QUIC packets.
This all motivates a very careful consideration of the IP and UDP
header information and how that needs to be handled on flow level or
per packet level to avoid breaking end-to-end transport properties
for the flow.
2. Review of IP Header
2.1. Base Header Fields
This section reviews the header fields in IPv4 [RFC0791] and IPv6
[RFC8200]. It will note which field are version specific. Size
differences are not considered in the review as it is focused on
functionality and impact.
2.1.1. Version
The proxy needs to know which IP version to use on the proxy to
target path. If an explicit IP address is included in the Connect-
UDP request, the proxy would know this directly from the IP address
format. In the case where a domain name is used to obtain the target
IP address, the IP version needs to be specified or be based on
preferences when resolving the domain name.
It should be noted that different IP versions may be used on the two
paths requiring the proxy to do translation. This can also lead to
scenarios whereby version specific information carried on one path
does not translate to the version used on the other path.
2.1.2. DSCP
Diffserv code points are primarily used to indicate forwarding
behavior. Codepoints on IP/UDP flows on the proxy to target path are
either set on a flow level or packet level. Codepoints set on a flow
level are set at flow instantiation and can be updated during ongoing
relaying.
A DSCP value received on IP/UDP packets in the target to proxy
direction may be propagated to the Proxy to client path. However,
that is problematic when the datagram is tunneled in QUIC. The DART
considerations [RFC7657] for connection oriented transport applies
here. If multiple DSCPs are used for a single connection, then there
would be a need for having separate congestion control states for the
different forwarding behaviors, which would likely require QUIC
protocol extensions. The same issue exists for packets sent by the
client on the client to proxy path.
Different forwarding behaviors in both directions of the path
connecting the client and the proxy could be enabled without a QUIC
extension by establishing individual QUIC connections per forwarding
behavior used. However, this requires that the proxy is able to bind
multiple QUIC connections received from the client into a single IP/
UDP flow on the proxy to target path. This could have significant
security model implications as authorization would be needed to add
subsequent bindings to an existing flow.
Let's consider the capability for the proxy to send packets with
packet-level DSCP marks towards the target. That would require at a
minimum a per packet indication mechanism and would enable different
forwarding behaviors on the proxy to target path. Similarly, a per
packet mechanism would be needed for the proxy to be able to relay
DSCP values received from the target towards the client. The
usefulness of the latter would be to ensure that the transport
protocol on top of MASQUE is able to determine whether there is a
need for multiple congestion control states for different sub-sets of
packets within the received IP/UDP flow. WebRTC IP/UDP flows could
have this property.
The most basic DSCP relay capability would be to set the same DSCP
value on all IP/UDP packets sent by the proxy to the same target for
a specific IP/UDP flow. This capability would only require
signalling of the desired value at flow establishment. A mechanism
to update the DSCP value for an ongoing flow should also be
considered.
Another issue with DSCP mapping to forwarding behaviors is that the
mappings are defined per network location, typically within one
administrative domain of routing. Therefore they may be remapped on
the different paths relative to the proxy. When the client and proxy
reside in two different administrative domains there will be an
additional challenge for the client to use the right DSCP value.
Thus, support for DSCP in the MASQUE protocol should either be
limited to consider per hop behavior or the use of a mapping table
such that the proxy can translate an incoming DSCP value to a locally
used value.
2.1.3. ECN
The Explicit Congestion Notification (ECN) [RFC3168] field carries
per packet path signals about congestion. The discussion of ECN
capability can be split into two parts, one for each path relative to
the proxy. On the client to proxy path the QUIC connection used for
tunneling the UDP datagrams can enable and use ECN on that path
specifically. Any congestion experienced (ECN-CE) marking on that
path impacts the congestion window of the client to proxy QUIC
connection, thus indirectly affecting the end-to-end flow.
The capability to use ECN on the proxy to target path requires proxy
protocol support. This will enable the end-to-end usage of ECN in
the upper layer transport protocol. To support ECN end-to-end when
using MASQUE proxy two functionalities need to exist.
First, the capability of setting the ECN field value (Not-ECT,
ECT(1), ECT(0)) on any IP/UDP packets sent from proxy to target.
This value can be set initially but may be changed during ongoing IP/
UDP flow proxying, as the end-to-end transport may subsequently
determine that the path is not ECN capable.
Secondly, the client must be able to receive per packet indications
of the ECN field value for every packet received by the proxy from
the target. This ensures that the upper layer transport protocol
receives ECN information per relayed UDP datagram. The information
will be used to react to ECN-CE (Congestion Experienced) marks and
for validation of the ECN path capability.
A solution like TURN's [RFC8656] translation of ECN markings between
the two paths is not possible for multiple reasons. First, ECN marks
on the client to proxy path will be consumed and reacted to by the
QUIC connection used for tunneling. Second, the previously discussed
lack of a one-to-one relationship of IP/UDP packets prevents accurate
tracking of the ECN markings and will make the end-to-end validation
fail. Therefore additional explicit signaling between the proxy and
the client would be needed.
2.1.4. Identification, Flags, and Fragmentation offset (IPv4 Only)
These fields are used for the IPv4 fragmentation solution. The
authors are of the opinion that IP level fragmentation should be
avoided. However, since there are no guarantees for a one-to-one
packet relation between the two paths relative to the proxy, any IPv4
fragments will need re-assembly upon reception by the endpoints and
the proxy.
To support Path MTU Discovery the Don't Fragment (DF) bit needs to be
set for all outgoing IP/UDP packets from the proxy to the target.
Per flow or per packet setting of this bit needs to be supported.
2.1.5. Flow Label (IPv6 Only)
The IPv6 flow label is used by the network to identify flows, for
example to prevent a single flow to be spread over multiple paths
when load balancing based on Equal Cost Multipath (ECMP) routing
[RFC6438] is performed. The flow label should be set by the endpoint
originating the IP/UDP flow, as it knows when a flow qualifies for a
unique IPv6 flow label. Thus, it is expected that one IPv6 flow
label will be used for the IP/UDP flow that carries the client to
proxy QUIC connection, and one for each IP/UDP flow established by
the MASQUE protocol to different target addresses.
Based on the above reasoning it does not seem like there is a need
for the MASQUE protocol to explicitly signal or indicate flow labels.
2.1.6. Total Length / Payload length
The Total Length (IPv4) / Payload length (IPv6) fields contain the
size of the IP packet, either directly for IPv4, or indirectly in
IPv6 (by providing the length after the fixed 40-byte header, i.e.
for extension headers and data).
These field are necessary for the processing on reception, however it
does not need to be communicated on a per packet-basis to the client,
or be provided by the client, with a single potential exception that
is discussed in the context of the UDP length field (Section 3.2).
2.1.7. Protocol / Next Header
The Protocol (IPv4) and Next Header (IPv6) fields provide the
identification of the upper layer protocol, in this case UDP. For
IPv6 one or more extension headers may first be identified in a chain
before arriving at UDP.
The use of UDP relaying will need to be signalled explicitly to
separate it from other types of relaying, such as the IP tunneling/
relaying discussed in the MASQUE charter.
2.1.8. Time to Live / Hop Limit
The purpose of the Time to Live (IPv4) and Hop Limit (IPv6) fields is
to prevent packets from having an infinite life time in case of
routing loops. The acronym TTL is used from here on to describe any
of these fields. TTL limits the number of routing hops a packet
survives and should result in an ICMP message back to the sender when
it expires. Therefore, it is possible to use TTL for investigating
network paths.
It is not clear if such a mechanism needs to be supported in a MASQUE
protocol context. If something like trace-route is to be supported,
per packet setting of the TTL field would be needed.
The need for echoing the TTL field value on reception of a IP/UDP
packet from the target to the client appears also very limited. The
value set on transmission of a packet is usually an operating system
set default value.
The authors believe that the proxy's default values are sufficient
for the MASQUE protocol functionality.
2.1.9. Header Checksum (IPv4 Only)
The IPv4 checksum field verifies the integrity against non-
intentional errors in transmission or processing of the IP header.
IPv6 lacks this checksum and instead relies on the transport protocol
checksum.
The value is generated when an IP packet is transmitted from the
proxy and verified on reception. No further functionality required.
2.1.10. Source and Destination Address
On the path from the proxy to the target, the source address will be
the proxy's external address applied when relaying the IP/UDP
packets. This address will be determined as part of the IP/UDP flow
tunneling establishment and should be signalled back to the client by
the proxy. The destination address used will be the target's IP
address.
The source addresses used on the client to proxy path are only needed
for the communication between the client and proxy and are part of
the QUIC connection's state. Thus, the possibility to change it will
depend on the mechanisms in QUIC for dealing with client address
migration or multi-path. Further discussion should not be needed.
In case of IP proxying, as the proxy cannot utilize port numbers, the
proxy might need to maintain multiple external IP addresses in order
to identify different forwarding processes for packet received from
multiple target servers. If the proxy is guaranteed to be on-path
between the client and server the proxy could also conserve the
client's source IP address as it's external address.
The source and destination addresses are therefore part of the
fundamental state for IP/UDP flow relaying and need to be established
at initiation. The 2 (IP proxying) or 5-tuple (IP/UDP proxying) from
the proxy to the target and the reverse tuple needs to be explicitly
signalled. The client either needs to explicitly provide the target
IP address or a domain name that the proxy can resolve to a target IP
address. The proxy needs to notify the client about which source IP
address it uses when sending on the proxy to target path.
2.2. IPv4 Options Header
The use of IPv4 Options header on the general Internet is very
limited. It is therefore likely that no functionality is required.
2.3. IPv6 Extension Headers
One IPv6 Extension header that needs discussion here is the
fragmentation header. Although it is the IP originating node that
adds the fragmentation header, the MASQUE protocol will likely need
to control whether IPv6 fragmentation should be used or not, in the
same way as for the IPv4 DF bit.
Some existing IPv6 extension headers could be added by the
originating node. Whether they require any explicit signalling or
relaying of data to the client needs to be investigated further.
Especially Hop-by-Hop options, such as the IPv6 Minimum Path MTU Hop-
by-Hop Option [I-D.ietf-6man-mtu-option].
There are also some individual proposals for extension header that
might matter in the future: Network Tokens
[I-D.yiakoumis-network-tokens], IPv6 Truncate Option
[I-D.leddy-6man-truncate]. Thus, consideration needs to be made if
there are necessary to future proof the Masque protocol, at least to
enable future extensions to support per packet Extension headers.
3. Review of UDP Header
3.1. Source and Destination Port
For UDP proxying, the UDP destination port is used by endpoints to
locate the destination process that should consume a specific UDP
datagram. Source Ports can be used by the receiving application to
separate flows based on the 5-tuple.
As discussed in Section 2.1.10 the UDP source and destination ports
are part of the 5-tuple and needs to be communicated on IP/UDP flow
establishment.
3.2. UDP Length
The UDP length field specifies the UDP payload length in octets.
The UDP length field normally indicates that the UDP payload fills up
the remainder of the IP packet. However, this is not always the
case. Specifically, UDP options [I-D.ietf-tsvwg-udp-options] are
designed to make use of the surplus area between the end of the UDP
data section and the end of the IP packet.
Thus, for the MASQUE protocol to preserve the capability to carry UDP
options in UDP relaying this surplus area and the UDP payload data
length field need to transmitted from client to proxy in both
directions.
3.3. UDP Checksum
The UDP checksum verifies the UDP datagram headers and payload and
the pseudo header with IP layer information. The UDP checksum should
always be verified by the receiving party. The UDP checksum may also
be set to zero to provide no verification. This is primarily used by
tunnel encapsulation formats. If it is desired to send IP/UDP flows
from the proxy to the target address with a zero UDP checksum, the
MASQUE protocol needs to support an indication of this desire.
The use of zero UDP checksum for IPv6 is more restricted than for
IPv4 due to the lack of a IP header checksum [RFC6936].
The authors do not believe it is necessary to be able to send IP/UDP
datagrams with a zero UDP checksum. It is also not necessary to
relay the UDP checksum generated by the target, since the QUIC
protocol will provide stronger cryptographic integrity verification
of the UDP datagram payload. Also, for the target generated checksum
to be meaningful to the client the complete datagram and pseudo
header would need to be reconstructed, which in would likely require
extra processing and copying of data. Though some cases of erroneous
handling of IP/UDP header fields by the MASQUE protocol could be
detected in this way, it is not deemed to be worth the effort.
For the packets the client sends to be relayed to the target
destination, having the client create a UDP checksum would provide
some protection against processing or implementation errors, although
the overhead and extra processing is likely not worth the effort.
In addition the calculation and verification of the transport header
checksum is one of these aspects that can be offloaded to hardware in
a proxy.
4. ICMP
Internet Control Message Protocol (ICMP) [RFC0792][RFC4443] messages
are useful hints on why sent IP packets appear to disappear in the
network. Especially for what might appear to be intermittent issues,
such as exceeding the MTU of the path.
Lets start with clarifying which ICMP messages may be relevant to
handle in the MASQUE protocol. The primary concern is to ensure that
the client receives any ICMP responses that are sent back to the
proxy as result of packets the client had relayed through the proxy
towards the target destination. The proxy should validate and
identify ICMP messages that relate to a particular IP/UDP flow that
the proxy sends. The relevant ICMP information, i.e. Type and Code
as well as any included bytes from the packets that can be used to
identify the actual IP/UDP packet in the client, should be sent to
the client. Such a functionality would enable the the client to
receive Packet Too Big messages that speeds up Path MTU Discovery
(Section 5). It also enables the client to learn when there appears
to be no one at the target address, i.e. the port, host or network
unreachable codes.
IP/UDP packets reaching the proxy from a target address may result in
that ICMP messages are sent back to the target. For example a port
unreachable message would be sent if a packet arrives after the
client has terminated the tunneling session.
Any ICMP packets that result from IP/UDP packets exchanged between
the client and the proxy, related to the QUIC connection is to be
validated and consumed by the QUIC implementation.
Thus, depending on the ICMP message, the MASQUE protocol needs to
consider a mechanism for the proxy to indicate to the client that it
has received and validated ICMP messages. If the ICMP message
indicates a connection failure, HTTP response error codes can be
used. However, for HTTP Connect-UDP the response code was sent when
the UDP socket was created, while an ICMP message would only be
received after UDP packets have been relayed.
5. Maximum Transmission Unit (MTU)
The MTU available for the UDP payload depends on whether the QUIC
connection uses datagrams or streams to carry the UDP payload on the
client to proxy path. When streams are used, the outer QUIC
connection can fragment and re-assembly UDP Payloads of any size. In
this case any MTU issues will arise on the proxy to target path.
When using datagrams, unless the QUIC datagram extension provides a
fragmentation solution, then the outer QUIC connection will provide a
MTU that is dependent on the largest datagram payload that can be
transmitted. A potential issue is that this might be variable over
time, both due to underlying path changes, and to variable elements
of the QUIC protocol. However, a base overhead from the QUIC headers
should be possible to calculate based on the maximum QUIC packet
size. Depending on the amount of per packet information needed to be
provided additional headroom for the MASQUE encapsulation may be
required.
To enable PMTUD discovery certain aspects are needed or greatly
simplify the process.
* Ensure that the DF bit is set to Don't Fragment on outgoing IPv4/
UDP packets from the proxy to the target. For IPv6 the use of the
fragmentation header needs to be prevented.
* Have the proxy signal back to the client its current interface MTU
limit for packets that will be sent from proxy to target.
* Have the tunneling QUIC connection expose the current MTU for
datagrams to the MASQUE implementation. This is likely dynamic
and can be updated at any point.
* Returning ICMP Packet Too Big (PTB) message from the proxy to the
client when packets are dropped due to MTU on the proxy to target
path.
It should be noted that unless the QUIC datagram extension provides a
fragmentation mechanism this will in many scenarios be the most
likely MTU bottleneck and there is no work around for it, the IP/UDP
tunneled traffic will need to fit or be dropped.
5.1. IPv6 Fragmentation
Reassembly of received traffic will occur on each node, Client,
Proxy, and Target. The need for IP level fragmentation should be
avoided, by having the working MTU be propagated up. Initial MTU
signaling should exist for the flow. However, this is potentially
dynamic and a PMTUD process running for the outer QUIC tunnel on the
client to proxy path will update the supported MTU.
An option for controlling if fragmentation should occur or not by the
Proxy should be considered.
5.2. IPv4 Fragmentation
As IPv4 fragmentation is flexible and allows an on-path node to
fragment a packet, enabling fragmentation may reduce the MTU issues.
However, Path MTU Discovery is recommended instead, for the following
reasons:
* Fragmentation increases the packet loss rate.
* IP fragments do not traverse NATs and Firewalls well. Which is
especially relevant for MASQUE as significant deployments will be
clients in access or residential networks that have NATs or
Firewalls on the path from the client to the proxy.
6. Summary
Lets sum up the aspects of the IP/UDP header fields and related
protocols in a couple of categorizes.
6.1. UDP Flow Information and configuration
This section contains header fields whose value either will be static
for a given IP/UDP flow, apply until changes, or can be used as
default values when per packet values are not given.
First of all, the IP source and destination address as well as UDP
source and port information is directly related to the establishment
of a bi-directional IP/UDP flow between proxy and the target. The
desired IP version also needs to be indicated if the address is
expressed as hostname, but otherwise would be given by the format of
the address. The target always needs to be provided by the client.
Since there are cases where the client would need to know the
external address used by the proxy towards the target, there needs to
be a way for the client to request this information.
The upper layer protocols between the client and the target might
have different capabilities of using ECN. Therefore it is necessary
to be able to signal whether the ECN fields in proxy to target
packets should be set to Not-ECT, ECT(0) or ECT(1).
If a DSCP marking other than 0, i.e. best effort, is desired then a
default value could be set as part of the relay establishment. This
value could potentially be overridden on a per packet basis or be
changed at a future point in the relayed flows lifetime.
A don't fragment setting could likely be defaulted to be always true.
However, we invite further discussion if there are cases where it
would be better to enable IP level fragmentation for some packets, or
for all.
The Hop Limit could likely be using node default values, unless
someone raises a use case where they actually want to modulate the
value on per packet basis or set an explicit value for the IP/UDP
flow.
The MTU known by the proxy towards the indicated target should be
signalled back at relay establishment by the proxy.
6.2. Potential Per Packet Information
This section contains information that would be necessary to
associate with a specific packet when the client request sending or
the proxy relays a received packet.
For each packet the proxy receives the ECN field's value need to be
relayed to the client so that the upper layer can respond to either a
CE marking indicating congestion, or any remarking.
There are examples where IP/UDP flows contain packets that do not
have uniform DSCP marking. To enable sending of such streams, any
DSCP value other the default value would need to be indicated by the
client to the proxy.
If it is desired to verify the UDP Checksum in the client to avoid
any potential errors the MASQUE protocol implementation may cause the
received UDP checksum value would need to be relayed to the client.
The UDP checksum value could also be calculated by the client and
included.
To support UDP Options, the UDP option and their values needs to be
signalled.
If support for including IPv6 Extension Headers that needs client
side information then the extension header information would need to
be indicated by the client and also tunnelled.
6.3. Event based Interactions
This section summarizes information that are triggered by events and
not directly connected to a specific packet in the IP/UDP flow being
relayed. Instead these are more of the nature of asynchronous events
caused by the network, the upper layer transport protocol, or
application.
The ECN setting for packets sent by the proxy to the target may need
to change. If the upper layer transport protocol determines that the
end-to-end path is not ECN capable it will need to change an ECT
marking to Not-ECT. Some protocols may defer probing for ECT
capability until after some initial handshake and packet exchange has
occurred.
The upper layer application may change its desired forwarding
behavior for the packets in the IP/UDP stream, thus the need for the
client to change the default applied DSCP value on packets sent by
the proxy to target.
The reception of ICMP messages by the proxy will likely be the result
of a packet sent to the target but the cause is likely in the
network. When this occurs the client needs to be informed of this
ICMP message, especially when this event leads to a connection
failure.
The proxy may detect an MTU change on the proxy to target path due to
received ICMP messages that are consumed in the IP stack of the proxy
and affecting the MTU for outgoing packets. Thus, the proxy may need
to indicate to the client that it no longer can send non-fragmented
packets larger than the new MTU.
7. Conclusion
This document shows that many of the fields in the IP/UDP header can
be signaled initially at flow establishment. Especially IP addresses
and potentially ports need to be communicated as those also need to
be used for flow identification. It also shows that certain field
values or related information needs to be relayed or signalled based
on asynchronous application or network events. Other field
information could need per packet relaying. The latter requires a
more active bidirectional communication channel between the proxy and
the client.
These aspects need to be taken into account in the future protocol
development of the CONNECT-UDP method to ensure that the MASQUE
protocol doesn't prevent future IP and transport protocol evolution.
8. References
8.1. Normative References
[I-D.ietf-masque-connect-udp]
Schinazi, D., "The CONNECT-UDP HTTP Method", Work in
Progress, Internet-Draft, draft-ietf-masque-connect-udp-
03, 5 January 2021,
<https://datatracker.ietf.org/doc/html/draft-ietf-masque-
connect-udp-03.txt>.
[I-D.ietf-tsvwg-udp-options]
Touch, J., "Transport Options for UDP", Work in Progress,
Internet-Draft, draft-ietf-tsvwg-udp-options-13, 19 June
2021, <https://datatracker.ietf.org/doc/html/draft-ietf-
tsvwg-udp-options-13.txt>.
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
DOI 10.17487/RFC0768, August 1980,
<https://datatracker.ietf.org/doc/html/rfc768>.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
DOI 10.17487/RFC0791, September 1981,
<https://datatracker.ietf.org/doc/html/rfc791>.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://datatracker.ietf.org/doc/html/rfc792>.
[RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC 2474,
DOI 10.17487/RFC2474, December 1998,
<https://datatracker.ietf.org/doc/html/rfc2474>.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP",
RFC 3168, DOI 10.17487/RFC3168, September 2001,
<https://datatracker.ietf.org/doc/html/rfc3168>.
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", STD 86, RFC 8200,
DOI 10.17487/RFC8200, July 2017,
<https://datatracker.ietf.org/doc/html/rfc8200>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
<https://datatracker.ietf.org/doc/html/rfc9000>.
8.2. Informative References
[I-D.ietf-6man-mtu-option]
Hinden, R. M. and G. Fairhurst, "IPv6 Minimum Path MTU
Hop-by-Hop Option", Work in Progress, Internet-Draft,
draft-ietf-6man-mtu-option-05, 28 April 2021,
<https://datatracker.ietf.org/doc/html/draft-ietf-6man-
mtu-option-05.txt>.
[I-D.leddy-6man-truncate]
Leddy, J., Bonica, R., and I. Lubashev, "IPv6 Packet
Truncation", Work in Progress, Internet-Draft, draft-
leddy-6man-truncate-05, 10 October 2018,
<https://datatracker.ietf.org/doc/html/draft-leddy-6man-
truncate-05.txt>.
[I-D.yiakoumis-network-tokens]
Yiakoumis, Y., McKeown, N., and F. Sorensen, "Network
Tokens", Work in Progress, Internet-Draft, draft-
yiakoumis-network-tokens-02, 22 December 2020,
<https://datatracker.ietf.org/doc/html/draft-yiakoumis-
network-tokens-02.txt>.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, DOI 10.17487/RFC0793, September 1981,
<https://datatracker.ietf.org/doc/html/rfc793>.
[RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet
Control Message Protocol (ICMPv6) for the Internet
Protocol Version 6 (IPv6) Specification", STD 89,
RFC 4443, DOI 10.17487/RFC4443, March 2006,
<https://datatracker.ietf.org/doc/html/rfc4443>.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
for Equal Cost Multipath Routing and Link Aggregation in
Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
<https://datatracker.ietf.org/doc/html/rfc6438>.
[RFC6936] Fairhurst, G. and M. Westerlund, "Applicability Statement
for the Use of IPv6 UDP Datagrams with Zero Checksums",
RFC 6936, DOI 10.17487/RFC6936, April 2013,
<https://datatracker.ietf.org/doc/html/rfc6936>.
[RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Semantics and Content", RFC 7231,
DOI 10.17487/RFC7231, June 2014,
<https://datatracker.ietf.org/doc/html/rfc7231>.
[RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services
(Diffserv) and Real-Time Communication", RFC 7657,
DOI 10.17487/RFC7657, November 2015,
<https://datatracker.ietf.org/doc/html/rfc7657>.
[RFC8656] Reddy, T., Ed., Johnston, A., Ed., Matthews, P., and J.
Rosenberg, "Traversal Using Relays around NAT (TURN):
Relay Extensions to Session Traversal Utilities for NAT
(STUN)", RFC 8656, DOI 10.17487/RFC8656, February 2020,
<https://datatracker.ietf.org/doc/html/rfc8656>.
Authors' Addresses
Magnus Westerlund
Ericsson
Email: magnus.westerlund@ericsson.com
Marcus Ihlar
Ericsson
Email: marcus.ihlar@ericsson.com
Zaheduzzaman Sarker
Ericsson
Email: zaheduzzaman.sarker@ericsson.com
Mirja Kuehlewind
Ericsson
Email: mirja.kuehlewind@ericsson.com
