| Network Working Group | A. Rousskov |
| Internet-Draft | The Measurement Factory |
| Updates: 3507 (if approved) | August 4, 2016 |
| Intended status: Informational | |
| Expires: February 5, 2017 |
ICAP Trailers
draft-rousskov-icap-trailers-00
This document defines an ICAP trailer feature which allows ICAP agents to reliably send message metadata after the message body. The ICAP trailer is independent from the HTTP trailer that might also be encapsulated in an ICAP message. ICAP changes defined here are backward compatible and address a long-standing ICAP specification errata entry.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 5, 2017.
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
ICAP [RFC3507] specification says that the Trailer header field is "defined in ICAP the same as [...] in HTTP". Unfortunately, that phrase alone is not enough for trailer-related interoperability in the ICAP context because of the following conflicting interpretations, requirements, and needs:
These problems led to a ban on ICAP trailers [Errata].
Several designs were considered for introducing proper ICAP trailers support:
Trailers allow an ICAP agent to transmit metadata after the message body. Such delayed transmission is useful when the same information was not available at the start of the message transmission. For example:
This section informally describes the overall feature operation. This description is deliberately imprecise and cannot be used to build compliant implementations. The following sections contain actual protocol requirements.
To announce feature support, ICAP agents exchange "Allow: trailers" settings during an OPTIONS transaction. To send a trailer at the end of a particular REQMOD or RESPMOD transaction, the agent first sends both "Allow: trailers" and "Trailer" header fields. The Trailer field lists header field names expected in the message trailer section. After sending the entire ICAP message body, the agent sends the trailer section (a.k.a. "trailer"). The trailer section is syntactically equivalent to the ICAP message header section. The trailer section does not have to contain any of the promised fields and might even have no fields at all.
A trailer makes sense only in an ICAP message with a body. However, a trailer could be sent if the ICAP message body encapsulates just HTTP headers.
If a trailer is sent, its bytes are always the last bytes sent during the entire ICAP transaction. Thus, a client never sends a trailer at the end of Preview unless it sent the "ieof" chunk extension as well. Similarly, a server never a sends trailer with a 100 (Continue) control message.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Conformance criteria and error handling considerations are defined in Section 2.5 of [RFC7230].
This specification uses the Augmented Backus-Naur Form (ABNF) notation of [RFC5234] with a list extension defined in Section 7 of [RFC7230]. All syntax rules not explicitly defined in this specification (e.g., header-field and CRLF) are defined in (or included by reference from) [RFC7230].
The "Allow/X" notation is defined in Section 5.
Allow = 1#token
The use of the Allow header field defined in sections 4.6 and 4.10.2 of [RFC3507] is extended while preserving its original syntax and general semantics (i.e., a comma-separated list of tokens, each identifying an ICAP feature supported by the sending agent): [RFC3507] defines Allow usage in OPTIONS requests and responses. This specification extends Allow usage to other messages and adds "trailers" to the list of possible Allow value tokens.
An agent MUST treat multiple Allow header fields as one Allow field with a comma-separated list of individual field value tokens, concatenated in the order of their appearance in the ICAP header. An ICAP agent MUST ignore Allow header tokens it does not understand. This document does not specify the significance of Allow tokens order and impact of repeated tokens.
An agent MAY send an Allow header in any message. Such header contains a list of ICAP features supported by the sending agent. For example, a client could send an "Allow: trailers" in a REQMOD request and receive an "Allow: 204, trailers, 206" header field in response. The exact meaning of each Allow token in a context of its message is defined by the corresponding feature specification. By sending Allow, the agent indicates compliance with each listed feature's specification but does not necessarily commit to offer or use any of the listed features during future transactions.
To keep this specification succinct, we introduce "Allow/X" notation to mean an ICAP Allow header field which value contains an "X" token, possibly among other tokens. For example, "Allow/trailers" stands for an Allow header field with a "trailers" token and possibly other tokens. This notation covers multi-field Allow headers as well because they are equivalent to a combined single-field Allow header.
The same approach to extending Allow header usage was successfully applied to the ICAP 206 extension (XXX: reference our expired ICAP icap-ext-partial-content draft). This specification is compatible with the ICAP 206 extension.
An ICAP message with a trailer is a concatenation of a regular ICAP message and an trailer section. The trailer section syntax is identical to the ICAP header syntax:
ICAP-message-without-trailer = <see [RFC3507]>
ICAP-message-with-trailer = ICAP-message-without-trailer
trailer-section
trailer-section = *( trailer-field CRLF )
CRLF
trailer-field = header-field
Note that any trailer, even a trailer without fields, ends with CRLF. That terminating sequence is essential for proper message framing on persistent ICAP connections.
A sender MUST NOT generate a trailer section that contains a field necessary for message framing (e.g., Encapsulated, Preview, and Trailer), routing (e.g., Host), or authentication.
The ICAP Trailer header value syntax is identical to the HTTP Trailer syntax [RFC7230]:
Trailer = 1#field-name
A Trailer header field sender SHOULD enumerate the names of all expected trailer fields. This a priori knowledge of trailer fields might help the recipient with trailer processing (e.g., certain message annotation actions could be delayed in anticipation of the trailer section). However, enumerating all expected trailer fields can be impractical or even impossible in some environments. A Trailer sender MAY send a trailer section with a set of field names that differs from the set of field names listed in the Trailer header field.
This specification does not place any restriction on the order of field names in the Trailer header field. Senders SHOULD NOT generate duplicate names for the Trailer header field.
A client compliant with this specification SHOULD send Allow/trailers in each OPTIONS request. A non-authenticating server cannot be expected to mark an Allow/trailers-sending client specially, but this support announcement requirement is meant to minimize interoperability problems associated with servers sending Allow/trailers in OPTIONS responses. Some servers that do not support trailers might not be able to ignore Allow/trailers in OPTIONS requests; therefore, a client SHOULD offer a configuration option or other means of disabling sending Allow/trailers in OPTIONS requests.
An ICAP service sending Allow/trailers in OPTIONS response is called a trailers-supporting service. That service designation, maintained by the client, starts upon receiving the OPTIONS service response carrying Allow/trailers and lasts until OPTIONS expiration or a new OPTIONS response from that service.
A client compliant with this specification SHOULD send Allow/trailers in each request to a trailers-supporting service. Doing so allows the service to respond with a trailer (and is also necessary for sending a client trailer, as detailed further below).
A client receiving both a Trailer header field and Allow/trailers in the response MUST expect a trailer section in that response. In all other cases, a client MUST use the usual trailer-free ICAP response syntax. A client receiving a Trailer header field without Allow/trailers in a response MAY treat the response as syntactically malformed and MUST NOT reuse the connection for any other messages (including pending pipelined requests, if any).
A client MAY send a trailer in any request that satisfies all of these conditions:
A client MUST NOT send a trailer in any other request.
To send a trailer, the client MUST send Allow/trailers and a Trailer header field in the same request. A client MUST NOT send a combination of those two header fields without sending a trailer.
A server compliant with this specification SHOULD send Allow/trailers in each successful response to an OPTIONS request carrying Allow/trailers. Although ICAP/1.0 [RFC3507] allows a list of features in the Allow header, some ICAP clients might not be able to handle an Allow header other than "Allow: 204"; therefore, a server SHOULD NOT send Allow/trailers in a response to a request without Allow/trailers.
A server compliant with this specification MAY send Allow/trailers in a response without a trailer to a request with Allow/trailers. The client receiving a no-trailer REQMOD or RESPMOD response with Allow/trailers ought to ignore Allow/trailers. Nevertheless, the server is allowed to respond with Allow/trailers in this context because doing so might simplify server implementation and configuration.
A server receiving both a Trailer header field and Allow/trailers in the request MUST expect a trailer in that request. In all other cases, a server MUST use the usual trailer-free ICAP request syntax. A server receiving a Trailer header field without Allow/trailers in a request MAY treat the request as syntactically malformed and MUST NOT reuse the connection for any future requests.
A server MAY send a trailer in any response that satisfies all of these conditions:
A server MUST NOT send a trailer in any other response.
To send a trailer, the server MUST send Allow/trailers and a Trailer header field in the same response. A server MUST NOT send a combination of those two header fields without sending a trailer.
The following examples illustrate trailer exchanges between ICAP agents compliant with this specification. To show important details, all CRLF sequences after major message parts are shown as "\r\n" lines. CRLF sequences at the end of other lines are implied.
All REQMOD and RESPMOD messages in these examples share the context of the following OPTIONS handshake.
Figure 1 shows an OPTIONS request and response when both the client and the server are compliant with this specification. The client also supports the ICAP 206 extension, but the server does not.
OPTIONS icap://example.net/sample-service ICAP/1.0
Host: example.net
User-Agent: Example-ICAP-Client/1.2
Allow: 204, trailers, 206
\r\n
ICAP/1.0 200 OK
Date: Sat, 13 Aug 2016 12:17:21 GMT
Methods: RESPMOD
Service: FOO Tech Server 1.0
ISTag: "W3E4R7U9-L2E4-2"
Encapsulated: null-body=0
Max-Connections: 1000
Options-TTL: 7200
Allow: 204
Allow: trailers
Preview: 0
Transfer-Complete: asp, bat, exe, com
Transfer-Ignore: html
Transfer-Preview: *
\r\n
Figure 1: OPTIONS handshake
The OPTIONS response in Figure 1 contains two Allow headers to illustrate one of several possible implementations. A compliant server can also send a single Allow header with a list of values, just like the client does in the above example.
Figure 2 is a RESPMOD request with an ICAP trailer that the client can send after receiving an OPTIONS response with Allow/trailers shown in Figure 1. Note that Trailer field value does not match the actual fields in the trailer.
RESPMOD icap://example.net/sample-service ICAP/1.0
Host: example.net
User-Agent: Example-ICAP-Client/1.2
Allow: 204, trailers
Trailer: TBD
Encapsulated: req-hdr=0, res-hdr=131, res-body=292
\r\n
GET /origin-resource HTTP/1.1
Host: www.example.com
Accept: text/html, text/plain, image/gif
Accept-Encoding: gzip, compress
\r\n
HTTP/1.1 200 OK
Date: Sat, 13 Aug 2016 12:17:22 GMT
Server: Testserver/1.0 (Unix)
ETag: "63840-1ab7-378d415b"
Content-Type: text/html
Content-Length: 24
\r\n
18
Origin server sent this.
0
\r\n
X-Client-Log-Lineno: 15612570
X-Client-Status: disconnected (at 1470262108)
\r\n
Figure 2: Request Trailer
Figure 3 is a RESPMOD response with an HTTP and ICAP trailers. The ICAP trailer contains X-Threat-Found and Connection field. To respond with an ICAP trailer, the server ought to receive Allow/trailers in the corresponding ICAP request.
ICAP/1.0 200 OK
Date: Sat, 13 Aug 2016 12:17:23 GMT
Server: ICAP-Server-Software/1.0
Connection: keep-alive
ISTag: "W3E4R7U9-L2E4-2"
Encapsulated: res-hdr=0, res-body=276
Allow: trailers
Trailer: X-ICAP-Log-Lineno, X-Threat-Found
\r\n
HTTP/1.1 200 OK
Date: Sat, 13 Aug 2016 12:17:22 GMT
Server: Testserver/1.0 (Unix)
ETag: "63840-1ab7-378d415b"
Content-Type: text/html
Content-Length: 24
Trailer: X-Content-Checksum
\r\n
18
Origin server sent this.
0
X-Content-Checksum: sha-short=366bc24373b
\r\n
X-Threat-Found: Type=0; Resolution=1; Threat=plain-text
Connection: close
\r\n
Figure 3: Two Trailers
XXX: Polish and validate offsets, lengths, etc. TODO: Figure out how to keep figure descriptions and figures on the same page without inserting empty space into HTML rendering of the draft.
Proper trailer support reduces old ICAP [RFC3507] security concerns because implementations unaware of trailer complexities are arguably more likely to misbehave when receiving HTTP or ICAP trailers.
Sending request headers with multiple Allow tokens could crash poor-quality ICAP servers unaware of this specification. Trailer support negotiation rules partially mitigate that risk by restricting unaware implementations exposure; such implementations are exposed only during OPTIONS exchanges. Since OPTIONS transaction has to precede any HTTP message processing, and since virtually all ICAP client-server relationships are stable, most poor-quality implementations would be detected early and reliably.
A naive implementation of trailer interpretation logic might update an already "frozen" or "committed" (at header parsing time) state of the ICAP transaction or connection, resulting in crashes and other problems. For example, such an implementation could panic after discovering a Connection trailer field value that contradicts the Connection header field value that has already been received and processed at the beginning of the same transaction.
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC3507] | Elson, J. and A. Cerpa, "Internet Content Adaptation Protocol (ICAP)", RFC 3507, DOI 10.17487/RFC3507, April 2003. |
| [RFC5234] | Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008. |
| [RFC7230] | Fielding, R. and J. Reschke, "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing", RFC 7230, DOI 10.17487/RFC7230, June 2014. |
| [Errata] | Various Authors, , "RFC 3507 Errata" |