Internet DRAFT - draft-johndoe-http2-large-header-blocks
draft-johndoe-http2-large-header-blocks
HTTPbis J. Doe
Internet-Draft ACME Corporation
Intended status: Standards Track M. Mustermann
Expires: January 3, 2015 ACME GmbH
July 2, 2014
HTTP/2 Extension: Large Header Blocks
draft-johndoe-http2-large-header-blocks-00
Abstract
This document defines an extension to HTTP/2 for transmitting large
header blocks.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 3, 2015.
Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Doe & Mustermann Expires January 3, 2015 [Page 1]
Internet-Draft CONTINUATION July 2014
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions and Terminology . . . . . . . . . . . . . . . . . 3
3. Additions and Changes to HTTP/2 . . . . . . . . . . . . . . . 3
4. Header Compression and Decompression . . . . . . . . . . . . . 3
5. Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
5.1. CONTINUATION . . . . . . . . . . . . . . . . . . . . . . . 4
5.2. HEADERS . . . . . . . . . . . . . . . . . . . . . . . . . 5
5.3. PUSH_PROMISE . . . . . . . . . . . . . . . . . . . . . . . 6
6. Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
7. HTTP Request/Response Exchange . . . . . . . . . . . . . . . . 7
7.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 8
8. Additional Requirements/Considerations . . . . . . . . . . . . 10
9. Security Considerations . . . . . . . . . . . . . . . . . . . 10
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
10.1. Frame Type Registry Update . . . . . . . . . . . . . . . . 11
10.2. Settings Registry Update . . . . . . . . . . . . . . . . . 11
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
12.1. Normative References . . . . . . . . . . . . . . . . . . . 11
12.2. Informative References . . . . . . . . . . . . . . . . . . 12
Appendix A. Change Log (to be removed by RFC Editor before
publication) . . . . . . . . . . . . . . . . . . . . 12
A.1. Since draft-johndoe-http2-large-header-blocks-00 . . . . . 12
Doe & Mustermann Expires January 3, 2015 [Page 2]
Internet-Draft CONTINUATION July 2014
1. Introduction
The Hypertext Transfer Protocol version 2 (HTTP/2 [http2]) introduced
a framing-based transport layer with multi-stream multiplexing.
HTTP/2 also introduced header compression. However, HTTP/2 header
blocks are restricted to the size of the payload of a single frame
(currently 16383 bytes).
This document defines an extension to HTTP/2 for transmitting large
header blocks. A new HTTP/2 CONTINUATION frame type is introduced
for virtually extending the length of a HEADERS or PUSH_PROMISE
frame.
Note that special care must be taken to implement this extension.
CONTINUATION frames allow a header block larger than one frame to be
fragmented into multiple frames, yet taken together, the frames are
logically equivalent to a single frame. Since header blocks can
modify the shared compression context in HTTP/2, other frames (from
any stream) MUST NOT occur between HEADERS frames and any
CONTINUATION frames that might follow, or between PUSH_PROMISE frames
and any CONTINUATION frames that might follow.
2. Conventions and Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Additions and Changes to HTTP/2
This document defines a new HTTP/2 CONTINUATION frame type.
This document defines a new HTTP/2 SETTINGS_ENABLE_CONTINUATION
setting.
This document adds an END_HEADERS flag to the HEADERS and
PUSH_PROMISE frame types.
4. Header Compression and Decompression
In HTTP/2, header sets are collections of zero or more header fields
serialized into a header block using HTTP Header Compression
[COMPRESSION]. However, blocks of headers are restricted to the size
of the payload of a single frame (currently 16383 bytes).
This document lifts the requirement for header blocks to fit within a
single frame. Instead, the serialized header block can be divided
into one or more octet sequences, called header block fragments. The
Doe & Mustermann Expires January 3, 2015 [Page 3]
Internet-Draft CONTINUATION July 2014
fragments are transmitted within the payload of HEADERS
(Section 5.2), PUSH_PROMISE (Section 5.3) or CONTINUATION
(Section 5.1) frames.
A complete header block consists of either:
o a single HEADERS or PUSH_PROMISE frame, with the END_HEADERS flag
set, or
o a HEADERS or PUSH_PROMISE frame with the END_HEADERS flag cleared
and one or more CONTINUATION frames, where the last CONTINUATION
frame has the END_HEADERS flag set.
The last frame in a header block MUST have the END_HEADERS flag set.
This allows a header block which contains one or more CONTINUATION
frames to be logically equivalent to a single frame.
Since HTTP/2 header compression is stateful (using a single
compression context for the entire connection), header blocks MUST be
transmitted as a contiguous sequence of frames, with no interleaved
frames of any other type or from any other stream.
Header block fragments can only be sent as the payload of HEADERS,
PUSH_PROMISE or CONTINUATION frames. Since these frames carry data
that can modify the compression context maintained by a receiver, an
endpoint receiving HEADERS, PUSH_PROMISE or CONTINUATION frames MUST
reassemble header blocks and perform decompression even if the frames
are to be discarded. A receiver MUST terminate the connection with a
connection error of type COMPRESSION_ERROR if it does not decompress
a header block.
A receiving endpoint reassembles a header block by concatenating its
fragments, then decompresses the block to reconstruct the header set.
Each header block is processed as a discrete unit.
5. Frames
This specification introduces a new CONTINUATION and a new flag for
the HEADERS and PUSH_PROMISE frame types.
5.1. CONTINUATION
The CONTINUATION frame (type=0x9) is used to continue a sequence of
header block fragments (Section 4). Any number of CONTINUATION
frames can be sent on an existing stream, as long as the preceding
frame is on the same stream and is a HEADERS, PUSH_PROMISE or
CONTINUATION frame without the END_HEADERS flag set.
Doe & Mustermann Expires January 3, 2015 [Page 4]
Internet-Draft CONTINUATION July 2014
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Header Block Fragment (*) ...
+---------------------------------------------------------------+
CONTINUATION Frame Payload
The CONTINUATION frame payload contains a header block fragment
(Section 4).
The CONTINUATION frame defines the following flag:
END_HEADERS (0x4): Bit 3 being set indicates that this frame ends a
header block (Section 4).
If the END_HEADERS bit is not set, this frame MUST be followed by
another CONTINUATION frame. A receiver MUST treat the receipt of
any other type of frame or a frame on a different stream as a
connection error of type PROTOCOL_ERROR.
The CONTINUATION frame changes the connection state as defined in
Section 4.
CONTINUATION frames MUST be associated with a stream. If a
CONTINUATION frame is received whose stream identifier field is 0x0,
the recipient MUST respond with a connection error of type
PROTOCOL_ERROR.
A CONTINUATION frame MUST be preceded by a HEADERS, PUSH_PROMISE or
CONTINUATION frame without the END_HEADERS flag set. A recipient
that observes violation of this rule MUST respond with a connection
error of type PROTOCOL_ERROR.
If a CONTINUATION frame size exceeds any defined limit, or is too
small to contain mandatory frame data, the receiver MUST terminate
the connection with a connection error of type FRAME_SIZE_ERROR.
5.2. HEADERS
This document modifies the definition of a HEADERS frame such that it
may contain a complete header block or a header block fragment.
This document modifies the definition of the END_STREAM flag and
defines a new END_HEADERS flag. The other flags remain as defined.
Doe & Mustermann Expires January 3, 2015 [Page 5]
Internet-Draft CONTINUATION July 2014
END_STREAM (0x1): Bit 1 being set indicates that the header block
(Section 4) is the last that the endpoint will send for the
identified stream. Setting this flag causes the stream to enter
one of "half closed" states.
A HEADERS frame that is followed by CONTINUATION frames carries
the END_STREAM flag that signals the end of a stream. A
CONTINUATION frame cannot be used to terminate a stream.
END_HEADERS (0x4): Bit 3 being set indicates that this frame
contains an entire header block (Section 4) and is not followed by
any CONTINUATION frames.
A HEADERS frame without the END_HEADERS flag set MUST be followed
by a CONTINUATION frame for the same stream. A receiver MUST
treat the receipt of any other type of frame or a frame on a
different stream as a connection error of type PROTOCOL_ERROR.
The payload of a HEADERS frame contains a header block fragment
(Section 4). A header block that does not fit within a HEADERS frame
is continued in a CONTINUATION frame (Section 5.1).
5.3. PUSH_PROMISE
This document modifies the definition of a PUSH_PROMISE frame such
that it may contain a complete header block or a header block
fragment.
This document defines a new END_HEADERS flag. The other existing
flags remain as defined.
END_HEADERS (0x4): Bit 3 being set indicates that this frame
contains an entire header block (Section 4) and is not followed by
any CONTINUATION frames.
A PUSH_PROMISE frame without the END_HEADERS flag set MUST be
followed by a CONTINUATION frame for the same stream. A receiver
MUST treat the receipt of any other type of frame or a frame on a
different stream as a connection error of type PROTOCOL_ERROR.
The payload of a PUSH_PROMISE frame contains a header block fragment
(Section 4). A header block that does not fit within a PUSH_PROMISE
frame is continued in a CONTINUATION frame (Section 5.1).
The inclusion of a header block fragment (Section 4) in a
PUSH_PROMISE frame potentially modifies the state maintained for
header compression.
Doe & Mustermann Expires January 3, 2015 [Page 6]
Internet-Draft CONTINUATION July 2014
6. Settings
This document defines the following new setting:
SETTINGS_ENABLE_CONTINUATION (TBD): Indicates the sender's ability
and willingness to accept CONTINUATION frames. An endpoint MUST
NOT send a CONTINUATION frame unless it receives this parameter
set to a value of 1.
An endpoint that has not set this parameter to 1, but receives a
CONTINUATION frame MAY accept it. Since CONTINUATION frames
likely contain data that will alter the compression context,
recipients should treat unwanted CONTINUATION frames as a
connection error of type PROTOCOL_ERROR.
The initial value is 0, which indicates an endpoint MUST NOT send
CONTINUATION frames. Any value other than 0 or 1 MUST be treated
as a connection error of type PROTOCOL_ERROR.
7. HTTP Request/Response Exchange
The sequence of frames in an HTTP message (request or response) is
updated as follows in consideration of CONTINUATION frames:
1. one HEADERS frame (followed by zero or more CONTINUATION frames)
containing the message headers (see [RFC7230], Section 3.2), and
2. zero or more DATA frames containing the message payload (see
[RFC7230], Section 3.3), and
3. optionally, one HEADERS frame, followed by zero or more
CONTINUATION frames containing the trailer-part, if present (see
[RFC7230], Section 4.1.2).
The last frame in the sequence bears an END_STREAM flag, noting that
a HEADERS frame bearing the END_STREAM flag can be followed by
CONTINUATION frames that carry any remaining portions of the header
block.
Other frames (from any stream) MUST NOT occur between either HEADERS
frame and any CONTINUATION frames that might follow.
Otherwise, frames MAY be interspersed on the stream between these
frames, but those frames do not carry HTTP semantics. In particular,
HEADERS frames (and any CONTINUATION frames that follow) other than
the first and optional last frames in this sequence do not carry HTTP
semantics.
Doe & Mustermann Expires January 3, 2015 [Page 7]
Internet-Draft CONTINUATION July 2014
Trailing header fields are carried in a header block that also
terminates the stream. That is, a sequence starting with a HEADERS
frame, followed by zero or more CONTINUATION frames, where the
HEADERS frame bears an END_STREAM flag. Header blocks after the
first that do not terminate the stream are not part of an HTTP
request or response.
7.1. Examples
This section shows HTTP/1.1 requests and responses, with
illustrations of equivalent HTTP/2 requests and responses. The
examples are intentionally similar to the examples in [XXX], but have
been modified to show the use of CONTINUATION frames and the
END_HEADERS flag on HEADERS frames.
An HTTP GET request includes request header fields and no body and is
therefore transmitted as a single HEADERS frame, followed by zero or
more CONTINUATION frames containing the serialized block of request
header fields. The HEADERS frame in the following has both the
END_HEADERS and END_STREAM flags set; no CONTINUATION frames are
sent:
GET /resource HTTP/1.1 HEADERS
Host: example.org ==> + END_STREAM
Accept: image/jpeg + END_HEADERS
:method = GET
:scheme = https
:path = /resource
host = example.org
accept = image/jpeg
Similarly, a response that includes only response header fields is
transmitted as a HEADERS frame (again, followed by zero or more
CONTINUATION frames) containing the serialized block of response
header fields.
HTTP/1.1 304 Not Modified HEADERS
ETag: "xyzzy" ==> + END_STREAM
Expires: Thu, 23 Jan ... + END_HEADERS
:status = 304
etag = "xyzzy"
expires = Thu, 23 Jan ...
An HTTP POST request that includes request header fields and payload
data is transmitted as one HEADERS frame, followed by zero or more
CONTINUATION frames containing the request header fields, followed by
one or more DATA frames, with the last CONTINUATION (or HEADERS)
frame having the END_HEADERS flag set and the final DATA frame having
Doe & Mustermann Expires January 3, 2015 [Page 8]
Internet-Draft CONTINUATION July 2014
the END_STREAM flag set:
POST /resource HTTP/1.1 HEADERS
Host: example.org ==> - END_STREAM
Content-Type: image/jpeg - END_HEADERS
Content-Length: 123 :method = POST
:path = /resource
{binary data} content-type = image/jpeg
CONTINUATION
+ END_HEADERS
host = example.org
:scheme = https
content-length = 123
DATA
+ END_STREAM
{binary data}
Note that data contributing to any given header field could be spread
between header block fragments. The allocation of header fields to
frames in this example is illustrative only.
A response that includes header fields and payload data is
transmitted as a HEADERS frame, followed by zero or more CONTINUATION
frames, followed by one or more DATA frames, with the last DATA frame
in the sequence having the END_STREAM flag set:
HTTP/1.1 200 OK HEADERS
Content-Type: image/jpeg ==> - END_STREAM
Content-Length: 123 + END_HEADERS
:status = 200
{binary data} content-type = image/jpeg
content-length = 123
DATA
+ END_STREAM
{binary data}
Trailing header fields are sent as a header block after both the
request or response header block and all the DATA frames have been
sent. The HEADERS frame starting the trailers header block has the
END_STREAM flag set.
Doe & Mustermann Expires January 3, 2015 [Page 9]
Internet-Draft CONTINUATION July 2014
HTTP/1.1 200 OK HEADERS
Content-Type: image/jpeg ==> - END_STREAM
Transfer-Encoding: chunked + END_HEADERS
Trailer: Foo :status = 200
content-length = 123
123 content-type = image/jpeg
{binary data} trailer = Foo
0
Foo: bar DATA
- END_STREAM
{binary data}
HEADERS
+ END_STREAM
+ END_HEADERS
foo = bar
8. Additional Requirements/Considerations
After sending a GOAWAY frame, the sender can discard frames for
streams with identifiers higher than the identified last stream.
However, any frames that alter connection state cannot be completely
ignored. For instance, HEADERS, PUSH_PROMISE and CONTINUATION frames
MUST be minimally processed to ensure the state maintained for header
compression is consistent (see Section 4).
CONTINUATION frames do not result in state transitions and are
effectively part of the HEADERS or PUSH_PROMISE that they follow.
PUSH_PROMISE frames can be sent by the server in response to any
client-initiated stream, but the stream MUST be in either the "open"
or "half closed (remote)" state with respect to the server.
PUSH_PROMISE frames are interspersed with the frames that comprise a
response, though they cannot be interspersed with HEADERS and
CONTINUATION frames that comprise a single header block.
9. Security Considerations
A large header block (Section 4) can cause an implementation to
commit a large amount of state. In servers and intermediaries,
header fields that are critical to routing, such as ":authority",
":path", and ":scheme" are not guaranteed to be present early in the
header block. In particular, values that are in the reference set
cannot be emitted until the header block ends.
This can prevent streaming of the header fields to their ultimate
destination, and forces the endpoint to buffer the entire header
block. Since there is no hard limit to the size of a header block,
Doe & Mustermann Expires January 3, 2015 [Page 10]
Internet-Draft CONTINUATION July 2014
an endpoint could be forced to exhaust available memory.
A server that receives a larger header block than it is willing to
handle can send an HTTP 431 (Request Header Fields Too Large) status
code [RFC6585]. A client can discard responses that it cannot
process. The header block MUST be processed to ensure a consistent
connection state, unless the connection is closed.
10. IANA Considerations
This document updates the registries for frame types and settings in
the "Hypertext Transfer Protocol (HTTP) 2 Parameters" section.
10.1. Frame Type Registry Update
This document updates the "HTTP/2 Frame Type" registry. The entries
in the following table are registered by this document.
+--------------+------+-------------+
| Frame Type | Code | Section |
+--------------+------+-------------+
| CONTINUATION | 0x9 | Section 5.1 |
+--------------+------+-------------+
10.2. Settings Registry Update
This document updates the "HTTP/2 Settings" registry. The entries in
the following table are registered by this document.
+------------------------------+------+-------------+---------------+
| Name | Code | Initial | Specification |
| | | Value | |
+------------------------------+------+-------------+---------------+
| SETTINGS_ENABLE_CONTINUATION | TBD | 0 | Paragraph 1 |
+------------------------------+------+-------------+---------------+
11. Acknowledgements
This document was ported from the HTTP/2 specification [http2] by
Keith Shearl Morgan.
12. References
12.1. Normative References
[COMPRESSION] Ruellan, H. and R. Peon, "HPACK - Header Compression
for HTTP/2", draft-ietf-httpbis-header-compression-08
(work in progress), June 2014.
Doe & Mustermann Expires January 3, 2015 [Page 11]
Internet-Draft CONTINUATION July 2014
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext
Transfer Protocol (HTTP/1.1): Message Syntax and
Routing", RFC 7230, June 2014.
[http2] Belshe, M., Peon, R., and M. Thomson, "Hypertext
Transfer Protocol version 2",
draft-ietf-httpbis-http2-13 (work in progress),
June 2014.
12.2. Informative References
[RFC6585] Nottingham, N. and R. Fielding, "Additional HTTP
Status Codes", RFC 6585, April 2012.
Appendix A. Change Log (to be removed by RFC Editor before publication)
A.1. Since draft-johndoe-http2-large-header-blocks-00
None (yet).
Authors' Addresses
John Doe
ACME Corporation
Max Mustermann
ACME GmbH
Doe & Mustermann Expires January 3, 2015 [Page 12]