| Network Working Group | J. Yasskin |
| Internet-Draft | |
| Intended status: Standards Track | July 08, 2019 |
| Expires: January 9, 2020 |
Bundled HTTP Exchanges
draft-yasskin-wpack-bundled-exchanges-01
Bundled exchanges provide a way to bundle up groups of HTTP request+response pairs to transmit or store them together. They can include multiple top-level resources with one identified as the default by a manifest, provide random access to their component exchanges, and efficiently store 8-bit resources.
Discussion of this draft takes place on the wpack mailing list (wpack@ietf.org), which is archived at https://www.ietf.org/mailman/listinfo/wpack.
The source code and issues list for this draft can be found in https://github.com/WICG/webpackage.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 9, 2020.
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
To satisfy the use cases in [I-D.yasskin-webpackage-use-cases], this document proposes a new bundling format to group HTTP resources. Several of the use cases require the resources to be signed: that’s provided by bundling signed exchanges ([I-D.yasskin-http-origin-signed-responses]) rather than natively in this format.
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
This specification defines how conformant bundle parsers work. It does not constrain how encoders produce a bundle: although there are some guidelines in Section 4, encoders MAY produce any sequence of bytes that a conformant parser would parse into the intended semantics.
This specification uses the conventions and terminology defined in the Infra Standard ([INFRA]).
A bundle is logically a set of HTTP exchanges, with a URL identifying the manifest(s) of the bundle itself.
While the order of the exchanges is not semantically meaningful, it can significantly affect performance when the bundle is loaded from a network stream.
A bundle is parsed from a stream of bytes, which is assumed to have the attributes and operations described in Section 2.1.
Bundle parsers support two operations, Load a bundle’s metadata (Section 2.2) and Load a response from a bundle (Section 2.3) each of which can return an error instead of their normal result.
A client is expected to load the metadata for a bundle as soon as it starts downloading it or otherwise discovers it. Then, when fetching ([FETCH]) a request, the client is expected to match it against the requests in the metadata, and if one matches, load that request’s response.
This takes the bundle’s stream and returns either an error (where an error is a “format error” or a “version error”), an error with a fallback URL (which is also the primaryUrl when the bundle parses successfully), or a map ([INFRA]) of metadata containing at least keys named:
The map may include other items added by sections defined in the Web Bundle Section Name Registry.
This operation only waits for a prefix of the stream that, if the bundle is encoded with the “responses” section last, ends before the first response.
This operation’s implementation is in Section 3.3.
If a bundle’s bytes are embedded in a longer sequence rather than being streamed, a parser can also load them starting from a pointer to the last byte of the bundle. This returns the same data as Section 2.2.
This operation’s implementation is in Section 3.3.6.
This takes the stream of bytes representing the bundle, a request ([FETCH]), and the ResponseMetadata returned from Section 2.2 for the appropriate content-negotiated resource within the request’s URL, and returns the response ([FETCH]) matching that request.
This operation can be completed without inspecting bytes other than those that make up the loaded response, although higher-level operations like proving that an exchange is correctly signed ([I-D.yasskin-http-origin-signed-responses]) may need to load other responses.
A client will generally want to load the response for a request that the client generated. For a URL with multiple variants, the client SHOULD use the algorithm in Section 4 of [I-D.ietf-httpbis-variants] to select the best variant.
This operation’s implementation is in Section 3.4.
This section is non-normative.
A bundle holds a series of named sections. The beginning of the bundle maps section names to the range of bytes holding that section. The most important section is the “index” (Section 3.3.1), which similarly maps serialized HTTP requests to the range of bytes holding that request’s serialized response. Byte ranges are represented using an offset from some point in the bundle after the encoding of the range itself, to reduce the amount of work needed to use the shortest possible encoding of the range.
Future specifications can define new sections with extra data, and if necessary, these sections can be marked “critical” (Section 3.3.4) to prevent older parsers from using the rest of the bundle incorrectly.
The bundle is a CBOR item ([CBORbis]) with the following CDDL ([CDDL]) schema:
webbundle = [ ; 🌐📦 in UTF-8. magic: h'F0 9F 8C 90 F0 9F 93 A6', version: bytes .size 4, primary-url: whatwg-url, section-lengths: bytes .cbor [* (section-name: tstr, length: uint) ], sections: [* any ], length: bytes .size 8, ; Big-endian number of bytes in the bundle. ] $section-name /= "index" / "manifest" / "signatures" / "critical" / "responses" $section /= index / manifest / signatures / critical / responses responses = [*response] whatwg-url = tstr
When served over HTTP, a response containing an application/webbundle payload MUST include at least the following response header fields, to reduce content sniffing vulnerabilities (Section 5.2):
A bundle holds a series of sections, which can be accessed randomly using the information in the section-lengths CBOR item, which holds a list of alternating section names and section lengths:
section-lengths = [* (section-name: tstr, length: uint) ],
To implement Section 2.2, the parser MUST run the following steps, taking the stream as input.
The “index” section defines the set of HTTP requests in the bundle and identifies their locations in the “responses” section. It consists of a map from URL strings to arrays consisting of a Variants header field value ([I-D.ietf-httpbis-variants]) followed by one location-in-responses pair for each of the possible combinations of available-values within the Variants value in lexicographic (row-major) order.
For example, given a variants-value of Accept-Encoding;gzip;br, Accept-Language;en;fr;ja, the list of location-in-responses pairs will correspond to the VariantKeys:
The order of variant-axes is important. If the variants-value were Accept-Language;en;fr;ja, Accept-Encoding;gzip;br instead, the location-in-responses pairs would instead correspond to:
As a special case, an empty variants-value indicates that there is only one resource at the specified URL and that no content negotiation is performed.
index = {* whatwg-url => [ variants-value, +location-in-responses ] }
variants-value = bstr
location-in-responses = (offset: uint, length: uint)
A ResponseMetadata struct identifies a byte range within the bundle stream, defined by an integer offset from the start of the stream and the integer number of bytes in the range.
To parse the index section, given its sectionContents, the sectionOffsets map, and the metadata map to fill in, the parser MUST do the following:
The “manifest” section records a single URL identifying the manifest of the bundle. The URL MUST refer to the one or more response(s) contained in the bundle itself.
The bundle can contain multiple resources at this URL, and the client is expected to content-negotiate for the best one. For example, a client might select the one with an accept header of application/manifest+json ([appmanifest]) and an accept-language header of es-419.
manifest = whatwg-url
To parse the manifest section, given its sectionContents and the metadata map to fill in, the parser MUST do the following:
The “signatures” section vouches for the resources in the bundle.
The section can contain as many signatures as needed, each by some authority, and each covering an arbitrary subset of the resources in the bundle. Intermediates, including attackers, can remove signatures from the bundle without breaking the other signatures.
The bundle parser’s client is responsible to determine the validity and meaning of each authority’s signatures. In particular, the algorithm below does not check that signatures are valid. For example, a client might:
A client might also choose different behavior for those kinds of authorities and keys.
signatures = [
authorities: [*authority],
vouched-subsets: [*{
authority: index-in-authorities,
sig: bstr,
signed: bstr ; Expected to hold a signed-subset item.
}],
]
authority = augmented-certificate
index-in-authorities = uint
signed-subset = {
validity-url: whatwg-url,
auth-sha256: bstr,
date: uint,
expires: uint,
subset-hashes: {+
whatwg-url => [variants-value, +resource-integrity]
},
* tstr => any,
}
resource-integrity = (header-sha256: bstr, payload-integrity-header: tstr)
The augmented-certificate CDDL rule comes from Section 3.3 of [I-D.yasskin-http-origin-signed-responses].
To parse the signatures section, given its sectionContents, the sectionOffsets map, and the metadata map to fill in, the parser MUST do the following:
The “critical” section lists sections of the bundle that the client needs to understand in order to load the bundle correctly. Other sections are assumed to be optional.
critical = [*tstr]
To parse the critical section, given its sectionContents and the metadata map to fill in, the parser MUST do the following:
This section does not modify the returned metadata.
The responses section does not add any items to the bundle metadata map. Instead, its offset and length are used in processing the index section (Section 3.3.1).
The length of a bundle is encoded as a big-endian integer inside a CBOR byte string at the end of the bundle.
+------------+-----+----+----+----+----+----+----+----+----+----+
| first byte | ... | 48 | 00 | 00 | 00 | 00 | 00 | BC | 61 | 4E |
+------------+-----+----+----+----+----+----+----+----+----+----+
/ \
0xBC614E-10=12345668 omitted bytes
Figure 1: Example trailing bytes
Parsing from the end allows the bundle to be appended to another format such as a self-extracting executable.
To implement Section 2.2.1, taking a sequence of bytes bytes, the client MUST:
The result of Load a bundle’s metadata maps each URL and Variant-Key ([I-D.ietf-httpbis-variants]) to a response, which consists of headers and a payload. The headers can be loaded from the bundle’s stream before waiting for the payload, and similarly the payload can be streamed to downstream consumers.
response = [headers: bstr .cbor headers, payload: bstr]
To implement Section 2.3, the parser MUST run the following steps, taking the bundle’s stream, a request ([FETCH]), and a responseMetadata returned by Section 2.2 .
Parsing a bundle involves parsing many CBOR items. All of these items need to be deterministically encoded.
To parse a CBOR item ([CBORbis]), optionally matching a CDDL rule ([CDDL]), from a sequence of bytes, bytes, the parser MUST do the following:
Bundles encode variable-length data in CBOR bytestrings, which are prefixed with their length. This algorithm returns the number of bytes in the variable-length item and sets the stream’s current offset to the first byte of the contents.
To get the length of a CBOR bytestring header from a bundle’s stream, the parser MUST do the following:
To parse the type and argument of a CBOR item from a bundle’s stream, the parser MUST do the following. This algorithm returns a pair of the CBOR major type 0–7 inclusive, and a 64-bit integral argument for the CBOR item:
Bundles represent HTTP requests and responses as a list of headers, matching the following CDDL ([CDDL]):
headers = {* bstr => bstr}
Pseudo-headers starting with a : provide the non-header information needed to create a request or response as appropriate
To convert a CBOR item item into a [FETCH] header list and pseudoheaders, parsers MUST do the following:
Bundles SHOULD consist of a single CBOR item satisfying the core deterministic encoding requirements (Section 3.5) and matching the webbundle CDDL rule in Section 3.1.
Bundles currently have no mechanism for ensuring that the signed exchanges they contain constitute a consistent version of those resources. Even if a website never has a security vulnerability when resources are fetched at a single time, an attacker might be able to combine a set of resources pulled from different versions of the website to build a vulnerable site. While the vulnerable site could have occurred by chance on a client’s machine due to normal HTTP caching, bundling allows an attacker to guarantee that it happens. Future work in this specification might allow a bundle to constrain its resources to come from a consistent version.
While modern browsers tend to trust the Content-Type header sent with a resource, especially when accompanied by X-Content-Type-Options: nosniff, plugins will sometimes search for executable content buried inside a resource and execute it in the context of the origin that served the resource, leading to XSS vulnerabilities. For example, some PDF reader plugins look for %PDF anywhere in the first 1kB and execute the code that follows it.
The application/webbundle format defined above includes URLs and request headers early in the format, which an attacker could use to cause these plugins to sniff a bad content type.
To avoid vulnerabilities, in addition to the response header requirements in Section 3.2, servers are advised to only serve an application/webbundle resource from a domain if it would also be safe for that domain to serve the bundle’s content directly, and to follow at least one of the following strategies:
If the server serves responses that are written by a potential attacker but then escaped, the application/webbundle format allows the attacker to use the length of the response to control a few bytes before the start of the response. Any existing mechanisms that prevent polyglot documents probably keep working in the face of this new attack, but we don’t have a guarantee of that.
To encourage servers to include the X-Content-Type-Options: nosniff header field, clients SHOULD reject bundles served without it.
IANA maintains the registry of Internet Media Types [RFC6838] at https://www.iana.org/assignments/media-types.
IANA is directed to create a new registry with the following attributes:
Name: Web Bundle Section Names
Review Process: Specification Required
Initial Assignments:
| Section Name | Specification | Metadata | Metadata Fields |
|---|---|---|---|
| “index” | Section 3.3.1 | Yes | “requests” |
| “manifest” | Section 3.3.2 | Yes | “manifest” |
| “signatures” | Section 3.3.3 | Yes | “authorities”, “vouched-subsets” |
| “critical” | Section 3.3.4 | Yes | |
| “responses” | Section 3.3.5 | No |
Requirements on new assignments:
Section Names MUST be encoded in UTF-8.
Assignments must specify whether the section is parsed during Load a bundle’s metadata (Metadata=Yes) or not (Metadata=No).
The section’s specification can use the bytes making up the section, the bundle’s stream (Section 2.1), and the sectionOffsets map (Section 3.3), as input, and MUST say if an error is returned, and otherwise what items, if any, are added to the map that Section 3.3 returns. A section’s specification MAY say that, if it is present, another section is not processed.
| [appmanifest] | Caceres, M., Christiansen, K., Lamouri, M., Kostiainen, A., Dolin, R. and M. Giuca, "Web App Manifest", World Wide Web Consortium WD WD-appmanifest-20180523, May 2018. |
| [CBORbis] | Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", Internet-Draft draft-ietf-cbor-7049bis-06, July 2019. |
| [CDDL] | Birkholz, H., Vigano, C. and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, June 2019. |
| [FETCH] | WHATWG, "Fetch", July 2019. |
| [I-D.ietf-httpbis-variants] | Nottingham, M., "HTTP Representation Variants", Internet-Draft draft-ietf-httpbis-variants-05, March 2019. |
| [I-D.yasskin-http-origin-signed-responses] | Yasskin, J., "Signed HTTP Exchanges", Internet-Draft draft-yasskin-http-origin-signed-responses-06, July 2019. |
| [INFRA] | WHATWG, "Infra", July 2019. |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC5234] | Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008. |
| [RFC7540] | Belshe, M., Peon, R. and M. Thomson, "Hypertext Transfer Protocol Version 2 (HTTP/2)", RFC 7540, DOI 10.17487/RFC7540, May 2015. |
| [RFC8174] | Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017. |
| [SRI] | Akhawe, D., Braun, F., Marier, F. and J. Weinberger, "Subresource Integrity", World Wide Web Consortium Recommendation REC-SRI-20160623, June 2016. |
| [URL] | WHATWG, "URL", July 2019. |
| [I-D.yasskin-webpackage-use-cases] | Yasskin, J., "Use Cases and Requirements for Web Packages", Internet-Draft draft-yasskin-webpackage-use-cases-01, March 2018. |
| [RFC6265] | Barth, A., "HTTP State Management Mechanism", RFC 6265, DOI 10.17487/RFC6265, April 2011. |
| [RFC6838] | Freed, N., Klensin, J. and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013. |
| [TLS1.3] | Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018. |
RFC EDITOR PLEASE DELETE THIS SECTION.
draft-01
Thanks to the Chrome loading team, especially Kinuko Yasuda and Kouhei Ueno for making the format work well when streamed.