Internet DRAFT - draft-hallambaker-dare-container
draft-hallambaker-dare-container
Network Working Group P. Hallam-Baker
Internet-Draft Comodo Group Inc.
Intended status: Informational August 27, 2018
Expires: February 28, 2019
Data At Rest Encryption Part 2: DARE Container
draft-hallambaker-dare-container-02
Abstract
This document describes DARE Container, a message and file syntax
that allows an append-only sequence of data frames to be represented
with cryptographic integrity, signature and encryption enhancements.
The format supports data integrity checks using digest chains and
Merkle trees. The simplest supports efficient write operations and
efficient read operations in either the forward or reverse direction.
Support for efficient random-access reads may be provided through the
use of binary trees or index records appended to the end of the file.
This document is also available online at
http://mathmesh.com/Documents/draft-hallambaker-dare-container.html
[1] .
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 28, 2019.
Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
Hallam-Baker Expires February 28, 2019 [Page 1]
Internet-Draft DARE Data Container August 2018
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Container Format . . . . . . . . . . . . . . . . . . . . 3
1.2. Write . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3. Read Access . . . . . . . . . . . . . . . . . . . . . . . 5
1.4. Encryption and Authentication . . . . . . . . . . . . . . 5
1.5. Integrity and Signature . . . . . . . . . . . . . . . . . 5
1.6. Redaction . . . . . . . . . . . . . . . . . . . . . . . . 6
1.7. Alternative approaches . . . . . . . . . . . . . . . . . 6
1.8. Efficiency . . . . . . . . . . . . . . . . . . . . . . . 7
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1. Related Specifications . . . . . . . . . . . . . . . . . 7
2.2. Requirements Language . . . . . . . . . . . . . . . . . . 8
3. Container Navigation . . . . . . . . . . . . . . . . . . . . 8
3.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2. Position Index . . . . . . . . . . . . . . . . . . . . . 9
3.3. Metadata Index . . . . . . . . . . . . . . . . . . . . . 10
4. Integrity Mechanisms . . . . . . . . . . . . . . . . . . . . 10
4.1. Digest Chain calculation . . . . . . . . . . . . . . . . 10
4.2. Binary Merkle tree calculation . . . . . . . . . . . . . 10
5. Reference . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.1. Container Headers . . . . . . . . . . . . . . . . . . . . 11
5.1.1. Structure: ContainerHeaderFirst . . . . . . . . . . . 11
5.1.2. Structure: ContainerHeader . . . . . . . . . . . . . 11
5.2. Content Metadata Structure . . . . . . . . . . . . . . . 12
5.2.1. Structure: ContentMeta . . . . . . . . . . . . . . . 12
5.3. Index Structures . . . . . . . . . . . . . . . . . . . . 13
5.3.1. Structure: ContainerIndex . . . . . . . . . . . . . . 13
5.3.2. Structure: IndexPosition . . . . . . . . . . . . . . 13
5.3.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 13
5.3.4. Structure: IndexMeta . . . . . . . . . . . . . . . . 13
5.4. Signature . . . . . . . . . . . . . . . . . . . . . . . . 14
6. Security Considerations . . . . . . . . . . . . . . . . . . . 14
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14
9. Appendix A: Examples and Test Vectors . . . . . . . . . . . . 14
9.1. Simple container . . . . . . . . . . . . . . . . . . . . 15
9.2. Payload and chain digests . . . . . . . . . . . . . . . . 16
9.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . . 17
Hallam-Baker Expires February 28, 2019 [Page 2]
Internet-Draft DARE Data Container August 2018
9.4. Signed container . . . . . . . . . . . . . . . . . . . . 19
9.5. Encrypted container . . . . . . . . . . . . . . . . . . . 21
10. Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . 25
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 25
11.1. Normative References . . . . . . . . . . . . . . . . . . 25
11.2. Informative References . . . . . . . . . . . . . . . . . 26
11.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction
DARE Container is a message and file syntax that allows a sequence of
data frames to be represented with cryptographic integrity,
signature, and encryption enhancements to be constructed in an append
only format. DARE Container was developed in response to needs that
arose out of the design of the Mathematical Mesh
[draft-hallambaker-mesh-architecture] . It is built on the binary
encodings of JSON data objects, JSON-B and JSON-C
[draft-hallambaker-jsonbcd] and the DARE Message format
[draft-hallambaker-dare-message] .
The format is designed to meet the requirements of a wide range of
use cases including:
o Recording transactions in persistent storage.
o Synchronizing transaction logs between hosts.
o File archive.
o Message spool.
o Signing and encrypting single data items.
o Incremental encryption and authentication of server logs.
1.1. Container Format
A DARE Container consists of a sequence of variable length frames.
Each frame consists of a forward length indicator, the framed data
and a reverse length indicator. The reverse length indicator is
written out backwards allowing the length and thus the frame to be
read in the reverse direction:
Hallam-Baker Expires February 28, 2019 [Page 3]
Internet-Draft DARE Data Container August 2018
[[This figure is not viewable in this format. The figure is
available at http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html [2].]]
JBCD Bidirectional Frame
Each frame contains a single DARE Message consisting of a Header,
Payload and Trailer (if required). The first frame in a container
describes the container format options and defaults. These include
the range of encoding options for frame metadata supported and the
container profiles to which the container conforms.
All internal data formats are 64 bit clean allowing for containers of
up to 18 exabytes to be written.
Five container types are currently specified but this may be reduced
if the Digest and Tree types are withdrawn. These are:
Simple The container does not provide any index or content integrity
checks.
Tree Frame headers contain entries that specify the start position
of previous frames at the apex of the immediately enclosing binary
tree. This enables efficient random access to any frame in the
file.
Digest Each frame trailer contains a PayloadDigest field.
Modification of the payload will cause verification of the
PayloadDigest value to fail on that frame.
Chain Each frame trailer contains PayloadDigest and ChainDigest
fields allowing modifications to the payload data to be detected.
Modification of the payload will cause verification of the
PayloadDigest value to fail on that frame and verification of the
ChainDigest value to fail on all subsequent frames.
Merkle Tree Frame headers contain entries that specify the start
position of previous frames at the apex of the immediately
enclosing binary tree. Frame Trailers contain
TreeDigestPartial and TreeDigestFinal entries forming a Merkle
digest tree.
1.2. Write
In normal circumstances, DARE Containers are written as an append
only log. As with DARE Messages, integrity information (payload
digest, signatures) is written to the message trailer. Thus, large
Hallam-Baker Expires February 28, 2019 [Page 4]
Internet-Draft DARE Data Container August 2018
payloads may be written without the need to buffer the payload data
provided that the content length is known in advance.
1.3. Read Access
The use of reverse length indicators allows DARE containers to
support efficient sequential access in either the forward or reverse
directions.
Random access to any part of a file MAY be supported by means of a
binary tree index and/or an index record providing direct access to
any part of the file.
1.4. Encryption and Authentication
Frame payloads and associated attributes MAY be encrypted and/or
authenticated using the approach described in
[draft-hallambaker-dare-message] .
Incremental encryption is supported allowing encryption parameters
from a single public key exchange operation to be applied to encrypt
multiple frames. The public key exchange information is specified in
the first encrypted frame and subsequent frames encrypted under those
parameters specify the location at which the key exchange information
is to be found by means of the ExchangePosition field.
The only restriction on the use of incremental encryption is that the
frame containing the key exchange information MUST precede the frames
that reference the exchange parameters.
To avoid cryptographic vulnerabilities resulting from key re-use, the
DARE key exchange requires that each encrypted sequence use an
encryption key and initialization vector derived from the master key
established in the public key exchange by means of a unique salt.
Each DARE Message and by extension, each DARE Container frame MUST
specify a unique salt value of at least 128 bits. Since the
encryption key is derived from the salt value by means of a Key
Derivation Function, erasure of the salt MAY be used as a means of
rendering the payload plaintext value inaccessible without changing
the payload value.
1.5. Integrity and Signature
Signatures MAY be applied to a payload digest, the final digest in a
chain or tree. The chain and tree digest modes allow a single
signature to be used to authenticate all frame payloads in a
container.
Hallam-Baker Expires February 28, 2019 [Page 5]
Internet-Draft DARE Data Container August 2018
The tree signature mode is particularly suited to applications such
as file archives as it allows files to be verified individually
without requiring the signer to sign each individually. Furthermore,
in applications such as code signing, it allows a single signature to
be used to verify both the integrity of the code and its membership
of the distribution.
As with DARE Message, the signature mechanism does not specify the
interpretation of the signature semantics. The presence of a
signature demonstrates that the holder of the private key applied it
to the specified digest value but not their motive for doing so.
Describing such semantics is beyond the scope of this document and is
deferred to future work.
1.6. Redaction
The chief disadvantage of using an append-only format is that
containers only increase in size. In many applications, much of the
data in the container becomes redundant or obsolete and a process
analogous to garbage collection is required. This process is called
redaction.
The simplest method of redaction is to create a new container and
sequentially copy each entry from the old container to the new,
discarding redundant frames and obsolete header information.
For example, partial index records may be consolidated into a single
index record placed in the last frame of the container. Unnecessary
signature and integrity data may be discarded and so on.
It is also possible but not necessarily advisable to perform such a
redaction in-place provided that the redaction process does not
increase the length of any individual frame and that appropriate
provision is made for file locking to prevent conflicts and to
provide for safe resumption should an interruption occur during the
process.
1.7. Alternative approaches
Many file proprietary formats are in use that support some or all of
these capabilities but only a handful have public, let alone open,
standards. DARE Container is designed to provide a superset of the
capabilities of existing message and file syntaxes, including:
o Cryptographic Message Syntax [RFC5652] defines a syntax used to
digitally sign, digest, authenticate, or encrypt arbitrary message
content.
Hallam-Baker Expires February 28, 2019 [Page 6]
Internet-Draft DARE Data Container August 2018
o The.ZIP File Format specification [ZIPFILE] developed by Phil
Katz.
o The BitCoin Block chain [BLOCKCHAIN] .
o JSON Web Encryption and JSON Web Signature
Attempting to make use of these specifications in a layered fashion
would require at least three separate encoders and introduce
unnecessary complexity. Furthermore, there is considerable overlap
between the specifications providing multiple means of achieving the
same ends, all of which must be supported if decoders are to work
reliably.
1.8. Efficiency
Every data format represents a compromise between different concerns,
in particular:
Compactness The space required to record data in the encoding.
Memory Overhead The additional volatile storage (RAM) required to
maintain indexes etc. to support efficient retrieval operations.
Number of Operations The number of operations required to retrieve
data from or append data to an existing encoded sequence.
Number of Disk Seek Operations Optimizing the response time of
magnetic storage media to random access read requests has
traditionally been one of the central concerns of database design.
The DARE Container format is designed to the assumption that this
will cease to be a concern as solid state media replaces magnetic.
While the cost of storage of all types has declined rapidly over the
past decades, so has the amount of data to be stored. DARE Container
represents a pragmatic balance of these considerations for current
technology. In particular, since payload volumes are likely to be
very large, memory and operational efficiency are considered higher
priorities than compactness.
2. Definitions
2.1. Related Specifications
DARE Container makes use of the following related standards and
specifications.
Hallam-Baker Expires February 28, 2019 [Page 7]
Internet-Draft DARE Data Container August 2018
Encoding Content frame headers are encoded using JavaScript Object
Notation (JSON) [RFC7159] , JSON-B or JSON-C
[draft-hallambaker-jsonbcd] .
Cryptography The encryption and signature schemes used are based on
JSON Web Signature [RFC7515] and JSON Web Encryption [RFC7516] as
applied in DARE Message [draft-hallambaker-dare-message] .
2.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119] .
3. Container Navigation
Three means of locating frames in a container are supported:
Sequential Access frames sequentially starting from the start or the
end of the container.
Binary search Access any container frame by frame number in
O(log2(n)) time by means of a binary tree constructed while the
container is written.
Index Access and container frame by frame number or by key by means
of an index record.
All DARE Containers support sequential access. Only tree and Merkle
tree containers support binary search access. An index frame MAY be
written appended to any container and provides O(1) access to any
frame listed in the index.
Two modes of compilation are considered:
Monolithic Frames are added to the container in a single operation,
e.g. file archives,
Incremental Additional frames are written to the container at
various intervals after it was originally created, e.g. server
logs, message spools.
In the monolithic mode, navigation requirements are best met by
writing an index frame to the end of the container when it is
complete. It is not necessary to construct a binary search tree
unless a Merkle tree integrity check is required.
Hallam-Baker Expires February 28, 2019 [Page 8]
Internet-Draft DARE Data Container August 2018
In the incremental mode, Binary search provides an efficient means of
locating frames by frame number but not by key. Writing a complete
index to the container every m write operations provides O(m) search
access but requires O(n2) storage.
Use of partial indexes provides a better compromise between speed and
efficiency. A partial index is written out every m frames where m is
a power of two. A complete index is written every time a binary tree
apex record is written. This approach provides for O(log2(n)) search
with incremental compilation with approximately double the overhead
of the monolithic case.
3.1. Tree
Binary search is supported by means of the TreePosition parameter
specified in the FrameHeader. This parameter specifies the value of
the immediately preceding apex.
Calculation of the immediately preceding apex is most easily
described by representing the array index in binary with base of 1
(rather than 0). An array index that is a power of 2 (2, 4, 8, 16,
etc.) will be the apex of a complete tree. Every other array index
has the value of the sum of a set of powers of 2 and the immediately
preceding apex will be the value of the next smallest power of 2 in
the sum.
For example, to find the immediately preceding apex for frame 5, we
add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame
is 4.
The values of Tree Position are shown for the first 8 frames in
figure xx below:
[[This figure is not viewable in this format. The figure is
available at http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html [3].]]
Merkle Tree Integrity check
An algorithm for efficiently calculating the immediately preceding
apex is provided in Appendix C.
3.2. Position Index
Contains a table of frame number, position pairs pointing to prior
locations in the file.
Hallam-Baker Expires February 28, 2019 [Page 9]
Internet-Draft DARE Data Container August 2018
3.3. Metadata Index
Contains a list of IndexMeta entries. Each entry contains a metadata
description and a list of frame indexes (not positions) of frames
that match the description.
4. Integrity Mechanisms
Frame sequences in a DARE container MAY be protected against a frame
insertion attack by means of a digest chain, a binary Merkle tree or
both.
4.1. Digest Chain calculation
A digest chain is simple to implement but can only be verified if the
full chain of values is known. Appending a frame to the chain has
O(1) complexity but verification has O(n) complexity:
[[This figure is not viewable in this format. The figure is
available at http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html [4].]]
Hash chain integrity check
The value of the chain digest for the first frame (frame 0) is
H(H(null)+H(Payload0)), where null is a zero length octet sequence
and payloadn is the sequence of payload data bytes for frame n
The value of the chain digest for frame n is H(H(Payloadn-1
+H(Payloadn)), where A+B stands for concatenation of the byte
sequences A and B.
4.2. Binary Merkle tree calculation
The tree index mechanism describe earlier may be used to implement a
binary Merkle tree. The value TreeDigest specifies the apex value of
the tree for that node.
Appending a frame to the chain has O(log2n) complexity provided that
the container format supports at least the binary tree index.
Verifying a chain has O(log2 n) complexity, provided that the set of
necessary digest inputs is known.
To calculate the value of the tree digest for a node, we first
calculate the values of all the sub trees that have their apex at
that node and then calculate the digest of that value and the
immediately preceding local apex.
Hallam-Baker Expires February 28, 2019 [Page 10]
Internet-Draft DARE Data Container August 2018
5. Reference
TBS stuff
5.1. Container Headers
TBS stuff
5.1.1. Structure: ContainerHeaderFirst
Inherits: ContainerHeader
Inherits: ContainerHeader
DataEncoding: String (Optional) Specifies the data encoding for the
header section of for the following frames. This value is ONLY
valid in Frame 0 which MUST have a header encoded in JSON.
5.1.2. Structure: ContainerHeader
Inherits: DAREHeader
Describes a container header. A container header MAY contain any
DARE Message header.
Index: Integer (Optional) The record index within the file. This
MUST be unique and satisfy any additional requirements determined
by the ContainerType.
ContainerType: String (Optional) Specifies the container type for
the following records.
IsMeta: Boolean (Optional) If true, the current frame is a meta
frame and does not contain a payload.
Note: Meta frames MAY be present in any container. Applications
MUST accept containers that contain meta frames at any position in
the file. Applications MUST NOT interpret a meta frame as a data
frame with an enpty payload.
UniqueID: String (Optional) Unique object identifier
ContentMeta: ContentMeta (Optional) Content meta data.
TreePosition: Integer (Optional) Position of the frame containing
the apex of the preceding sub-tree.
Hallam-Baker Expires February 28, 2019 [Page 11]
Internet-Draft DARE Data Container August 2018
IndexPosition: Integer (Optional) Specifies the position in the file
at which the last index entry is to be found
ExchangePosition: Integer (Optional) Specifies the position in the
file at which the key exchange data is to be found
ContainerIndex: ContainerIndex (Optional) An index of records in the
current container up to but not including this one.
PayloadDigest: Binary (Optional) If present, contains the digest of
the Payload.
ChainDigest: Binary (Optional) If present, contains the digest of
the PayloadDigest values of this frame and the frame immediately
preceding.
TreeDigest: Binary (Optional) If present, contains the Binary Merkle
Tree digest value.
Event: String (Optional) Unique object identifier
Labels: String [0..Many] List of labels that are applied to the
payload of the frame.
KeyValues: KeyValue [0..Many] List of key/value pairs describing the
payload of the frame.
First: Integer (Optional) Frame number of the first object instance
value.
Previous: Integer (Optional) Frame number of the immediately prior
object instance value
5.2. Content Metadata Structure
TBS stuff
5.2.1. Structure: ContentMeta
Information describing the object instance
ContentType: String (Optional) The content type field as specified
in JWE
Paths: String [0..Many] List of filename paths for the payload of
the frame.
UniqueID: String (Optional) Unique object identifier
Hallam-Baker Expires February 28, 2019 [Page 12]
Internet-Draft DARE Data Container August 2018
Created: DateTime (Optional) Initial creation date.
Modified: DateTime (Optional) Date of last modification.
5.3. Index Structures
TBS stuff
5.3.1. Structure: ContainerIndex
A container index
Full: Boolean (Optional) If true, the index is complete and contains
position entries for all the frames in the file. If absent or
false, the index is incremental and only contains position entries
for records added since the last frame containing a
ContainerIndex.
Positions: IndexPosition [0..Many] List of container position
entries
Metas: IndexMeta [0..Many] List of container position entries
5.3.2. Structure: IndexPosition
Specifies the position in a file at which a specified record index is
found
Index: Integer (Optional) The record index within the file.
Position: Integer (Optional) The record position within the file
relative to the index base.
5.3.3. Structure: KeyValue
Specifies a key/value entry
Key: String (Optional) The key
Value: String (Optional) The value corresponding to the key
5.3.4. Structure: IndexMeta
Specifies the list of index entries at which a record with the
specified metadata occurrs.
Index: Integer [0..Many] List of record indicies within the file
where frames matching the specified criteria are found.
Hallam-Baker Expires February 28, 2019 [Page 13]
Internet-Draft DARE Data Container August 2018
ContentType: String (Optional) Content type parameter
Paths: String [0..Many] List of filename paths for the current
frame.
Labels: String [0..Many] List of labels that are applied to the
current frame.
5.4. Signature
Payload data MAY be signed using a JWS [RFC7515] as applied in the
DARE Message format [draft-hallambaker-dare-message] .
Signatures are specified by the Signatures parameter in the content
header. The data that the signature is calculated over is defined by
the typ parameter of the Signature as follows.
Payload The value of the PayloadDigest parameter
Chain The value of the ChainDigest parameter
Tree The value of the TreeDigestFinal parameter
If the typ parameter is absent, the value Payload is implied.
A frame MAY contain multiple signatures created with the same signing
key and different typ values.
The use of signatures over chain and tree digest values permit
multiple frames to be validated using a single signature verification
operation.
6. Security Considerations
7. IANA Considerations
8. Acknowledgements
9. Appendix A: Examples and Test Vectors
The data payloads in all the following examples are identical, only
the authentication and/or encryption is different.
o Frame 1..n consists of 300 bytes being the byte sequence 00, 01,
02, etc. repeating after 256 bytes.
Hallam-Baker Expires February 28, 2019 [Page 14]
Internet-Draft DARE Data Container August 2018
For conciseness, the raw data format is omitted for examples after
the first, except where the data payload has been transformed, (i.e.
encrypted).
9.1. Simple container
the following example shows a simple container with first frame and a
single data frame:
f4 5d
f0 59
f0 00
5d f4
f5 01 40
f0 0f
f1 01 2c
40 01 f5
Figure 1
Since there is no integrity check, there is no need for trailer
entries. The header values are:
Frame 0
{
"Index":0,
"ContainerType":"List",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 2
Frame 1
{
"Index":1}
[Empty trailer]
Figure 3
Hallam-Baker Expires February 28, 2019 [Page 15]
Internet-Draft DARE Data Container August 2018
9.2. Payload and chain digests
The following example shows a chain container with a first frame and
three data frames. The headers of these frames is the same as before
but the frames now have trailers specifying the PayloadDigest and
ChainDigest values:
Frame 0
{
"Index":0,
"ContainerType":"Chain",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 4
Frame 1
{
"Index":1}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 5
Frame 2
{
"Index":2}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 6
Frame 3
Hallam-Baker Expires February 28, 2019 [Page 16]
Internet-Draft DARE Data Container August 2018
{
"Index":3}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 7
9.3. Merkle Tree
The following example shows a chain container with a first frame and
six data frames. The trailers now contain the TreePosition and
TreeDigest values:
Frame 0
{
"Index":0,
"ContainerType":"Merkle",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 8
Frame 1
{
"Index":1,
"TreePosition":0}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 9
Frame 2
Hallam-Baker Expires February 28, 2019 [Page 17]
Internet-Draft DARE Data Container August 2018
{
"Index":2,
"TreePosition":319}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ
QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}
Figure 10
Frame 3
{
"Index":3,
"TreePosition":319}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 11
Frame 4
{
"Index":4,
"TreePosition":1451}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvKSj8
gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"}
Figure 12
Frame 5
Hallam-Baker Expires February 28, 2019 [Page 18]
Internet-Draft DARE Data Container August 2018
{
"Index":5,
"TreePosition":1451}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 13
Frame 6
{
"Index":6,
"TreePosition":2586}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9SZ0m
dr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"}
Figure 14
9.4. Signed container
The following example shows a tree container with a signature in the
final record. The signing key parameters are:
Hallam-Baker Expires February 28, 2019 [Page 19]
Internet-Draft DARE Data Container August 2018
{
"PrivateKeyRSA":{
"kid":"MBWNO-2J43U-ESWKW-XQWL6-6YGEW-UOPWU-A",
"n":"1NzbmakMalVH1mRv7TEDEhwXDNojn5wZbq1tv1gp5PgZwzX-klYXuFhj0-
MpO0zcwptsUaYJhwdvvgW_1udUpISQYluXOB3UMj2e_0yl64MvnqTL47SZQuAN3QQ
9cuCw_-_Eyj_jerspauqa6RpNzGcabZrtRl7J7DPVZ3SNlw-H_Wxd4HkrFVW_Yqup
htNL1JciQJYm2DSu9dbetqPZ80x6IBargY850mBYOzvNNE5S-dRJQHoJY4SG-ESYF
BuAHtBlOMgbI0XNiq96EegA-vPW9XRF-SHdX5mkafefDGK4rT_RoE4gRwhDM3jbZ8
1-F2ZA_GpNVEvB-25_vF96lQ",
"e":"AQAB",
"d":"k_v_h7Jo-TvUt44X6jSax-pTdBHrljk1zSYxGEe4yIBbmMVe-Gl2ECkTLe
nNbnafO4RGJ_Vgxkk7PEZO-p7Uz5OBtX-rf83tCgihEyg8aaFIZ-h1_xY9Pqr5uGA
MQGNJaoVMsLb99QNNZhE4JTquP56mVvDQaI3Zn6bhhA0ZqpxS_x6iRUV5KnHCRd47
DKGHcAsQR_caxGec7M_XNPqpl0tcoGQz46-I0SVVcqtjb_YysEVez4eJhb_ZTU4C7
pz4wXDj6B0ppFJVZEMaIhKo8FCnoQdXEJl4vSiBzUFrSlPc6gjQk2CBhxc_kb782w
VwW5wtgOVxhV1k2piQ2NrlOQ",
"p":"6osXfrJLiPfK0ky17vqzRs-M5mOxZU2LEFGyHTXxx6EYTWixEsx2Sdf6kx
UD5K_QdYnqqd3yobbGdDpMwwEuwgogWVCz90nc9QdCUy4MCpz8lpOQd1i_tXMmtOg
GBu9mapMTEOwc7HlLlSDRezv_TzTAH4izv-CUEZ_M5EcwyFM",
"q":"6FYD6NV4rKU1ACtGQyIvwGrkrS_F6phB-whx0nSVFkbkwppJiPnC6XqjZ3
OYPCZylxaTHFnxCs0nntrraeNEcWfNPrpTN5XIZjbOiIaKA2iCQkWlDoi8oduEtTK
oIcuy32oBz6MpUeCWzl0ZQ4EeTF3RyCc_jpf8oRvZ0e3ItHc",
"dp":"hfjbe9BmWx-HqCaPSanEW-9UQYmym_X2OGUiA5N7vxci5ZymgOFvs_B9v
iQj7C4NOgaEl3EjFgJsS5m9nSoAxm-4WKxDkD6NyxzRYugLkshnc69otvNn1kKnWn
CqeK2o57mJC4KDZwRGCzIK1oTH6jtsfta8Lh8fFQ4doEuV7uc",
"dq":"r6R_ViE0Foja1aLhflU09mmZMViBbkXm86nBqtHZ97pmrLvJRdVTxgCh0
c6w0yBZ1uEJHBDeykSoZE6qVCWtE3Le1kI0MTx6ANQENXBInCUA_Kr8Ck3TFSYIYJ
fIRaxiMMZKUjfOQAji2WXGeKL_TcpLkt4hDWLXaNDOTgdOiSc",
"qi":"DfHtLB1Ox1Kgp3E4jqy5Qxeb7-v7_uv8n_5-E1OQ3NLSRV2m_auojkR19
nY3gokHKNSXM41qKlJLU00lROjOO2KUq57s8GZkheVfbJLNCJ6KAw_aRT2IgyJm2b
e2v5OCHSkm88tgJWbtKj-OPKTFV5gOMVdeCzGX286ErjDHGCM"}}
Figure 15
The container headers and trailers are:
Frame 0
{
"Index":0,
"ContainerType":"Merkle",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 16
Hallam-Baker Expires February 28, 2019 [Page 20]
Internet-Draft DARE Data Container August 2018
Frame 1
{
"Index":1,
"TreePosition":0}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Figure 17
Frame 2
{
"dig":"S512",
"Index":2,
"TreePosition":319}
{
"PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ
QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}
Figure 18
9.5. Encrypted container
The following example shows a container in which all the frame
payloads are encrypted under the same master secret established in a
key agreement specified in the first frame.
Frame 0
Hallam-Baker Expires February 28, 2019 [Page 21]
Internet-Draft DARE Data Container August 2018
{
"enc":"A256CBC",
"Salt":"Th7V_b2--KdV5rETQZbHsw",
"recipients":[{
"kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
"epk":{
"PublicKeyDH":{
"kid":"MANS7-VSJ2X-75NSK-4SEOM-FRITG-6Q7EO-A",
"Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
"Public":"MYw__h5Ii4honclOGFKY42KdvJed5T47CZI3QTRSBXUPiOD
0D0W49YIyf-m30K-BvMWVzNsh9qAQFpK53A5UJO_pQmfbmrZLHq3AjdYoRlc44YOn
usUokZN43xtGly4m7Rcf_42M3cofa80UWejL3mz3zQKTojxwXz3cykLA2tsTjHdKE
O0Q6jWef952YkPQNKJmR-42aX7Pfz5kZt1QtgfUV4QYk_b0NFfHSNk9CAs6Klav5K
qGjxL4mfIxvmrksyIR5solCo2BMbh_AOdyZNuEa4nURvS9YtWrXZRjQtMZPKB8MXG
_mSAxUco3U1em2hmdkgIkTveF7-da6otILw"}},
"wmk":"0HfYk15gAvxngunLoHymHSunrb0KlGjGuTERlpSxB5JLpFJj6KJZXA"}
],
"Index":0,
"ContainerType":"List",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 19
Frame 1
{
"enc":"A256CBC",
"Salt":"BmY8ZYeD8Kg1snIrBIAUyA",
"Index":1}
[Empty trailer]
Figure 20
Frame 2
{
"enc":"A256CBC",
"Salt":"pA-kigxfhhsGC5lM9xrAZQ",
"Index":2}
[Empty trailer]
Figure 21
Hallam-Baker Expires February 28, 2019 [Page 22]
Internet-Draft DARE Data Container August 2018
Here are the container bytes. Note that the content is now encrypted
and has expanded by 25 bytes. These are the salt (16 bytes), the AES
padding (4 bytes) and the JSON-B framing (5 bytes).
f5 03 3c
f1 03 27
f0 10
3c 03 f5
f5 01 7c
f0 47
f1 01 30
7c 01 f5
f5 01 7c
f0 47
f1 01 30
7c 01 f5
Figure 22
The following example shows a container in which all the frame
payloads are encrypted under separate key agreements specified in the
payload frames.
Frame 0
{
"Index":0,
"ContainerType":"List",
"ContentMeta":{},
"DataEncoding":"JSON"}
[Empty trailer]
Figure 23
Frame 1
Hallam-Baker Expires February 28, 2019 [Page 23]
Internet-Draft DARE Data Container August 2018
{
"enc":"A256CBC",
"Salt":"LUAkC3V-ztXJfF7OS-RNwQ",
"recipients":[{
"kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
"epk":{
"PublicKeyDH":{
"kid":"MCYIZ-PH7YZ-XAY6Q-2EMQR-GMRXG-66YPK-A",
"Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
"Public":"4cDrUAlsZpyGZGhgVbLlunEyl7O9a8ku-6IGVZaSqyOmsb5
rCt_pjnN49fjy7hqsS364S4eUDRj7e3vfIBEav-zeynaiJB6lY6eJBMmlaEuomyyy
01Srq5eaX6i0A8CRWuuRK8IJAHQ31Q89Bah5XayMxc3bqPxciOL1gqU75w65fKmY9
4mRdAIw2Yk-PTcQVztVDk2ooGJuvLENuIk29g-leFbMmCAXND9hhDErVupBND3Ljh
BXylTvfIx-mrw0SMPO_IsVLbhrXQ6lTMygucttT3GPf25lgWBaEdiHVuPqmTBtagB
Vgq9-gHySYtuCNrK7RfB5rtz6CUmENEwx1QA"}},
"wmk":"25yqAeqIG1S6BrCSQxU_M8WKB4t8J_lHWI-68DNYORTUoCpaitosig"}
],
"Index":1}
[Empty trailer]
Figure 24
Frame 2
{
"enc":"A256CBC",
"Salt":"EVrEz8ai_v9s9hZ0ZXMB1Q",
"recipients":[{
"kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
"epk":{
"PublicKeyDH":{
"kid":"MB43P-C7BVX-SROQ6-Z5LW5-5DLRO-PS675-A",
"Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
"Public":"mJRyK45fAf2ksxW7M8W961hCKI1g7dQqn9rA2K6vdSh1o4v
gLqjHDzLYqr9yQKteGUfl0PLIuHAOFq3p_oJDhaioJKX6Akv-y-dcfDdi2WgOOr_k
R9NPtl-nmsbJ8fx6v1QdXLzdoGZjsj_t6wyIo5w287p7DylFdYkhriqVv7tgBELZI
53Pur4F5QbK-qJYeSwY3fAksQYrviMZbFVwic8v5QQNoAGcoGguBOOz-aljFuo1g9
0o7pNT-Soj3DAPfmXS1hmPR-hpqyzELAat-q_O22P4d_NtUOmaLwZXSZVYUBLE8Fj
1okt4_8zGfCsGpsHl9rwqlyoVIZ4VadldWw"}},
"wmk":"P6UjESsvq5bATPHc5vPfEcs5u1pCkLVBCv5_aYdHdwQG3rpPihHj2g"}
],
"Index":2}
[Empty trailer]
Figure 25
Hallam-Baker Expires February 28, 2019 [Page 24]
Internet-Draft DARE Data Container August 2018
10. Appendix B
public long PreviousFrame (long Frame) {
long x2 = Frame + 1;
long d = 1;
while (x2 > 0) {
if ((x2 & 1) == 1) {
return x2 == 1 ? (d / 2) - 1 : Frame - d;
}
d = d * 2;
x2 = x2 / 2;
}
return 0;
}
Figure 26
11. References
11.1. Normative References
[draft-hallambaker-dare-message]
Hallam-Baker, P., "Data At Rest Encryption Part 1: DARE
Message", draft-hallambaker-dare-message-01 (work in
progress), July 2018.
[draft-hallambaker-jsonbcd]
Hallam-Baker, P., "Binary Encodings for JavaScript Object
Notation: JSON-B, JSON-C, JSON-D", draft-hallambaker-
jsonbcd-13 (work in progress), July 2018.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997.
[RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data
Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March
2014.
[RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web
Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May
2015.
[RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)",
RFC 7516, DOI 10.17487/RFC7516, May 2015.
Hallam-Baker Expires February 28, 2019 [Page 25]
Internet-Draft DARE Data Container August 2018
11.2. Informative References
[BLOCKCHAIN]
Chain.com, "Blockchain Specification".
[draft-hallambaker-mesh-architecture]
Hallam-Baker, P., "Mathematical Mesh: Architecture",
draft-hallambaker-mesh-architecture-04 (work in progress),
September 2017.
[RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70,
RFC 5652, DOI 10.17487/RFC5652, September 2009.
[ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format
Specification", October 2014.
11.3. URIs
[1] http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html
[2] http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html
[3] http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html
[4] http://mathmesh.com/Documents/draft-hallambaker-dare-
container.html
Author's Address
Phillip Hallam-Baker
Comodo Group Inc.
Email: philliph@comodo.com
Hallam-Baker Expires February 28, 2019 [Page 26]
