| Network Working Group | P. Hallam-Baker |
| Internet-Draft | April 4, 2019 |
| Intended status: Informational | |
| Expires: October 6, 2019 |
Mathematical Mesh Part III : Data At Rest Encryption (DARE)
draft-hallambaker-mesh-dare-01
This document describes the Data At Rest Encryption (DARE) Message and Container syntax.
The DARE Message syntax is used to digitally sign, digest, authenticate, or encrypt arbitrary message content.
The DARE Container syntax describes an append-only sequence of data frames, each containing a DARE Message. DARE Containers may support cryptographic integrity verification of the entire data container content by means of a Merkle tree.
This document is also available online at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html .
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on October 6, 2019.
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
This document describes the Data At Rest Encryption (DARE) Message and Container Syntax. The DARE Message syntax is used to digitally sign, digest, authenticate, or encrypt arbitrary message content. The DARE Container syntax describes an append-only sequence of data frames, each containing a DARE Message that supports efficient incremental signature and encryption.
The DARE Message Syntax is based on a subset of the JSON Web Signature [RFC7515] and JSON Web Encryption [RFC7516] standards and shares many fields and semantics. The processing model and data structures have been streamlined to remove alternative means of specifying the same content and to enable multiple data sequences to be signed and encrypted under a single master encryption key without compromise to security.
A DARE Message consists of a Header, Payload and an optional Trailer. To enable single pass encoding and decoding, the Header contains all the information required to perform cryptographic processing of the Payload and authentication data (digest, MAC, signature values) MAY be deferred to the Trailer section.
A DARE Container is an append-only log format consisting of a sequence of frames. Cryptographic enhancements (signature, encryption) may be applied to individual frames or to sets of frames. Thus, a single key exchange may be used to provide a master key to encrypt multiple frames and a single signature may be used to authenticate all the frames in the container up to and including the frame in which the signature is presented.
The DARE Message syntax may be used either as a standalone cryptographic message syntax or as a means of presenting a single DARE Container frame together with the complete cryptographic context required to verify the contents and decrypt them.
An important innovation in the DARE Message Syntax is the separation of key exchange and data encryption operations so that a Master Key (MK) established in a single exchange to be applied to multiple data sequences. This means that a single public key operation MAY be used to encrypt and/or authenticate multiple parts of the same DARE Message or multiple frames in a DARE Container.
To avoid reuse of the key and to avoid the need to communicate separate IVs, each octet sequence is encrypted under a different encryption key (and IV if required) derived from the Master Key by means of a salt that is unique for each octet sequence that is encrypted. The same approach is used to generate keys for calculating a MAC over the octet sequence if required. This approach allows encryption and integrity protections to be applied to the message payload, to header or trailer fields or to application defined Enhanced Data Sequences in the header or trailer.
Traditional cryptographic containers describe the application of a single key exchange to encryption of a single octet sequence. Examples include PCKS#7/CMS [RFC2315] , OpenPGP [RFC4880] and JSON Web Encryption [RFC7516] .
To encrypt a message using RSA, the encoder first generates a random encryption key and initialization vector (IV). The encryption key is encrypted under the public key of each recipient to create a per-recipient decryption entry. The encryption key, plaintext and IV are used to generate the ciphertext (figure 1).
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
Monolithic Key Exchange and Encrypt
This approach is adequate for the task of encrypting a single octet stream. It is less than satisfactory when encrypting multiple octet streams or very long streams for which a rekeying operation is desirable.
In the DARE approach, key exchange and key derivation are separate operations and keys MAY be derived for encryption or integrity purposes or both. A single key exchange MAY be used to derive keys to apply encryption and integrity enhancements to multiple data sequences.
The DARE key exchange begins with the same key exchange used to produce the CEK in JWE but instead of using the CEK to encipher data directly, it is used as one of the inputs to a Key Derivation Function (KDF) that is used to derive parameters for each block of data to be encrypted. To avoid the need to introduce additional terminology, the term 'CEK' is still used to describe the output of the key agreement algorithm (including key unwrapping if required) but it is more appropriately described as a Master Key (figure 2).
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
Exchange of Master Key
A Master Key may be used to encrypt any number of data items. Each data item is encrypted under a different encryption key and IV (if required). This data is derived from the Master Key using the HKDF function [RFC5869] using a different salt for each data item and separate info tags for each cryptographic function (figure 3).
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
Data item encryption under Master Key and per-item salt.
This approach to encryption offers considerably greater flexibility allowing the same format for data item encryption to be applied at the transport, message or field level.
Each encrypted DARE Message specifies a unique Master Salt value of at least 128 bits which is used to derive the salt values used to derive cryptographic keys for the message payload and annotations.
Erasure of the Master Salt value MAY be used to effectively render the message payload and annotations undecipherable without altering the message payload data. The work factor for decryption will be O(2^128) even if the decryption key is compromised.
As with encryption, DARE Message signatures MAY be applied to an individual message or a sequence of messages.
When an individual plaintext message is signed, the digest value used to create the signature is calculated over the binary value of the payload data. That is, the value of the payload before the encoding (Base-64, JSON-B) is applied.
When an individual plaintext message is signed, the digest value used to create the signature is calculated over the binary value of the payload data. That is, the value of the payload after encryption but before the encoding (Base-64, JSON-B) is applied.
Use of signing and encryption in combination presents the risk of subtle attacks depending on the order in which signing and encryption take place [Davis2001] .
Na?ve approaches in which a message is encrypted and then signed present the possibility of a surreptitious forwarding attack. For example, Alice signs a message and sends it to Mallet who then strips off Alice's signature and sends the message to Bob.
Na?ve approaches in which a message is signed and then encrypted present the possibility of an attacker claiming authorship of a ciphertext. For example, Alice encrypts a ciphertext for Bob and then signs it. Mallet then intercepts the message and sends it to Bob.
While neither attack is a concern in all applications, both attacks pose potential hazards for the unwary and require close inspection of application protocol design to avoid exploitation.
To prevent these attacks, each signature on a message that is signed and encrypted MUST include a witness value that is calculated by applying a MAC function to the signature value as described in section XXX.
To sign multiple messages with a single signature, we first construct a Merkle tree of the message payload digest values and then sign the root of the Merkle tree.
[This is not yet implemented but will be soon.]
DARE Container is a message and file syntax that allows a sequence of data frames to be represented with cryptographic integrity, signature, and encryption enhancements to be constructed in an append only format.
The format is designed to meet the requirements of a wide range of use cases including:
A Container consists of a sequence of variable length Frames. Each frame consists of a forward length indicator, the framed data and a reverse length indicator. The reverse length indicator is written out backwards allowing the length and thus the frame to be read in the reverse direction:
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
JBCD Bidirectional Frame
Each frame contains a single DARE Message consisting of a Header, Payload and Trailer (if required). The first frame in a container describes the container format options and defaults. These include the range of encoding options for frame metadata supported and the container profiles to which the container conforms.
All internal data formats support use of pointers of up to 64 bits allowing containers of up to 18 exabytes to be written.
Five container types are currently specified:
In normal circumstances, Containers are written as an append only log. As with Messages, integrity information (payload digest, signatures) is written to the message trailer. Thus, large payloads may be written without the need to buffer the payload data provided that the content length is known in advance.
Should exceptional circumstances require, Container entries MAY be erased by overwriting the Payload and/or parts of the Header content without compromising the ability to verify other entries in the container. If the entry Payload is encrypted, it is sufficient to erase the container salt value to render the container entry effectively inaccessible (though recovery might still be possible if the original salt value can be recovered from the storage media.
Frame payloads and associated attributes MAY be encrypted and/or authenticated in the same manner as Messages.
Incremental encryption is supported allowing encryption parameters from a single public key exchange operation to be applied to encrypt multiple frames. The public key exchange information is specified in the first encrypted frame and subsequent frames encrypted under those parameters specify the location at which the key exchange information is to be found by means of the ExchangePosition field which MUST specify a location that is earlier in the file.
To avoid cryptographic vulnerabilities resulting from key re-use, the DARE key exchange requires that each encrypted sequence use an encryption key and initialization vector derived from the master key established in the public key exchange by means of a unique salt.
Each Message and by extension, each Container frame MUST specify a unique salt value of at least 128 bits. Since the encryption key is derived from the salt value by means of a Key Derivation Function, erasure of the salt MAY be used as a means of rendering the payload plaintext value inaccessible without changing the payload value.
Signatures MAY be applied to a payload digest, the final digest in a chain or tree. The chain and tree digest modes allow a single signature to be used to authenticate all frame payloads in a container.
The tree signature mode is particularly suited to applications such as file archives as it allows files to be verified individually without requiring the signer to sign each individually. Furthermore, in applications such as code signing, it allows a single signature to be used to verify both the integrity of the code and its membership of the distribution.
As with DARE Message, the signature mechanism does not specify the interpretation of the signature semantics. The presence of a signature demonstrates that the holder of the private key applied it to the specified digest value but not their motive for doing so. Describing such semantics is beyond the scope of this document and is deferred to future work.
The chief disadvantage of using an append-only format is that containers only increase in size. In many applications, much of the data in the container becomes redundant or obsolete and a process analogous to garbage collection is required. This process is called redaction.
The simplest method of redaction is to create a new container and sequentially copy each entry from the old container to the new, discarding redundant frames and obsolete header information.
For example, partial index records may be consolidated into a single index record placed in the last frame of the container. Unnecessary signature and integrity data may be discarded and so on.
While redaction could in principle be effected by moving data in-place in the existing container, supporting this approach in a robust fashion is considerably more complex as it requires backward references in subsequent frames to be overridden as each frame is moved.
Many file proprietary formats are in use that support some or all of these capabilities but only a handful have public, let alone open, standards. DARE Container is designed to provide a superset of the capabilities of existing message and file syntaxes, including:
Attempting to make use of these specifications in a layered fashion would require at least three separate encoders and introduce unnecessary complexity. Furthermore, there is considerable overlap between the specifications providing multiple means of achieving the same ends, all of which must be supported if decoders are to work reliably.
Every data format represents a compromise between different concerns, in particular:
While the cost of storage of all types has declined rapidly over the past decades, so has the amount of data to be stored. DARE Container represents a pragmatic balance of these considerations for current technology. In particular, since payload volumes are likely to be very large, memory and operational efficiency are considered higher priorities than compactness.
The DARE Message and Container formats are based on the following existing standards and specifications.
DARE Container makes use of the following related standards and specifications.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] .
The terms "Authentication Tag", "Content Encryption Key", "Key Management Mode", "Key Encryption", "Direct Key Agreement", "Key Agreement with Key Wrapping" and "Direct Encryption" are defined in the JWE specification [RFC7516] .
The terms "Authentication", "Ciphertext", "Digital Signature", "Encryption", "Initialization Vector (IV)", "Message Authentication Code (MAC)", "Plaintext" and "Salt" are defined by the Internet Security Glossary, Version 2 [RFC4949] .
A DARE Message is a sequence of three parts:
For example, the following sequence is a JSON encoded Message with an empty header, a payload of zero length and an empty trailer:
[ {}, "", {} ]
DARE Messages MAY be encoded using JSON serialization or a binary serialization for greater efficiency.
DARE Message processors MUST support JSON serialization and SHOULD support JSON-B serialization.
The DARE Message Syntax supports single pass encoding and decoding without buffering of data. All the information required to begin processing a DARE message (key agreement information, digest algorithms), is provided in the message header. All the information that is derived from message processing (authentication codes, digest values, signatures) is presented in the message trailer.
The choice of message encoding does not affect the semantics of message processing. A DARE Message MAY be reserialized under the same serialization or converted from any of the specified serialization to any other serialization without changing the semantics or integrity properties of the message.
A header MAY contain header fields describing the payload content. These include:
For example, consider the following mail message:
From: Alice@example.com To: bob@example.com Subject: TOP-SECRET Product Launch Today! The CEO told me the product launch is today. Tell no-one!
Existing encryption approaches require that header fields such as the subject line be encrypted with the body of the message or not encrypted at all. Neither approach is satisfactory. In this example, the subject line gives away important information that the sender probably assumed would be encrypted. But if the subject line is encrypted together with the message body, a mail client must retrieve at least part of the message body to provide a 'folder' view.
The plaintext form of the equivalent DARE Message encoding is:
[{
"cty":"application/example-mail",
"Annotations":["iAEBiBdGcm9tOiBBbGljZUBleGFtcGxlLmNvbYgA",
"iAECiBNUbzogYm9iQGV4YW1wbGUuY29tiAA",
"iAEDiClTdWJqZWN0OiBUT1AtU0VDUkVUIFByb2R1Y3QgTGF1bmNoIFRvZGF5
IYgA"
]},
"VGhlIENFTyB0b2xkIG1lIHRoZSBwcm9kdWN0IGxhdW5jaCBpcyB0b2RheS4gVGVs
bCBuby1vbmUh"
]
This contains the same information as before but the mail message headers are now presented as a list of Encoded Data Sequences.
An encoded data sequence (EDS) is a sequence of octets that encodes a data sequence according to cryptographic enhancements specified in the context in which it is presented. An EDS MAY be encrypted and MAY be authenticated by means of a MAC. The keys and other cryptographic parameters used to apply these enhancements are derived from the cryptographic context and a Salt prefix specified in the EDS itself.
An EDS sequence contains exactly three binary fields encoded in JSON-B serialization as follows:
Requiring all three fields to be present, even in cases where they are unnecessary simplifies processing at the cost of up to six additional data bytes.
The encoding of the 'From' header of the previous example as a plaintext EDS is as follows:
88 01 01 88 17 46 72 6f 6d 3a 20 41 6c 69 63 65 40 65 78 61 6d 70 6c 65 2e 63 6f 6d 88 00
Encryption and integrity protections MAY be applied to any DARE Message Payload and Annotations.
The following is an encrypted version of the message shown earlier. The payload and annotations have both increased in size as a result of the block cipher padding. The header now includes Recipients and Salt fields to enable the content to be decoded.
[{
"enc":"A256CBC",
"Salt":"r1yXNHkhxZnQcAloDzscDg",
"cty":"application/example-mail",
"Annotations":["iAEBiCDY_GflmQT6FRGDzxUxAv0LrHRP_b7jvZvprQWwWPFk
TA",
"iAECiCAsiGoiMi0xFnoZVsMUKyGgTX0E448MKqg36hZZxnyBYg",
"iAEDiDDvGyqYBVJSa_d1c0v5Z4DN9wKWRmdnllyJJecdlsJ4szg95xtLrKoW
xgym6Ngqnqo"
],
"recipients":[{
"kid":"MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk":{
"PublicKeyECDH":{
"crv":"Ed25519",
"Public":"7isN-IhDOvyIlTc8NvH7j3lTQ31z7POV12c2YwtyMPE"}},
"wmk":"DicCqnjnsm7tyTaoo7pyCFCU0zHQ_gOP5cW35nRtpjrm10GGlE64
rg"}
]},
"Vq__pS87wSZnJOaKEDMVojU28yAhAzNv8ddLnjSlgoR1QhCW28NLV7_thL01UegX
2-ud1OwnCMdOXlxkrMrpxg"
]
The DARE key exchange is based on the JWE key exchange except that encryption modes are intentionally limited and the output of the key exchange is the DARE Master Key rather than the Content Encryption Key.
A DARE Key Exchange MAY contain any number of Recipient entries, each providing a means of decrypting the Master Key using a different private key.
If the Key Exchange mechanism supports message recovery, Direct Key Agreement is used, in all other cases, Key Wrapping is used.
This approach allows messages with one intended recipient to be handled in the exact same fashion as messages with multiple recipients. While this does require an additional key wrapping operation, that could be avoided if a message has exactly one intended recipient, this is offset by the reduction in code complexity.
If the key exchange algorithm does not support message recovery (e.g. Diffie Hellman and Elliptic Curve Diffie-Hellman), the HKDF Extract-and-Expand Key Derivation Function is used to derive a master key using the following info tag:
The master key length is the maximum of the key size of the encryption algorithm specified by the key exchange header, the key size of the MAC algorithm specified by the key exchange header (if used) and 256.
The JWE/JWS specifications define a kid field for use as a key identifier but not how the identifier itself is constructed. All DARE key identifiers are either UDF key fingerprints [draft-hallambaker-mesh-udf] or Mesh/Recrypt Group Key Identifiers.
A UDF fingerprint is formed as the digest of an IANA content type and the digested data. A UDF key fingerprint is formed with the content type application/pkix-keyinfo and the digested data is the ASN.1 DER encoded PKIX certificate keyInfo sequence for the corresponding public key.
A Group Key Identifier has the form <fingerprint>@<domain>. Where <fingerprint> is a UDF key fingerprint and <domain> is the DNS address of a service that provides the encryption service to support decryption by group members.
A Master Salt is a sequence of 16 or more octets that is specified in the Salt field of the header.
The Master Salt is used to derive salt values for the message payload and associated encoded data sequences as follows.
Encoders SHOULD NOT generate salt values that exceed 1024 octets.
The salt value is opaque to the DARE encoding but MAY be used to encode application specific semantics including:
For data erasure to be effective, the salt MUST be constructed so that the difficulty of recovering the key is sufficiently high that it is infeasible. For most purposes, a salt with 128 bits of appropriately random data is sufficient.
Encryption and/or authentication keys are derived from the Master Key using a Extract-and-Expand Key Derivation Function as follows:
The application specific information inputs are:
While encryption and integrity enhancements can be applied to any part of a DARE message, signatures are only applied to payload digest values calculated over one or more message payloads.
The payload digest value for a message is calculated over the binary payload data. That is, after any encryption enhancement has been applied but before the message encoding is applied. This allows messages to be converted from one encoding to another without affecting signature verification.
Verification of a multiple payload signature naturally requires the additional digest values required to construct the Merkle Tree. These are provided in the Trailer in a format that permits multiple signers to reference the same tree data.
The key wrapping and derivation algorithms.
Since the means of public key exchange is determined by the key identifier of the recipient key, it is only necessary to specify the algorithms used for key wrapping and derivation.
The default (and so far only) algorithm is kwd-aes-sha2-256-256.
Advanced Encryption Standard (AES) Key Wrap with Padding Algorithm [RFC3394] is used to wrap the Master Exchange Key. AES 256 is used.
HMAC-based Extract-and-Expand Key Derivation Function [RFC5869] is used for key derivation. SHA-2-256 is used for the hash function.
Three means of locating frames in a container are supported:
All DARE Containers support sequential access. Only tree and Merkle tree containers support binary search access. An index frame MAY be written appended to any container and provides O(1) access to any frame listed in the index.
Two modes of compilation are considered:
In the monolithic mode, navigation requirements are best met by writing an index frame to the end of the container when it is complete. It is not necessary to construct a binary search tree unless a Merkle tree integrity check is required.
In the incremental mode, Binary search provides an efficient means of locating frames by frame number but not by key. Writing a complete index to the container every m write operations provides O(m) search access but requires O(n^2) storage.
Use of partial indexes provides a better compromise between speed and efficiency. A partial index is written out every m frames where m is a power of two. A complete index is written every time a binary tree apex record is written. This approach provides for O(log_2(n)) search with incremental compilation with approximately double the overhead of the monolithic case.
Binary search is supported by means of the TreePosition parameter specified in the FrameHeader. This parameter specifies the value of the immediately preceding apex.
Calculation of the immediately preceding apex is most easily described by representing the array index in binary with base of 1 (rather than 0). An array index that is a power of 2 (2, 4, 8, 16, etc.) will be the apex of a complete tree. Every other array index has the value of the sum of a set of powers of 2 and the immediately preceding apex will be the value of the next smallest power of 2 in the sum.
For example, to find the immediately preceding apex for frame 5, we add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame is 4.
The values of Tree Position are shown for the first 8 frames in figure xx below:
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
Merkle Tree Integrity check
An algorithm for efficiently calculating the immediately preceding apex is provided in Appendix C.
Contains a table of frame number, position pairs pointing to prior locations in the file.
Contains a list of IndexMeta entries. Each entry contains a metadata description and a list of frame indexes (not positions) of frames that match the description.
Frame sequences in a DARE container MAY be protected against a frame insertion attack by means of a digest chain, a binary Merkle tree or both.
A digest chain is simple to implement but can only be verified if the full chain of values is known. Appending a frame to the chain has O(1) complexity but verification has O(n) complexity:
[[This figure is not viewable in this format. The figure is available at http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html.]]
Hash chain integrity check
The value of the chain digest for the first frame (frame 0) is H(H(null)+H(Payload_0)), where null is a zero length octet sequence and payloadn is the sequence of payload data bytes for frame n
The value of the chain digest for frame n is H(H(Payload_n-1 +H(Payloadn)), where A+B stands for concatenation of the byte sequences A and B.
The tree index mechanism describe earlier may be used to implement a binary Merkle tree. The value TreeDigest specifies the apex value of the tree for that node.
Appending a frame to the chain has O(log_2 (n)) complexity provided that the container format supports at least the binary tree index. Verifying a chain has O(log_2 (n)) complexity, provided that the set of necessary digest inputs is known.
To calculate the value of the tree digest for a node, we first calculate the values of all the sub trees that have their apex at that node and then calculate the digest of that value and the immediately preceding local apex.
Payload data MAY be signed using a JWS [RFC7515] as applied in the Message.
Signatures are specified by the Signatures parameter in the content header. The data that the signature is calculated over is defined by the typ parameter of the Signature as follows.
If the typ parameter is absent, the value Payload is implied.
A frame MAY contain multiple signatures created with the same signing key and different typ values.
The use of signatures over chain and tree digest values permit multiple frames to be validated using a single signature verification operation.
A DARE Message consists of a Header, an Enhanced Data Sequence (EDS) and an optional trailer. This section describes the JSON data fields used to construct headers, trailers and complete messages.
Wherever possible, fields from JWE, JWS and JWK have been used. In these cases, the fields have the exact same semantics. Note however that the classes in which these fields are presented have different structure and nesting.
A DARE Message contains a single DAREMessageSequence in either the JSON or Compact serialization as directed by the protocol in which it is applied.
A DARE Message containing Header, EDS and Trailer in JSON object encoding. Since a DAREMessage is almost invariably presented in JSON sequence or compact encoding, use of the DAREMessage subclass is preferred.
Although a DARE Message is functionally an object, it is serialized as an ordered sequence. This ensures that the message header field will always precede the body in a serialization, this allowing processing of the header information to be performed before the entire body has been received.
A DARE Message sequence MUST contain a (possibly empty) DAREHeader and MAY contain a DARETrailer.
A DARE Message Trailer
A DARE Message Header. Since any field that is present in a trailer MAY be placed in a header instead, the message header inherits from the trailer.
DARE Message uses the same fields as JWE and JWS but with different structure. In particular, DARE messages MAY have multiple recipients and multiple signers.
The signature value
The signature value
Recipient information
TBS stuff
TBS stuff
Describes a container header. A container header MAY contain any DARE Message header.
TBS stuff
Information describing the object instance
TBS stuff
A container index
Specifies the position in a file at which a specified record index is found
Specifies a key/value entry
Specifies the list of index entries at which a record with the specified metadata occurrs.
This section describes security considerations arising from the use of DaRE in general applications.
Additional security considerations for use of DaRE in Mesh services and applications are described in the Mesh Security Considerations guide [draft-hallambaker-mesh-security] .
In the following examples, Alice's encryption private key parameters are:
{
"PrivateKeyECDH":{
"crv":"Ed25519",
"Private":"eutl5W-yj45k4ME_mh2SbR3E5AN61tgDbfmF-dJmTVo"}}
Alice's signature private key parameters are:
{
"PrivateKeyECDH":{
"crv":"Ed25519",
"Private":"Bw6qDvg3D8IunEgMDBoDHFc1X-wnd577-PiXUR9RfFU"}}
The body of the test message is the UTF8 representation of the following string:
"This is a test long enough to require multiple blocks"
The EDS sequences, are the UTF8 representation of the following strings:
"Subject: Message metadata should be encrypted" "2018-02-01"
A plaintext message without associated EDS sequences is an empty header followed by the message body:
{
"DareMessage":[{},
"VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS
BibG9ja3M"
]}
If a plaintext message contains EDS sequences, these are also in plaintext:
{
"DareMessage":[{
"Annotations":["iAEBiC1TdWJqZWN0OiBNZXNzYWdlIG1ldGFkYXRhIHNob3
VsZCBiZSBlbmNyeXB0ZWSIAA",
"iAECiAoyMDE4LTAyLTAxiAA"
]},
"VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS
BibG9ja3M"
]}
The creator generates a master session key:
78 52 E2 3D 76 A6 54 63 3A B3 8A C9 76 C5 64 29 54 56 8C A0 2B F3 40 6A 3B D3 F4 B3 B7 58 80 1F
For each recipient of the message:
The creator generates an ephemeral key:
{
"PrivateKeyECDH":{
"crv":"Ed25519",
"Private":"rDiO3m5PEKiuBDdYwvvHJJXqXTU6md8_FpR0HevDolQ"}}
The key agreement value is calculated:
38 95 F8 24 4B C0 71 F7 2C 80 2D 9C 27 E2 E0 81 20 9B 40 03 3D 74 90 E3 60 6E E1 28 E3 B0 63 0E
The key agreement value is used as the input to a HKDF key derivation function with the info parameter master to create the key used to wrap the master key:
B1 AC 17 1F 82 51 52 E5 73 32 6E 2B C3 9F 82 3F CE 20 BF A5 19 02 DF EB D3 19 3D 6D 39 91 F3 E2
The wrapped master key is:
02 FC 2C 53 E0 85 F1 40 20 98 11 1D 7F FC FE 63 B1 26 D8 B4 91 51 7A 19 B8 4E 92 D7 FC A3 2A C1 BA EE 2C 6C A2 22 B8 A6
This information is used to calculate the Recipient information shown in the example below.
To encrypt a message, we first generate a unique salt value:
34 2F 8F 81 84 94 86 A7 F3 8A 63 4D 95 FC 9F 65
The salt value and master key are used to generate the payload encryption key:
8C 2F 2C B7 86 C4 1B 38 72 FF CF 2F 8C 4B 02 B3 9B 68 81 95 C1 D8 04 CC C5 CA 9F EC 28 42 20 35
Since AES is a block cipher, we also require an initializarion vector:
92 2B 73 F5 A9 C4 AD 7E 68 44 4F 51 AC 13 AD 79
The output sequence is the encrypted bytes:
7A F1 D3 73 BB 98 8E EA 54 F5 D6 6F 45 1F 45 F9 DF AE C8 22 9F 24 01 24 52 C3 26 38 87 11 1A F1 20 28 54 9E F4 50 C4 EC D9 11 C7 8C 5A 15 42 A4 71 72 F4 C8 47 04 32 5D 9E F7 4F 65 A1 C6 DE A8
Since the message is not signed, there is no need for a trailer. The completed message is:
{
"DareMessage":[{
"enc":"A256CBC",
"Salt":"NC-PgYSUhqfzimNNlfyfZQ",
"recipients":[{
"kid":"MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk":{
"PublicKeyECDH":{
"crv":"Ed25519",
"Public":"qiPa2xdwOl61qs5yUQooA5vHE4C6yA9BqGzSgdF36vI"}},
"wmk":"AvwsU-CF8UAgmBEdf_z-Y7Em2LSRUXoZuE6S1_yjKsG67ixsoi
K4pg"}
]},
"evHTc7uYjupU9dZvRR9F-d-uyCKfJAEkUsMmOIcRGvEgKFSe9FDE7NkRx4xaFU
KkcXL0yEcEMl2e909locbeqA"
]}
Signed messages specify the digest algorithm to be used in the header and the signature value in the trailer. Note that the digest algorithm is not optional since it serves as notice that a decoder should digest the payload value to enable signature verification.
{
"DareMessage":[{
"dig":"S512"},
"VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS
BibG9ja3M",
{
"signatures":[{
"signature":"6WT83zSqqsXlCg5qV2lcQKWvjaqfGn40iuMfCMJj3W3u
p-suh93GUmnJRU3G4N7rrfdSszwqNF3QXGcB6TFyCQ"}
],
"PayloadDigest":"raim8SV5adPbWWn8FMM4mrRAQCO9A2jZ0NZAnFXWlG0x
F6sWGJbnKSdtIJMmMU_hjarlIPEoY3vy9UdVlH5KAg"}
]}
A signed and encrypted message is encrypted and then signed. The signer proves knowledge of the payload plaintext by providing the plaintext witness value.
{
"DareMessage":[{
"enc":"A256CBC",
"dig":"S512",
"Salt":"sBoqFymhJw40xB5g3ojv1g",
"recipients":[{
"kid":"MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk":{
"PublicKeyECDH":{
"crv":"Ed25519",
"Public":"S8VmzAf5zVrRwSYMicKHFKTycHdieNTMjmLNABO9vto"}},
"wmk":"pIfhfZ1wZhJ9ZvRnJQlFFDUHilpYyllfHhPzwQpO5Ttzkd4r5x
Q9yg"}
]},
"eR6QEDzxlgive00ysoeXt9YhmfAYCCx7rYKNuuM3Tr7GR7NcqRvnzz_ZRlCPds
hdA9AgEIjveTLPjfPWqgMGXQ",
{
"signatures":[{
"signature":"WdlXU4CZx5Lqg7BWwBdec_zrRyNj1ozh1s65akpo5QbZ
cHQCMGCfMR3yqp-_GCfB4BIlNTp8Dy3AeftFVIrMBw",
"witness":"H4ZdcRAn3N5HptphfHgyGKi80qb6GqoIv0LFX5IMy-s"}
],
"PayloadDigest":"fT5sn-m-AzCCOYpqWf4oPZQJ2Sg6Tb_mHAT5SIMu1dqX
o0G4RWWavoc7gCWSunW9B4Gm9x1aVhclvQ82vMOhUg"}
]}
The data payloads in all the following examples are identical, only the authentication and/or encryption is different.
For conciseness, the raw data format is omitted for examples after the first, except where the data payload has been transformed, (i.e. encrypted).
the following example shows a simple container with first frame and a single data frame:
f4 5d f0 59 f0 00 5d f4 f5 01 40 f0 0f f1 01 2c 40 01 f5
Since there is no integrity check, there is no need for trailer entries. The header values are:
Frame 0
{
"Index": 0,
"ContainerType": "List",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"Index": 1}
[Empty trailer]
The following example shows a chain container with a first frame and three data frames. The headers of these frames is the same as before but the frames now have trailers specifying the PayloadDigest and ChainDigest values:
Frame 0
{
"Index": 0,
"ContainerType": "Chain",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"Index": 1}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR
Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 2
{
"Index": 2}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR
Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 3
{
"Index": 3}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR
Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
The following example shows a chain container with a first frame and six data frames. The trailers now contain the TreePosition and TreeDigest values:
Frame 0
{
"Index": 0,
"ContainerType": "Merkle",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"Index": 1,
"TreePosition": 0}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV
z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 2
{
"Index": 2,
"TreePosition": 325}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk
FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}
Frame 3
{
"Index": 3,
"TreePosition": 325}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV
z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 4
{
"Index": 4,
"TreePosition": 1469}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvK
Sj8gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"}
Frame 5
{
"Index": 5,
"TreePosition": 1469}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV
z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 6
{
"Index": 6,
"TreePosition": 2616}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9S
Z0mdr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"}
The following example shows a tree container with a signature in the final record. The signing key parameters are:
{
"PrivateKeyECDH":{
"crv":"Ed25519",
"Private":"Bw6qDvg3D8IunEgMDBoDHFc1X-wnd577-PiXUR9RfFU"}}
The container headers and trailers are:
Frame 0
{
"Index": 0,
"ContainerType": "Merkle",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"Index": 1,
"TreePosition": 0}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV
z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}
Frame 2
{
"Index": 2,
"TreePosition": 325}
{
"PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD
lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
"TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk
FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}
The following example shows a container in which all the frame payloads are encrypted under the same master secret established in a key agreement specified in the first frame.
Frame 0
{
"enc": "A256CBC",
"Salt": "Q_LKYdTGwOHRZNjw4PrUbg",
"recipients": [{
"kid": "MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk": {
"PublicKeyECDH": {
"crv": "Ed25519",
"Public": "fuPEPja6OemH5ZZf_m5l83wMIEJQFWi_S71CF1yoE24"}},
"wmk": "3H77ypek_SUIs16CfApMCoBD4j8MD0LO3xOx5Njs6QVtV3qAlcygRg"}],
"Index": 0,
"ContainerType": "List",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"enc": "A256CBC",
"Salt": "TjNsKUyuXpKTujdd7E30KQ",
"Index": 1}
[Empty trailer]
Frame 2
{
"enc": "A256CBC",
"Salt": "HpQHRfWJv8JEOXGPFY5ncw",
"Index": 2}
[Empty trailer]
Here are the container bytes. Note that the content is now encrypted and has expanded by 25 bytes. These are the salt (16 bytes), the AES padding (4 bytes) and the JSON-B framing (5 bytes).
f5 01 c0 f1 01 ab f0 10 c0 01 f5 f5 01 7c f0 47 f1 01 30 7c 01 f5 f5 01 7c f0 47 f1 01 30 7c 01 f5
The following example shows a container in which all the frame payloads are encrypted under separate key agreements specified in the payload frames.
Frame 0
{
"Index": 0,
"ContainerType": "List",
"ContentMeta": {},
"DataEncoding": "JSON"}
[Empty trailer]
Frame 1
{
"enc": "A256CBC",
"Salt": "grVcSJ0gnISD1d3cueTkIQ",
"recipients": [{
"kid": "MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk": {
"PublicKeyECDH": {
"crv": "Ed25519",
"Public": "dKyTOgrG3IK5AxtLOsBVIq_ZqDKAIMI6ZpezK2JXlg4"}},
"wmk": "VRTC-LEJm5gDbsQZXxj4NVSnpEoVPjzBWUhcmyFdmDSzJEDMIIQWMQ"}],
"Index": 1}
[Empty trailer]
Frame 2
{
"enc": "A256CBC",
"Salt": "YEgzV_m5hgA1EmK15PsTpQ",
"recipients": [{
"kid": "MCNZ-6W5N-NWWW-FTHM-YSIM-P2JG-EBM7",
"epk": {
"PublicKeyECDH": {
"crv": "Ed25519",
"Public": "QMkq7pio3N_PoM_DV60f_RPo3hx1wZ4jqrmS64en_nE"}},
"wmk": "FIF2-JJMrPBgMP6Jeomdu_ldYv8FL5DEWq8TOJ_dPk9NIt7NxVLGRA"}],
"Index": 2}
[Empty trailer]
public long PreviousFrame (long Frame) {
long x2 = Frame + 1;
long d = 1;
while (x2 > 0) {
if ((x2 & 1) == 1) {
return x2 == 1 ? (d / 2) - 1 : Frame - d;
}
d = d * 2;
x2 = x2 / 2;
}
return 0;
}
The following issues need to be addressed.
| Issue | Description |
|---|---|
| X25519 | The examples currently use Edwards Curve25519 for encryption. This should be Curve25519 |
| Indexing | No examples are given of indexing a container |
| Archive | Should include a file archive example |
| File Path | Mention the file path security issue in the security considerations |
| Security Considerations | Write Security considerations |
| AES-GCM | Switch to using AES GCM in the examples |
| Witness | Complete handling of witness values. |
| Schema | Complete the schema documentation |
| Container Redo | Rework the container/header objects so that these are separate classes and Header is an entry in the Container header. |
| [draft-hallambaker-jsonbcd] | Hallam-Baker, P., "Binary Encodings for JavaScript Object Notation: JSON-B, JSON-C, JSON-D", Internet-Draft draft-hallambaker-jsonbcd-13, July 2018. |
| [draft-hallambaker-mesh-security] | , "[Reference Not Found!]" |
| [draft-hallambaker-mesh-udf] | Hallam-Baker, P., "Mathematical Mesh Part II: Uniform Data Fingerprint.", Internet-Draft draft-hallambaker-mesh-udf-01, February 2019. |
| [IANAJOSE] | , "[Reference Not Found!]" |
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC2315] | Kaliski, B., "PKCS #7: Cryptographic Message Syntax Version 1.5", RFC 2315, DOI 10.17487/RFC2315, March 1998. |
| [RFC3394] | Schaad, J. and R. Housley, "Advanced Encryption Standard (AES) Key Wrap Algorithm", RFC 3394, DOI 10.17487/RFC3394, September 2002. |
| [RFC4880] | Callas, J., Donnerhacke, L., Finney, H., Shaw, D. and R. Thayer, "OpenPGP Message Format", RFC 4880, DOI 10.17487/RFC4880, November 2007. |
| [RFC4949] | Shirey, R., "Internet Security Glossary, Version 2", FYI 36, RFC 4949, DOI 10.17487/RFC4949, August 2007. |
| [RFC5869] | Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand Key Derivation Function (HKDF)", RFC 5869, DOI 10.17487/RFC5869, May 2010. |
| [RFC6838] | Freed, N., Klensin, J. and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013. |
| [RFC7159] | Bray, T., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2014. |
| [RFC7515] | Jones, M., Bradley, J. and N. Sakimura, "JSON Web Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2015. |
| [RFC7516] | Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", RFC 7516, DOI 10.17487/RFC7516, May 2015. |
| [RFC7517] | Jones, M., "JSON Web Key (JWK)", RFC 7517, DOI 10.17487/RFC7517, May 2015. |
| [RFC7518] | Jones, M., "JSON Web Algorithms (JWA)", RFC 7518, DOI 10.17487/RFC7518, May 2015. |
| [BLOCKCHAIN] | Chain.com, "Blockchain Specification" |
| [Davis2001] | Davis, D., "Defective Sign & Encrypt in S/MIME, PKCS#7, MOSS, PEM, PGP, and XML", May 2001. |
| [RFC5652] | Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, RFC 5652, DOI 10.17487/RFC5652, September 2009. |
| [ZIPFILE] | PKWARE Inc, "APPNOTE.TXT - .ZIP File Format Specification", October 2014. |