Internet DRAFT - draft-rundgren-deterministic-cbor
draft-rundgren-deterministic-cbor
Network Working Group A. Rundgren, Ed.
Internet-Draft Independent
Intended status: Informational 22 August 2023
Expires: 23 February 2024
CBOR Deterministic Encoding Profile (CDEP)
draft-rundgren-deterministic-cbor-23
Abstract
This document describes CDEP, a deterministic encoding profile for
CBOR, intended for usage in high-end computing platforms like mobile
phones, Web browsers, and Web servers. In addition to enhancing
interoperability, deterministic encoding also enables performing
cryptographic operations like signing "raw" CBOR data items,
something which otherwise would require wrapping such data in byte
strings, or introduce dependencies on non-standard canonicalization
procedures.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 23 February 2024.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Rundgren Expires 23 February 2024 [Page 1]
Internet-Draft CDEP August 2023
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Background . . . . . . . . . . . . . . . . . . . . . . . 2
1.2. Objectives . . . . . . . . . . . . . . . . . . . . . . . 3
1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3
2. Specification . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. General Requirements . . . . . . . . . . . . . . . . . . 4
2.2. CBOR Data Items . . . . . . . . . . . . . . . . . . . . . 4
2.3. Encoding of Numbers . . . . . . . . . . . . . . . . . . . 5
2.3.1. Integer Numbers . . . . . . . . . . . . . . . . . . . 6
2.3.2. Special Floating Point Numbers . . . . . . . . . . . 7
2.3.3. "Ordinary" Floating Point Numbers . . . . . . . . . . 7
3. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8
4. Security Considerations . . . . . . . . . . . . . . . . . . . 8
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 9
5.1. Normative References . . . . . . . . . . . . . . . . . . 9
5.2. Informative References . . . . . . . . . . . . . . . . . 9
Appendix A. Incorrectly Encoded Numbers . . . . . . . . . . . . 9
Appendix B. Implementation Constraints . . . . . . . . . . . . . 11
Appendix C. Decoder Considerations . . . . . . . . . . . . . . . 11
Appendix D. Reference Implementations . . . . . . . . . . . . . 11
Appendix E. Online Tools . . . . . . . . . . . . . . . . . . . . 11
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 12
Document History . . . . . . . . . . . . . . . . . . . . . . . . 12
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 14
1. Introduction
This specification introduces a deterministic encoding profile for
data expressed in the CBOR [RFC8949] format. This profile is
subsequently referred to as CDEP.
Note that this document is not on the IETF standards track. However,
a conformant implementation is supposed to adhere to the specified
behavior for security and interoperability reasons.
1.1. Background
[RFC8949] supports a number of deterministic encoding options. Some
of these options are not necessarily interoperable, like Rule 1-3 in
Section 4.2.2 (https://rfc-editor.org/rfc/rfc8949#section-4.2.2).
This could potentially hamper large scale rollout of applications
depending on deterministically encoded CBOR.
Rundgren Expires 23 February 2024 [Page 2]
Internet-Draft CDEP August 2023
1.2. Objectives
The main objective of CDEP is providing an interoperable CBOR
encoding profile, _primarily_ targeting high-end computing platforms
like mobile phones, Web browsers, and Web servers. In addition, due
to the underpinning deterministic representation of data, CDEP also
enables performing cryptographic operations like signatures over
"raw" (unwrapped) CBOR data items since signatures depend on a
_unified_ representation of the data to be signed. Furthermore,
building on the same foundation, CDEP also permits decoded CBOR data
to be subjected to simple and secure _transformation_ and
_reencoding_ operations.
The deterministic encoding profile described in this document is
characterized by being _bidirectional_ also when CBOR is provided in
_diagnostic notation_ (Section 8 of [RFC8949]), making CDEP
comparatively easy to understand, debug, and implement.
Although CDEP is a _deterministic_ encoding profile, the intent is
that the encoding scheme should be equally useful for applications
that do not depend on this particular feature. See also Appendix C.
In spite of the enhanced functionality, this specification retains
full compatibility with [RFC8949].
See also [I-D.mcnally-deterministic-cbor] which represents an
alternative approach to deterministic encoding.
1.3. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2. Specification
The CDEP deterministic encoding profile builds on Section 4.2 of
[RFC8949].
The following sections contain some additional clarifications and
explicit choices, in order to facilitate an interoperable encoding
scheme.
Rundgren Expires 23 February 2024 [Page 3]
Internet-Draft CDEP August 2023
2.1. General Requirements
Occurrences of unknown or malformed CBOR data items MUST be rejected.
Map keys MUST only be compared and sorted based on their bytewise
lexicographic order of their deterministic encoding. In practical
terms this means that if the integer number 0 and the floating point
numbers 0.0, and -0.0 were used as map keys, they would represent the
proper sort order for the distinct keys 00, f90000, and f98000,
respectively. Duplicate map keys MUST be rejected.
For applications that depend on _deterministic reencoding_ of CBOR
data items, compliant decoder implementations MUST be able to
recreate such data in its original form. This means for example that
the string component of date items (tag 0) MUST be preserved "as is"
in order to maintain consistency.
The _optional_ numerical extensions described in Section 3.4.4 of
[RFC8949] MUST be treated as _distinct_ data items as well as not be
subjected to any transformations at the encoding level.
2.2. CBOR Data Items
Compliant CDEP implementations SHOULD as a _minimum_ support the
following CBOR data items:
Rundgren Expires 23 February 2024 [Page 4]
Internet-Draft CDEP August 2023
+================+=====================+
| Data Item | Encoding |
+================+=====================+
| integer | Major type 0 and 1 |
+----------------+---------------------+
| bignum | 0xc2 and 0xc3 |
+----------------+---------------------+
| floating point | 0xf9, 0xfa and 0xfb |
+----------------+---------------------+
| byte string | Major type 2 |
+----------------+---------------------+
| text string | Major type 3 |
+----------------+---------------------+
| false | 0xf4 |
+----------------+---------------------+
| true | 0xf5 |
+----------------+---------------------+
| null | 0xf6 |
+----------------+---------------------+
| array | Major type 4 |
+----------------+---------------------+
| map | Major type 5 |
+----------------+---------------------+
| tag | Major type 6 |
+----------------+---------------------+
Table 1: CBOR Data Items
See also Appendix B.
2.3. Encoding of Numbers
The following sub sections hold examples of numeric values expressed
in _diagnostic notation_ (Section 8 of [RFC8949]) and their CDEP
encoded counterpart (expressed in hexadecimal). See also Appendix A.
To achieve a _fixed and bidirectional_ representation of numbers,
Rule 2 in Section 4.2.2 of [RFC8949] MUST be adhered to. In
addition, integer and floating point data items MUST use preferred
serialization as described in Section 4.2.1 (https://rfc-
editor.org/rfc/rfc8949#section-4.2.1).
Note that the values and encodings are supposed to work in _both_
directions.
Rundgren Expires 23 February 2024 [Page 5]
Internet-Draft CDEP August 2023
2.3.1. Integer Numbers
The following table holds a set of integer/bignum values. Note that
bignum data items MUST use preferred serialization as described in
Section 3.4.3 of [RFC8949].
+=======================+========================+
| Value | Encoding |
+=======================+========================+
| 0 | 00 |
+-----------------------+------------------------+
| -1 | 20 |
+-----------------------+------------------------+
| 23 | 17 |
+-----------------------+------------------------+
| 24 | 1818 |
+-----------------------+------------------------+
| -24 | 37 |
+-----------------------+------------------------+
| -25 | 3818 |
+-----------------------+------------------------+
| 255 | 18ff |
+-----------------------+------------------------+
| 256 | 190100 |
+-----------------------+------------------------+
| -256 | 38ff |
+-----------------------+------------------------+
| -257 | 390100 |
+-----------------------+------------------------+
| 65535 | 19ffff |
+-----------------------+------------------------+
| 65536 | 1a00010000 |
+-----------------------+------------------------+
| 1099511627775 | 1b000000ffffffffff |
+-----------------------+------------------------+
| 18446744073709551615 | 1bffffffffffffffff |
+-----------------------+------------------------+
| 18446744073709551616 | c249010000000000000000 |
+-----------------------+------------------------+
| -18446744073709551616 | 3bffffffffffffffff |
+-----------------------+------------------------+
| -18446744073709551617 | c349010000000000000000 |
+-----------------------+------------------------+
Table 2: Integer Numbers
Rundgren Expires 23 February 2024 [Page 6]
Internet-Draft CDEP August 2023
2.3.2. Special Floating Point Numbers
The following table holds the set of special IEEE 754 [IEEE754]
values. Note that "signaling" NaN values MUST NOT be present.
+===========+==========+
| Value | Encoding |
+===========+==========+
| 0.0 | f90000 |
+-----------+----------+
| -0.0 | f98000 |
+-----------+----------+
| Infinity | f97c00 |
+-----------+----------+
| -Infinity | f9fc00 |
+-----------+----------+
| NaN | f97e00 |
+-----------+----------+
Table 3: Special
Floating Point Numbers
2.3.3. "Ordinary" Floating Point Numbers
The following table holds a set of "ordinary" IEEE 754 [IEEE754]
values including some edge cases. Note that subnormal floating point
values MUST be supported.
Rundgren Expires 23 February 2024 [Page 7]
Internet-Draft CDEP August 2023
+==========================+====================+
| Value | Encoding |
+==========================+====================+
| -5.960464477539062e-8 | fbbe6fffffffffffff |
+--------------------------+--------------------+
| -5.960464477539063e-8 | f98001 |
+--------------------------+--------------------+
| -5.960464477539064e-8 | fbbe70000000000001 |
+--------------------------+--------------------+
| -5.960465188081798e-8 | fab3800001 |
+--------------------------+--------------------+
| 0.00006097555160522461 | f903ff |
+--------------------------+--------------------+
| 65504.0 | f97bff |
+--------------------------+--------------------+
| 65504.00390625 | fa477fe001 |
+--------------------------+--------------------+
| 65536.0 | fa47800000 |
+--------------------------+--------------------+
| 10.559998512268066 | fa4128f5c1 |
+--------------------------+--------------------+
| 10.559998512268068 | fb40251eb820000001 |
+--------------------------+--------------------+
| 3.4028234663852886e+38 | fa7f7fffff |
+--------------------------+--------------------+
| 3.402823466385289e+38 | fb47efffffe0000001 |
+--------------------------+--------------------+
| 1.401298464324817e-45 | fa00000001 |
+--------------------------+--------------------+
| 1.1754942106924411e-38 | fa007fffff |
+--------------------------+--------------------+
| 5.0e-324 | fb0000000000000001 |
+--------------------------+--------------------+
| -1.7976931348623157e+308 | fbffefffffffffffff |
+--------------------------+--------------------+
Table 4: "Ordinary" Floating Point Numbers
3. IANA Considerations
This document has no IANA actions.
4. Security Considerations
This specification inherits all the security considerations of CBOR
[RFC8949].
Rundgren Expires 23 February 2024 [Page 8]
Internet-Draft CDEP August 2023
Applications that exploit the uniqueness of deterministic encoding
should verify that the used decoder actually rejects incorrectly
formatted CBOR data items.
5. References
5.1. Normative References
[IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", IEEE
754-2019, DOI 10.1109/IEEESTD.2019.8766229,
<https://ieeexplore.ieee.org/document/8766229>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8949] Bormann, C. and P. Hoffman, "Concise Binary Object
Representation (CBOR)", STD 94, RFC 8949,
DOI 10.17487/RFC8949, December 2020,
<https://www.rfc-editor.org/info/rfc8949>.
5.2. Informative References
[I-D.mcnally-deterministic-cbor]
McNally, W. and C. Allen, "Gordian dCBOR: Deterministic
CBOR Implementation Practices", Work in Progress,
Internet-Draft, draft-mcnally-deterministic-cbor-01, 4 May
2023, <https://datatracker.ietf.org/doc/html/draft-
mcnally-deterministic-cbor-01>.
Appendix A. Incorrectly Encoded Numbers
The following table holds a few examples of numeric CBOR data items
that MUST be rejected because their respective encoding does not
conform to CDEP. "PS" in the table is a short form for
"Preferred Serialization".
Rundgren Expires 23 February 2024 [Page 9]
Internet-Draft CDEP August 2023
+==========================+============================+
| Encoded | Error Description |
+==========================+============================+
| f97e01 | NaN "signaling" |
+--------------------------+----------------------------+
| f97c01 | Invalid NaN |
+--------------------------+----------------------------+
| fb7ff8000000000000 | PS: f97e00 |
+--------------------------+----------------------------+
| fb8000000000000000 | PS: f98000 |
+--------------------------+----------------------------+
| faff800000 | PS: f9fc00 |
+--------------------------+----------------------------+
| fa477fe000 | PS: f97bff |
+--------------------------+----------------------------+
| fab3800000 | PS: f98001 |
+--------------------------+----------------------------+
| fbbe70000000000000 | PS: f98001 |
+--------------------------+----------------------------+
| fa00000000 | PS: f90000 |
+--------------------------+----------------------------+
| fb36a0000000000000 | PS: fa00000001 |
+--------------------------+----------------------------+
| fb380fffffc0000000 | PS: fa007fffff |
+--------------------------+----------------------------+
| 1800 | PS: 00 |
+--------------------------+----------------------------+
| 1817 | PS: 17 |
+--------------------------+----------------------------+
| 1900ff | PS: 18ff |
+--------------------------+----------------------------+
| 1a000000ff | PS: 18ff |
+--------------------------+----------------------------+
| 1a0000ffff | PS: 19ffff |
+--------------------------+----------------------------+
| 1b00000000ffffffff | PS: 1affffffff |
+--------------------------+----------------------------+
| 3b00000000ffffffff | PS: 3affffffff |
+--------------------------+----------------------------+
| c2488000000000000000 | PS: 1b8000000000000000 |
+--------------------------+----------------------------+
| c348ffffffffffffffff | PS: 3bffffffffffffffff |
+--------------------------+----------------------------+
| c24a00800000000000000000 | PS: c249800000000000000000 |
+--------------------------+----------------------------+
Table 5: Incorrectly Encoded Numbers
Rundgren Expires 23 February 2024 [Page 10]
Internet-Draft CDEP August 2023
Appendix B. Implementation Constraints
This section is non-normative.
Note that even if an application does not support (or need) bignum or
floating point data items, CDEP is still applicable, since a _strict
subset_ is upwardly compatible with full-blown implementations. Low-
end platforms typically also restrict CBOR map keys to integer and
text string data items. Since these issues are application specific,
they are out of scope for this specification.
Appendix C. Decoder Considerations
This section is non-normative.
To not unnecessarily create incompatibilities with the existing CBOR
ecosystem, CDEP decoders may benefit from supporting a _non-
deterministic mode_, where the map key ordering and preferred
serialization checks are disabled.
Appendix D. Reference Implementations
This section is non-normative.
Reference implementations that conform to this specification include:
* JavaScript: <https://github.com/cyberphone/CBOR.js#cborjs>
* JDK 17+: <https://github.com/cyberphone/openkeystore#cbor-support>
* Android/Java: <https://github.com/cyberphone/android-cbor#cbor-
for-android>
Appendix E. Online Tools
This section is non-normative.
The following online tools enable testing CDEP without installing any
software:
* <https://cyberphone.github.io/CBOR.js/doc/playground.html>
* <https://test.webpki.org/csf-lab/convert>
Rundgren Expires 23 February 2024 [Page 11]
Internet-Draft CDEP August 2023
Acknowledgements
This document incorporates much appreciated suggestions and feedback
by Eliot Lear, Wolf McNally, Laurence Lundblade, Joe Hildebrand, and
Carsten Bormann.
Document History
[[ This section to be removed by the RFC Editor before publication as
an RFC ]]
Version 00:
* Initial publication.
Version 01:
* Added Table 1: Supported CBOR Data Types
Version 02:
* Added bidirectional + reencoding to 2
Version 03:
* Added ref to 3.4.4. Decimal Fractions and Bigfloats.
* Type => Data Item (throughout the spec).
Version 04-00:
* Document name spelling error.
Version 01:
* Minor tweaks.
Version 02:
* ISE submission and associated changes.
Version 03:
* Number table clarifications.
Version 04:
* Word-smithing.
Rundgren Expires 23 February 2024 [Page 12]
Internet-Draft CDEP August 2023
Version 05:
* ISE input resulted in Background section.
Version 06:
* Word-smithing.
Version 07:
* Word-smithing.
Version 08:
* Explained universality.
Version 09:
* Stream added.
Version 10:
* External "Section" refs made into links.
Version 11:
* IEEE 754 ref.
Version 12:
* Language nit.
Version 13:
* Major restructuring of "Specification".
Version 14:
* Word-smithing.
Version 15:
* Word-smithing.
Version 16:
* Added section references to RFC8949 for numbers.
Rundgren Expires 23 February 2024 [Page 13]
Internet-Draft CDEP August 2023
Version 17:
* Acknowledgements.
Version 18:
* D-CBOR => CDEP.
* Added "Decoder Considerations".
Version 19:
* Minor restruction of Number section.
Version 20:
* "Incorrectly Encoded Numbers" added.
Version 21:
* Improved integer table text.
Version 22:
* Map key example.
Version 23:
* Map key duplicate handling.
Author's Address
Anders Rundgren (editor)
Independent
Montpellier
France
Email: anders.rundgren.net@gmail.com
URI: https://www.linkedin.com/in/andersrundgren/
Rundgren Expires 23 February 2024 [Page 14]
