Media Types for Sensor Markup Language (SENML)
draft-jennings-core-senml-03

Abstract

This specification defines media types for representing simple sensor measurements and device parameters in the Sensor Markup Language (SenML). Representations are defined in JavaScript Object Notation (JSON), Concise Binary Object Representation (CBOR), eXtensible Markup Language (XML), and Efficient XML Interchange (EXI), which share the common SenML data model. A simple sensor, such as a temperature sensor, could use this media type in protocols such as HTTP or CoAP to transport the measurements of the sensor or to be configured.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on July 4, 2016.

Copyright Notice

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Overview
• 2. Requirements and Design Goals
• 3. Terminology
• 4. Semantics
• 5. Associating Meta-data
• 6. JSON Representation (application/senml+json)
• 6.1. Examples
• 6.1.1. Single Datapoint
• 6.1.2. Multiple Datapoints
• 6.1.3. Multiple Measurements
• 6.1.4. Collection of Resources
• 7. CBOR Representation (application/senml+cbor)
• 8. XML Representation (application/senml+xml)
• 9. EXI Representation (application/senml-exi)
• 10. Usage Considerations
• 11. IANA Considerations
• 11.1. Units Registry
• 11.2. Media Type Registration
• 11.2.1. senml+json Media Type Registration
• 11.2.2. senml+cbor Media Type Registration
• 11.2.3. senml+xml Media Type Registration
• 11.2.4. senml-exi Media Type Registration
• 11.3. XML Namespace Registration
• 11.4. CoAP Content-Format Registration
• 12. Security Considerations
• 13. Privacy Considerations
• 14. Acknowledgement
• 15. References
• 15.1. Normative References
• 15.2. Informative References
• Authors' Addresses

1. Overview

Connecting sensors to the internet is not new, and there have been many protocols designed to facilitate it. This specification defines new media types for carrying simple sensor information in a protocol such as HTTP or CoAP called the Sensor Markup Language (SenML). This format was designed so that processors with very limited capabilities could easily encode a sensor measurement into the media type, while at the same time a server parsing the data could relatively efficiently collect a large number of sensor measurements. There are many types of more complex measurements and measurements that this media type would not be suitable for. A decision was made not to carry most of the meta data about the sensor in this media type to help reduce the size of the data and improve efficiency in decoding. Instead meta-data about a sensor resource can be described out-of-band using the CoRE Link Format [RFC6690]. The markup language can be used for a variety of data flow models, most notably data feeds pushed from a sensor to a collector, and the web resource model where the sensor is requested as a resource representation (e.g., “GET /sensor/temperature”).

SenML is defined by a data model for measurements and simple meta-data about measurements and devices. The data is structured as a single array that contains base value object(s) and array(s) of entries. Each entry is an object that has attributes such as a unique identifier for the sensor, the time the measurement was made, and the current value. Serializations for this data model are defined for JSON [RFC7159], CBOR [RFC7049], XML, and Efficient XML Interchange (EXI) [W3C.REC-exi-20110310].

For example, the following shows a measurement from a temperature gauge encoded in the JSON syntax.

[{}, [{ "n": "urn:dev:ow:10e2073a01080063", "v":23.5, "u":"Cel" }]]

In the example above, the first element of the root array is empty object since there are no base values. The second array inside the root array has a single measurement for a sensor named “urn:dev:ow:10e2073a01080063” with a temperature of 23.5 degrees Celsius.

2. Requirements and Design Goals

The design goal is to be able to send simple sensor measurements in small packets on mesh networks from large numbers of constrained devices. Keeping the total size of payload under 80 bytes makes this easy to use on a wireless mesh network. It is always difficult to define what small code is, but there is a desire to be able to implement this in roughly 1 KB of flash on a 8 bit microprocessor. Experience with Google power meter and large scale deployments has indicated that the solution needs to support allowing multiple measurements to be batched into a single HTTP or CoAP request. This “batch” upload capability allows the server side to efficiently support a large number of devices. It also conveniently supports batch transfers from proxies and storage devices, even in situations where the sensor itself sends just a single data item at a time. The multiple measurements could be from multiple related sensors or from the same sensor but at different times.

3. Terminology

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119].

4. Semantics

Each SenML representation carries a single array that represents a set of measurements and/or parameters. This array contains a base object with several optional attributes described below and a mandatory array of one or more entries.

Base Name:

This is a string that is prepended to the names found in the entries. This attribute is optional.

Base Time:

A base time that is added to the time found in an entry. This attribute is optional.

Base Units:

A base unit that is assumed for all entries, unless otherwise indicated. This attribute is optional.

Version:

Version number of media type format. This attribute is optional positive integer and defaults to 2 if not present.

The measurement or parameter entries array contains values for sensor measurements or other generic parameters, such as configuration parameters. There must be at least one entry in the array. This array is called simply “measurement array” in the following text.

Each array entry contains several attributes, some of which are optional and some of which are mandatory:

Name:

Name of the sensor or parameter. When appended to the Base Name attribute, this must result in a globally unique identifier for the resource. The name is optional, if the Base Name is present. If the name is missing, Base Name must uniquely identify the resource. This can be used to represent a large array of measurements from the same sensor without having to repeat its identifier on every measurement.

Units:

Units for a measurement value. Optional.

Value

Value of the entry. Optional if a Sum value is present, otherwise required. Values are represented using three basic data types, Floating point numbers (“v” field for “Value”), Booleans (“bv” for “Boolean Value”) and Strings (“sv” for “String Value”). Exactly one of these three fields MUST appear.

Sum:

Integrated sum of the values over time. Optional. This attribute is in the units specified in the Unit value multiplied by seconds.

Time:

Time when value was recorded. Optional.

Update Time:

A time in seconds that represents the maximum time before this sensor will provide an updated reading for a measurement. This can be used to detect the failure of sensors or communications path from the sensor. Optional.

The SenML format can be extended with further custom attributes placed in a base object, or in an entry. Extensions in a base object pertain to all entries following the base object, whereas extensions in an entry object only pertain to that entry.

Systems reading one of the objects MUST check for the Version attribute. If this value is a version number larger than the version which the system understands, the system SHOULD NOT use this object. This allows the version number to indicate that the object contains mandatory to understand attributes. New version numbers can only be defined in an RFC that updates this specification or it successors.

The Name value is concatenated to the Base Name value to get the name of the sensor. The resulting name needs to uniquely identify and differentiate the sensor from all others. If the object is a representation resulting from the request of a URI [RFC3986], then in the absence of the Base Name attribute, this URI is used as the default value of Base Name. Thus in this case the Name field needs to be unique for that URI, for example an index or subresource name of sensors handled by the URI.

Alternatively, for objects not related to a URI, a unique name is required. In any case, it is RECOMMENDED that the full names are represented as URIs or URNs [RFC2141]. One way to create a unique name is to include a EUI-48 or EUI-64 identifier (a MAC address) or some other bit string that has guaranteed uniqueness (such as a 1-wire address) that is assigned to the device. Some of the examples in this draft use the device URN type as specified in [I-D.arkko-core-dev-urn]. UUIDs [RFC4122] are another way to generate a unique name.

The resulting concatenated name MUST consist only of characters out of the set “A” to “Z”, “a” to “z”, “0” to “9”, “-“, “:”, “.”, or “_” and it MUST start with a character out of the set “A” to “Z”, “a” to “z”, or “0” to “9”. This restricted character set was chosen so that these names can be directly used as in other types of URI including segments of an HTTP path with no special encoding. [RFC5952] contains advice on encoding an IPv6 address in a name.

If either the Base Time or Time value is missing, the missing attribute is considered to have a value of zero. The Base Time and Time values are added together to get the time of measurement. A time of zero indicates that the sensor does not know the absolute time and the measurement was made roughly “now”. A negative value is used to indicate seconds in the past from roughly “now”. A positive value is used to indicate the number of seconds, excluding leap seconds, since the start of the year 1970 in UTC.

A measurement array MAY be followed by another base object and measurement array. The new base object can add, change, and/or remove base values from the previous base object(s). The new base values are applied to the following measurement arrays. Every base object MUST be followed by a measurement array, and hence base objects are found in the root array at even indexes and measurement arrays at odd indexes.

Representing the statistical characteristics of measurements can be very complex. Future specification may add new attributes to provide better information about the statistical properties of the measurement.

5. Associating Meta-data

SenML is designed to carry the minimum dynamic information about measurements, and for efficiency reasons does not carry more static meta-data about the device, object or sensors. Instead, it is assumed that this meta-data is carried out of band. For web resources using SenML representations, this meta-data can be made available using the CoRE Link Format [RFC6690].

The CoRE Link Format provides a simple way to describe Web Links, and in particular allows a web server to describe resources it is hosting. The list of links that a web server has available, can be discovered by retrieving the /.well-known/core resource, which returns the list of links in the CoRE Link Format. Each link may contain attributes, for example title, resource type, interface description, and content-type.

The most obvious use of this link format is to describe that a resource is available in a SenML format in the first place. The relevant media type indicator is included in the Content-Type (ct=) attribute.

Further semantics about a resource can be included in the Resource Type and Interface Description attributes. The Resource Type (rt=) attribute is meant to give a semantic meaning to that resource. For example rt=”outdoor-temperature” would indicate static semantic meaning in addition to the unit information included in SenML. The Interface Description (if=) attribute is used to describe the REST interface of a resource, and may include e.g. a reference to a WADL description [WADL].

6. JSON Representation (application/senml+json)

Base object variables:

Measurement or Parameter Entries:

All of the data is UTF-8, but since this is for machine to machine communications on constrained systems, only characters with code points between U+0001 and U+007F are allowed which corresponds to the ASCII [RFC0020] subset of UTF-8.

The root content consists of an array with even amount of JSON objects where the first (and then every odd) element is a base object and the second (and then every even) element is a measurements array. The base object MAY contain a “bn” attribute with a value of type string. The object MAY contain a “bt” attribute with a value of type number. The object MAY contain a “bu” attribute with a value of type string. The object MAY contain a “ver” attribute with a value of type number. The object MAY contain other attribute value pairs. The base object MUST be followed by an array. The array MUST have one or more measurement or parameter objects.

If the root array has more than one base object, each following base object modifies the base values using the JSON merge patch format [RFC7396]. That is, base values can be added or modified by defining their new values and existing base values can removed by defining the value as “null”.

Inside each measurement or parameter object the “n”, “u”, and “sv” attributes are of type string, the “t” and “ut” attributes are of type number, the “bv” attribute is of type boolean, and the “v” and “s” attributes are of type floating point. All the attributes are optional, but as specified in Section 4, one of the “v”, “sv”, or “bv” attributes MUST appear unless the “s” attribute is also present. The “v”, and “sv”, and “bv” attributes MUST NOT appear together.

Systems receiving measurements MUST be able to process the range of floating point numbers that are representable as an IEEE double-precision floating-point numbers [IEEE.754.1985]. The number of significant digits in any measurement is not relevant, so a reading of 1.1 has exactly the same semantic meaning as 1.10. If the value has an exponent, the “e” MUST be in lower case. The mantissa SHOULD be less than 19 characters long and the exponent SHOULD be less than 5 characters long. This allows time values to have better than micro second precision over the next 100 years.

6.1. Examples

6.1.1. Single Datapoint

The following shows a temperature reading taken approximately “now” by a 1-wire sensor device that was assigned the unique 1-wire address of 10e2073a01080063:

[{}, [{ "n": "urn:dev:ow:10e2073a01080063", "v":23.5, "u":"Cel" }]]

6.1.2. Multiple Datapoints

The following example shows voltage and current now, i.e., at an unspecified time. The device has an EUI-64 MAC address of 0024befffe804ff1.

[{"bn": "urn:dev:mac:0024befffe804ff1/"},
 [ { "n": "voltage", "t": 0, "u": "V", "v": 120.1 },
   { "n": "current", "t": 0, "u": "A", "v": 1.2 } ]
]

The next example is similar to the above one, but shows current at Tue Jun 8 18:01:16 UTC 2010 and at each second for the previous 5 seconds.

[{"bn": "urn:dev:mac:0024befffe804ff1/",
  "bt": 1276020076,
  "bu": "A",
  "ver": 2},
 [ { "n": "voltage", "u": "V", "v": 120.1 },
   { "n": "current", "t": -5, "v": 1.2 },
   { "n": "current", "t": -4, "v": 1.30 },
   { "n": "current", "t": -3, "v": 0.14e1 },
   { "n": "current", "t": -2, "v": 1.5 },
   { "n": "current", "t": -1, "v": 1.6 },
   { "n": "current", "t": 0,  "v": 1.7 } ]
]

Note that in some usage scenarios of SenML the implementations MAY store or transmit SenML in a stream-like fashion, where data is collected over time and continuously added to the object. This mode of operation is optional, but systems or protocols using SenML in this fashion MUST specify that they are doing this. In this situation the SenML stream can be sent and received in a partial fashion, i.e., a measurement entry can be read as soon as it is received and only not when the entire SenML object is complete.

For instance, the following stream of measurements may be sent from the producer of a SenML object to the consumer of that SenML object, and each measurement object may be reported at the time it arrives:

[{"bn": "http://[2001:db8::1]",
  "bt": 1320067464,
  "bu": "%RH"},
 [ { "v": 21.2, "t": 0 },
   { "v": 21.3, "t": 10 },
   { "v": 21.4, "t": 20 },
   { "v": 21.4, "t": 30 },
   { "v": 21.5, "t": 40 },
   { "v": 21.5, "t": 50 },
   { "v": 21.5, "t": 60 },
   { "v": 21.6, "t": 70 },
   { "v": 21.7, "t": 80 },
   { "v": 21.5, "t": 90 },
...

6.1.3. Multiple Measurements

The following example shows humidity measurements from a mobile device with an IPv6 address 2001:db8::1, starting at Mon Oct 31 13:24:24 UTC 2011. The device also provides position data, which is provided in the same measurement or parameter array as separate entries. Note time is used to for correlating data that belongs together, e.g., a measurement and a parameter associated with it. Finally, the device also reports extra data about its battery status at a separate time.

[{"bn": "http://[2001:db8::1]",
  "bt": 1320067464,
  "bu": "%RH"},
 [ { "v": 20.0, "t": 0 },
   { "v": 24.30621, "u": "lon", "t": 0 },
   { "v": 60.07965, "u": "lat", "t": 0 },  
   { "v": 20.3, "t": 60 },
   { "v": 24.30622, "u": "lon", "t": 60 },
   { "v": 60.07965, "u": "lat", "t": 60 },
   { "v": 20.7, "t": 120 },
   { "v": 24.30623, "u": "lon", "t": 120 },
   { "v": 60.07966, "u": "lat", "t": 120 },
   { "v": 98.0, "u": "%EL", "t": 150 },
   { "v": 21.2, "t": 180 },
   { "v": 24.30628, "u": "lon", "t": 180 },
   { "v": 60.07967, "u": "lat", "t": 180 } ]
]

6.1.4. Collection of Resources

The following example shows how to query one device that can provide multiple measurements. The example assumes that a client has fetched information from a device at 2001:db8::2 by performing a GET operation on http://[2001:db8::2] at Mon Oct 31 16:27:09 UTC 2011, and has gotten two separate values as a result, a temperature and humidity measurement.

[{"bn": "http://[2001:db8::2]/",
  "bt": 1320078429,
  "ver": 2},
 [ { "n": "temperature", "v": 27.2, "u": "Cel" },
   { "n": "humidity", "v": 80, "u": "%RH" } ]
]

7. CBOR Representation (application/senml+cbor)

The CBOR [RFC7049] representation is equivalent to the JSON representation, with the following changes:

• For compactness, the CBOR representation uses integers for the map keys defined in Table 1. This table is conclusive, i.e., there is no intention to define any additional integer map keys; any extensions will use string map keys.
• For JSON Numbers, the CBOR representation can use integers, floating point numbers, or decimal fractions (CBOR Tag 4); the common limitations of JSON implementations are not relevant for these. For the version number, however, only an unsigned integer is allowed.

<?xml version="1.0" encoding="UTF-8"?>
<senml xmlns="urn:ietf:params:xml:ns:senml"
       bn="urn:dev:mac:0024befffe804ff1/"
       bt="1276020076"
       ver="2" bu="A">
  <e n="voltage" u="V" v="120.1" />
  <e n="current" t="-5" v="1.2" />
  <e n="current" t="-4" v="1.30" />
  <e n="current" t="-3" v="0.14e1" />
  <e n="current" t="-2" v="1.5" />
  <e n="current" t="-1" v="1.6" />
  <e n="current" t="0" v="1.7" />
</senml>

default namespace = "urn:ietf:params:xml:ns:senml"
namespace rng = "http://relaxng.org/ns/structure/1.0"

e = element e {
  attribute n { xsd:string }?,
  attribute u { xsd:string }?,
  attribute v { xsd:float }?,
  attribute sv { xsd:string }?,
  attribute bv { xsd:boolean }?,
  attribute s { xsd:decimal }?,
  attribute t { xsd:int }?,
  attribute ut { xsd:int }?
}

senml =
  element senml {
    attribute bn { xsd:string }?,
    attribute bt { xsd:int }?,
    attribute bu { xsd:string }?,
    attribute ver { xsd:int }?,
    e*
  }

start = senml
    

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="urn:ietf:params:xml:ns:senml" xmlns:ns1="urn:ietf:params:xml:ns:senml">
  <xs:element name="e">
    <xs:complexType>
      <xs:attribute name="n" type="xs:string"/>
      <xs:attribute name="u" type="xs:string"/>
      <xs:attribute name="v" type="xs:float"/>
      <xs:attribute name="sv" type="xs:string"/>
      <xs:attribute name="bv" type="xs:boolean"/>
      <xs:attribute name="s" type="xs:decimal"/>
      <xs:attribute name="t" type="xs:int"/>
      <xs:attribute name="ut" type="xs:int"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="senml">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="ns1:e"/>
      </xs:sequence>
      <xs:attribute name="bn" type="xs:string"/>
      <xs:attribute name="bt" type="xs:int"/>
      <xs:attribute name="bu" type="xs:string"/>
      <xs:attribute name="ver" type="xs:int"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

<?xml version="1.0" encoding="UTF-8"?>
<senml xmlns="urn:ietf:params:xml:ns:senml"
       bn="urn:dev:ow:10e2073a01080063" >
  <e n="voltage" t="0" v="120.1" u="V" />
  <e n="current" t="0" v="1.2" u="A" />
</senml>

0000000 a0 30 49 cd 95 b9 b5 b0 b5 d8 c8 b5 9d 95 b8 b9
0000010 e1 cd 90 81 d7 57 26 e3 a6 46 57 63 a6 f7 73 a3
0000020 13 06 53 23 03 73 36 13 03 13 03 83 03 03 63 36
0000030 21 2e cd ed 8e 8c 2c ec a8 00 01 ab 2b 10 98 00
0000040 42 58 dd 5c 9c 99 5b 9d 10 00 03 41 41 90 08   
000004f

<?xml version="1.0" encoding="UTF-8"?>
<senml xmlns="urn:ietf:params:xml:ns:senml"
       bn="urn:dev:ow:10e2073a01080063" >
  <e n="temp" v="23.1" u="Cel" />
</senml>

00000000  a0 00 48 82 4e 6c ad cd  ad 85 ae c6 45 ac ec ad  |..H.Nl......E...|
00000010  c5 cf 0e 6c 80 01 00 1d  75 72 6e 3a 64 65 76 3a  |...l....urn:dev:|
00000020  6f 77 3a 31 30 65 32 30  37 33 61 30 31 30 38 30  |ow:10e2073a01080|
00000030  30 36 33 03 01 06 74 65  6d 70 03 05 43 65 6c 01  |063...temp..Cel.|
00000040  00 e7 01 01 00 01                                 |......|
00000046