core A. Bierman
Internet-Draft YumaWorks, Inc.
Intended status: Standards Track P. van der Stok
Expires: February 17, 2017 consultant
August 16, 2016

Numeric YANG Identifiers
draft-bierman-core-yid-00

Abstract

This document describes an encoding of YANG object identifiers using numeric values instead of string values. It combines several techniques to provide optimized serialization in protocol messages.

Note

Discussion and suggestions for improvement are requested, and should be sent to core@ietf.org.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on February 17, 2017.

Copyright Notice

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.


Table of Contents

1. Introduction

This document describes mechanisms for mapping YANG schema nodes to numeric values. Existing YANG-based protocol use path expression strings or element hierarchies that use element names to identify a particular YANG data node. These object identifiers can represent a significant percentage of protocol message payload content.

Constrained environments require efficient protocols and a numeric (binary) representation of YANG object identifiers can help reduce payload overhead in protocol messages.

The effective use of YANG Identifiers requires three components:

This document replaces the YANG Hash draft [I-D.bierman-core-yang-hash]. The core concepts from that draft have been incorporated into this document.

1.1. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

Readers of this specification should be familiar with all the terms and concepts discussed in [RFC2578].

The following terms are defined in the NETCONF protocol [RFC6241]: client, configuration data, datastore, and server.

The following terms are defined in the YANG data modelling language [RFC6020]: container, data node, key, key leaf, leaf, leaf-list, and list.

The following terms are defined in RESTCONF protocol [I-D.ietf-netconf-restconf]: data resource, data-store resource, edit operation, query parameter, target resource, and unified data-store.

The following terms are defined in this document:

YANG module schema tree:
The conceptual tree of YANG objects comprised of all "rpc", "data-def" and "notification" statements within a particular revision of a module and all submodules included by this module.
YANG Identifier:
Numeric object identifier, which is a fixed-length numeric value that represents a particular schema node within a YANG module schema tree. The length and format of a YANG identifier are defined in the YANG Identifier Registry. Also called a "YID".
YANG Identifier File:
Also called a YID File. Contains a representation of the manual numeric path assignments for a YANG module.

1.1.1. Examples

Some text within examples throughout the document are split into multiple lines for display purposes only. When a line ends with backslash ('\') as the last character, the line is wrapped for display purposes. It is to be considered to be joined to the next line by deleting the backslash, the following line break, and the leading whitespace of the next line.

1.1.2. Tree Diagrams

A simplified graphical representation of the data model is used in the YANG modules specified in this document. The meaning of the symbols in these diagrams is as follows:

1.2. Design Objectives

This work is motivated by the need to minimize the size of object identifiers within protocol messages representing YANG data, encoded using binary format. The string encoding size of YANG identifier names can be very significant since designers are encouraged to provide long descriptive names to help human readability of the YANG module.

It is therefore desirable to produce an object numbering algorithm that can be serialized efficiently in protocol messages with binary encoding formats such as CBOR.

There are several design objectives for this work:

1.3. Solution Approach

This document describes a solution with four components:

YANG Identifier Registry:
A YANG Identifier registry contains manually administered information for each YANG module. A minimal amount of information is needed for each module. A numeric module identifier needs to be assigned for each module in the registry. Most registry information is permanent. New entries can be added but naming information for old entries cannot be changed. Each module can be either be numbered automatically using YANG Hash, or manually using registry assignment.
YANG Identifier Format:
A YANG Identifier format is a structured binary identifier that allows a hierarchical YANG schema node to be represented as a single integer. The mapping is based on a murmur32 hash of the YANG object identifier string for each data definition node. The YANG path identifier is not allowed to change value after it is first assigned, so it can be safely and easily used as a permanent identifier. The value does not depend on module statement order or any field that is allowed to change in YANG in new module revisions.
YANG Hash:
The YANG Hash automatic numbering algorithm, defined in Section 5, is used to automatically assign the numeric identifiers to YANG data definition schema nodes. Hash values are scoped by the module identifier, so clashes between modules are not possible. If a hash collision does occur within the same module, then a registry entry is used to map the clashing data nodes to manually assigned identifiers. Hash collisions are not allowed in the registry. They are manually avoided without any need to modify the YANG module.
Manual Numbering:
A manual numbering mechanism and a YANG Identifier template to specify manually assigned local identifiers is needed. This allows compact numbering of objects and therefore the most efficient use of identifier numbering space.

2. YANG Identifier Registry

A YANG Identifier registry is a data structure representing the numeric mapping information for a set of YANG modules. This is a global registry, meaning that all servers that use the same registry use the same numeric assignments.

Ideally, all servers will use the same registry and there will be only one registry. It is expected that a server will use the official registry maintained by IANA, but other registries including locally administered registries are possible. Each registry is self-contained. Registries cannot be mixed or reference other registries. (This is a subject for future work). All YANG identifier values are hierarchical, and scoped by the numeric identifier assigned in the registry for the module.

The YANG Identifier registry supports automatically numbered objects within a module using YANG Hash. It also supports manually numbered objects within a module using object path to number mappings. If YANG Hash is used, then some of the numbering space for the module is reserved for manual assignments in case there are any hash collisions within the module.

The size of the YANG identifiers, and the size of each field within the YANG identifier, are configured in the registry. The type of local identifier numbering is also configured, on a per-module basis. A server can only support one registry at a time, and it is expected to advertise which registry it is using via the management protocol in use.

Each registry entry represents the numeric mapping information for one YANG module. Only one entry per module is maintained, for the latest revision. Registry information is never allowed to change. Information for new schema nodes can be added but information for existing schema nodes can never be altered.

A YANG Identifier registry provides the following global parameters that apply to all modules:

The registry also has a list of modules that support numeric YANG identifiers. Each module entry has the following parameters that apply to all revisions of the module:

If the 'local-type' is equal to 'manual' for a module entry, then the manually assigned numbers are provided in the module registry entry. This information can either be in-line in the registry entry or accessible via an external URI.

The following information is provided for every YANG identifier in the module if the mapping parameters are local:

The following information is provided for the entire module if the mapping parameters are remote:

2.1. YANG Identifier Registry Example

The following example shows a JSON representation of a "yid-registry" container. There are 3 modules (ietf-system, ietf-yang-library, and example-address). The registry uses 32 bit identifiers. 16 bits are used for local identifiers in each module.

  { "yid-registry" : {
      "name" : "example-reg",
      "revision" : 0x7e00701,
      "module-bits" : 16,
      "local-bits" : 16,
      "module" : [
        {
          "module-id" : 1,
          "name" : "ietf-system",
          "revision" : 0x7de0806,
          "local-type" : "hash"
        },
        {
          "module-id" : 2,
          "name" : "ietf-yang-library",
          "revision" : 0x7e00615,
          "local-type" : "hash"
        },
        {
          "module-id" : 3,
          "name" : "example-address",
          "revision" : 0x7de0508,
          "local-type" : "manual",
          "mapping-url" : "http://example.com/yang-ids-47.json",
          "mapping-type" : "application/json"
        }
      ]
    }
  }

3. YANG Identifier

A YANG Identifier is constructed of 2 fields: a module identifier called "module-id" and a local identifier within the scope of the module, called "local-id".

   YANG Identifier Formats:

       Form 1: YANG Hash Used

          +-------------+  +-------------+
          |  module-id  |  |R| local-id  |
          +-------------+  +-------------+

       Form 2: Manual Identifiers used

          +-------------+  +-------------+
          |  module-id  |  |  local-id   |
          +-------------+  +-------------+

The local identifier can either be a YANG Hash if the 'local-type' for the module registry entry is 'hash', or manual numbering if the 'local-type' is 'manual'. If YANG Hash is used, then one bit is reserved within the local-id as the "rehash bit".

A YANG Identifier is derived from the 'module-id', 'local-bits', and 'local-id' fields, using the following formula:


   YID = (module-id * (2 ^^ local-bits)) + local-id

3.1. module-id

The module-id field is a mandatory fixed-width number that must be unique within the registry. This number is manually assigned for each YANG module and is specified in the registry entry information for the YANG module. All other YANG identifier sub-fields are scoped by this module-id. It is not possible for YANG identifier values from different modules within a YANG Identifier registry to collide.

Module identifiers are expected to be assigned by the registry maintainer, not by YANG module authors. It is critical that assignments are unique and stable. Registry maintenance procedures are discussed in the IANA Considerations section . Section 7.

3.2. local-id

The local-id is a fixed width number defined in the YANG Identifier registry. Local identifiers can be one of two formats: YANG Hash or Manually Assigned.

3.2.1. YANG Hash Numbered local-id

The automatic numbering of local identifier values is based on the YANG Hash for the path identifier for the schema node. An absolute path expression is used to identify the schema node.

The YANG registry value of 'local-bits' determines the size of the YANG hash value. One bit is reserved as the "rehash" bit and "local-bits - 1" number of bits are used for the YANG hash value.

A YANG Hash is calculated for the path expression for each schema node according to the procedure defined in Section 5. The lowest order bits are used.

If any hash collisions occur, then a manual registry entry is needed for all but one of the colliding nodes. Since the value zero is reserved, a hash value of zero is considered a clash, and must be fixed using a manually assigned entry for the node.

For example, if 8 bits were used for the 'local-bits' value, then the low order 7 bits are for hash values. The local-id values 1 - 127 would be allowed for hash values. The values 128 - 255 would be allowed for manually assigned rehash values.

3.2.2. Manually Numbered local-id

A YANG Identifier data structure is used to represent the manually assigned schema node assignments for a YANG module. Each data node is explicitly assigned a numeric value that is unique within the YANG module where it is defined. Data nodes that augment external modules are defined in the augmenting module. Once an assignment is made it can never be changed. Only new assignments can be made in new revisions of a module registry entry.

If local identifiers are used then the mapping between the YANG object identifier and numeric value MUST be provided via the registry. The format for 'local' entries is defined in the YANG module in Section 4.1.

The format of registry entry files that use the 'remote' form of local-id identification is outside the scope of this document. The only requirements are that the fields "local-id" and "path" MUST be used in each entry to represent the YANG identifier to numeric mapping. The 'mapping-type' field is used to determine the syntax and semantics of the data represented by the 'mapping-url' resource.

4. YANG Identifier Registry Format

A YANG Identifier Registry is a data structure that represents the module identifier encoding details for all modules in the registry. It is expected that the protocol using YANG Identifiers will indicate which YANG Identifier Registry it is using.

The YANG Identifier Registry syntax and semantics are defined using a YANG module called "ietf-yid". This allows instances that conform to the registry structure to be serialized in different encoding formats, such as XML, JSON, or CBOR.

The following YANG tree diagram represents the contents of the ietf-yid module:

   +--rw yid-registry!
      +--rw name           string
      +--rw revision       yid-revision-id
      +--rw module-bits    uint8
      +--rw local-bits     uint8
      +--rw module* [module-id]
         +--rw module-id       yid-module-id
         +--rw name            string
         +--rw revision        yid-revision-id
         +--rw local-type      enumeration
         +--rw (local-type-parms)?
            +--:(local)
            |  +--rw mapping* [local-id]
            |     +--rw local-id    yid-local-id
            |     +--rw path        string
            +--:(remote)
               +--rw mapping-url?    inet:uri
               +--rw mapping-type?   string

4.1. ietf-yid Module

<CODE-BEGINS file=ietf-yid@2016-08-16.yang>

module ietf-yid {
  namespace "urn:ietf:params:xml:ns:yang:ietf-yid";
  prefix "yid";
  import ietf-inet-types { prefix inet; }

  organization
    "IETF CORE (Constrained RESTfull Environments) Working Group";

  contact
    "WG Web:   <http://tools.ietf.org/wg/core/>
     WG List:  <mailto:core@ietf.org>

     Author:   Andy Bierman
               <mailto:andy@yumaworks.com>

     Author:   Peter Van der Stock
               <mailto:consultancy@vanderstok.org>
    ";

  description
    "This module contains a conceptual YANG specification
     for a YANG Identifier Registry.

     Copyright (c) 2016 IETF Trust and the persons identified as
     authors of the code.  All rights reserved.

     Redistribution and use in source and binary forms, with or
     without modification, is permitted pursuant to, and subject
     to the license terms contained in, the Simplified BSD License
     set forth in Section 4.c of the IETF Trust's Legal Provisions
     Relating to IETF Documents
     (http://trustee.ietf.org/license-info).

     This version of this YANG module is part of RFC XXXX; see
     the RFC itself for full legal notices.";

  // RFC Ed.: replace XXXX with actual RFC number and remove this
  // note.

  // RFC Ed.: remove this note
  // Note: extracted from draft-bierman-core-yid-00.txt

  // RFC Ed.: update the date below with the date of RFC publication
  // and remove this note.
  revision 2016-08-16 {
    description
      "Initial revision.";
    reference
      "RFC XXXX: Numeric YANG Identifiers.";
  }

  extension permanent-data {
    description
      "This extension can be used as a sub-statement in any data
       definition statement.  It indicates that an instance of
       the associated data node is considered permanent.

       The value of an instance of a data node that contains this
       extension MUST NOT change after it is assigned in
       a published version of an instance document.

       Note that a work-in-progress is not a published document.
       An RFC or IANA registry entry are examples of
       published document.

       There is no parameter value for this extension.";
  }

  typedef yid-revision-id {
    type uint32;
    description
      "Contains a numeric representation of the YANG Identifier
       registry version.  This value SHOULD be constructed
       from a revision date string, using the following
       unsigned integer format: YYYYMMDD

        where:

         YYYY is a 16-bit integer representation of the year
         MM is a 8-bit integer representation of the month (1 - 12)
         DD is a 8-bit integer representation of the day in the month
      ";
  }

  typedef yid-module-id {
    type uint32 {
      range "1 .. max";
    }
    description
      "Represents a YANG Identifier module-id field.";
  }

  typedef yid-local-id {
    type uint32 {
      range "1 .. max";
    }
    description
      "Represents a YANG Identifier local-id field.";
  }

  grouping yid-registry {
    description
      "A grouping that representing the syntax and semantics of a
       YANG Identifier Registry.";

    container yid-registry {
      presence "Instantiation of the YID registry is required.";

      description
        "Represents an YANG Identifier Registry";

      leaf name {
        type string;
        mandatory true;
        description
          "An administrative name for the YANG identifier registry.
           This value SHOULD NOT change once assigned.";
      }

      leaf revision {
        type yid-revision-id;
        mandatory true;
        description
          "A revision identifier that is unique within all
           revisions of a YANG Identifier registry.
           This value MUST NOT change once assigned in
           the first revision of the YANG Identifier registry.";
      }

      leaf module-bits {
        yid:permanent-data;
        type uint8 {
          range "4 .. 32";
        }
        mandatory true;
        description
          "The number of bits reserved for all module identifier
           values used within the registry.

           This value MUST NOT be changed after initial
           publication of the registry.";
      }

      leaf local-bits {
        yid:permanent-data;
        type uint8 {
          range "4 .. 32";
        }
        mandatory true;
        description
          "The number of bits reserved for all local identifier
           values used within each module.

           The value 'module-bits + local-bits' represents
           the total number of bits contained in all full-form
           YANG Identifier values used within the registry.

           This value MUST NOT be changed after initial
           publication of the registry.";
      }

      list module {
        key module-id;
        unique name;

        description
          "Represents one module entry in the registry.
           Only complete YANG module schema trees are
           represented in a YID registry entry.
           Sub-module or incomplete representations are
           not allowed.";

        uses yid-module-entry;
      }  // list module
    }  // container yid-registry
  }  // grouping yid-registry


  grouping yid-module-entry {

    description
      "Represents one module entry in the registry.
       Only complete YANG module schema trees are
       represented in a YID registry entry.
       Sub-module or incomplete representations are
       not allowed.";

    leaf module-id {
      yid:permanent-data;
      type yid-module-id;
      description
        "The numeric identifier assigned to this module.
         This value MUST NOT be changed once the the module
         entry is created.";

    }

    leaf name {
      yid:permanent-data;
      type string;
      mandatory true;
      description
        "The module name string. This exactly matches
         the value used in the associated YANG module.

         This value MUST NOT be changed once the the module
         entry is created.";
    }

    leaf revision {
        type yid-revision-id;
        mandatory true;
        description
          "A revision identifier that is unique within all
           revisions of the specified YANG module.

           This value MUST be updated to the new publication
           identifier value each time the YANG module is updated
           in the registry. Only one entry per module is
           maintained.";
    }

    leaf local-type {
      yid:permanent-data;
      type enumeration {
        enum hash {
          value 0;
          description
            "All local-id values where the most significant bit
             is set are YANG hash values.

             All local-id values where the most significant bit
             is not set are manually assigned YANG rehash values.";
        }
        enum manual {
          value 1;
          description
            "All local-id values are manually assigned identifiers.";
        }
      }
      mandatory true;
      description
        "The type of numbering used in local identifiers.";
    }

    choice local-type-parms {
      description
        "The manual local-id assignments can be done inline
         or contained in a remote YID file representation.
         This choice MUST be provided if any local identifier
         mappings are used in the module.  This includes
         all manually assigned values, and and rehash entries,
         if the associated 'local-type' value is 'hash'.";
      case local {
        description
          "Case arm for inline local-id manual assignments.
           The YID node list MAY be updated in the registry for
           each new revision of the associated YANG module.";

        list mapping {
          key "local-id";
          unique path;
          description
            "Represents one manual schema node assignment.";

          leaf local-id {
            yid:permanent-data;
            type yid-local-id;
            description
              "The manually assigned local-id value for this entry.
               The actual allowed range for this list is controlled
               by the value of 'local-bits'.  The number of entries
               allowed is (2 ^^ local-bits) - 2. The 'local-id' value
               zero is reserved and cannot be used.";
          }

          leaf path {
            yid:permanent-data;
            type string;
            mandatory true;
            description
              "The path expression representing this schema node.
               This value MUST NOT be changed in new revisions
               of an instance of a YID file.";
          }
        }
      }
      case remote {
        leaf mapping-url {
          type inet:uri;
          description
            "Case arm for remote specification of the manual
             assignments. The value is a URI for a 
             representation of the YID File associated with the
             manually assigned numbers for a set of YANG schema
             nodes with a YANG module.

             This value MAY be updated in the registry for
             each new revision of the associated YANG module.";
        }
        leaf mapping-type {
          type string;
          description
            "The value is a string identifying the media-type
             for the representation of mapping information
             identified by the associated 'mapping-url' value.

             This value MAY be updated in the registry for
             each new revision of the associated YANG module.";
        }
      }   // case remote
    }  // choice local-parm-types

  } // grouping yid-module-entry


  uses yid-registry;

}

<CODE ENDS>

5. YANG Hash Generation

Hash based local identifiers are defined in this section. The association between a path string value and numeric value is done through a hash algorithm. The 'N' least significant bits of the "murmur3" 32-bit hash algorithm are used. The value of 'N' is defined by the "local-bits" value in the YANG Identifier registry. This hash algorithm is described online at [murmur3]. Implementations are available online [murmur-imp]. When converting 4 input bytes to a 32-bit integer in the hash algorithm, the Little-Endian convention MUST be used.

The "murmur3_32" hash function is executed for the entire path string.

The value '42' is used as the seed for the hash function. The YANG hash is subsequently calculated by taking the 'N - 1' least significant bits, where the value of N is defined by the "local-bits' value for the YANG Identifier registry.

The resulting number is used by the server. If the value is already being used for a different object within the same YANG module, then the assignment is considered invalid, and MUST be resolved in the registry entry. This is done by separating the clashing objects by manually assigning identifiers.

The hash is generated for the string representing the object path identifier. A canonical representation of the path identifier is used.

The module name is used to identify the namespace of the object node. The prefix cannot be used because it is allowed to change over time. The module name is never allowed to change.

The module name MUST be present in the identifier for the first node in the object path identifier.

If a child node in the object path identifier is from the same module namespace as its parent, then the module-name MUST NOT be used in the identifier.

If a child node in the object path identifier is not from the same module namespace as its parent, then the module-name MUST be used in the identifier.

Choice and case node names are not included in the path expression. Only 'container', 'list', 'leaf', 'leaf-list', and 'anyxml' nodes are listed in the path expression.

The YANG Hash value is calculated for all data nodes in the module. The hash values are calculated, even if the server only implements a subset of these objects. This includes all "data-def" statements.

Example: the following identifier is for the 'mtu' leaf in the ietf-interfaces module:


   /ietf-interfaces:interfaces/interface/mtu

Example: the following identifier is for the 'ipv4' container in the ietf-ip module, which augments the 'interface' list in the ietf-interfaces module:


   /ietf-interfaces:interfaces/interface/ietf-ip:ipv4

6. Security Considerations

The exchange of names by numbers does not affect the security of the transmitted requests.

7. IANA Considerations

This document requests the following actions be taken by IANA

 1) add the "ietf-yid" YANG module to the YANG Module Names Registry
 2) add a new "module-id" parameter to the YANG Module Names Registry
 3) create a new YANG Identifiers Registry

7.1. YANG Module Library Entry: ietf-yid

This document registers one URI as a namespace in the IETF XML registry [RFC3688]. Following the format in RFC 3688, the following registration is requested:

   URI: urn:ietf:params:xml:ns:yang:ietf-yid
   Registrant Contact: The CORE WG of the IETF.
   XML: N/A, the requested URI is an XML namespace.

This document registers one YANG module in the YANG Module Names registry [RFC6020]:

   name:         ietf-yid
   namespace:    urn:ietf:params:xml:ns:yang:ietf-yid
   prefix:       yid
   // RFC Ed.: replace XXXX with RFC number and remove this note
   reference:    RFCXXXX

7.2. YANG Module Names Parameter Addition: module-id

This document requests that a new parameter named "module-id" be added to the YANG Module Names Registry. This value is a 20 bit number, greater than zero, that represents an arbitrary integer assignment (by IANA) for the YANG module.

This is only required for registry entries that represent YANG modules, not YANG sub-modules. The YANG typedef "yid-module-id" in the "ietf-yid" YANG module defines the syntax and semantics of a module-id value.

7.3. YANG Identifier Registry

This document requests that a new YANG Identifier Registry be created and maintained by IANA. This registry is used to archive numeric numbering information for YANG modules found in the YANG Module Names registry.

This is only required for registry entries that represent YANG modules, not YANG sub-modules. The YANG data node "yid-registry" in the "ietf-yid" YANG module defines the syntax and semantics of a YANG Identifier registry.

The syntax and semantics of the registry structure is described by the "yid-registry" grouping definition in the "ietf-yid" YANG module defined in Section 4.1.

A "module" entry needs to be maintained for each YANG module maintained by IANA. The "yid-module-entry" grouping in the "ietf-yid" YANG module defines the syntax and semantics of each module entry. The "module-id" field in the entry must match the "module-id" assigned to the YANG module in the YANG Module Names registry.

TBD: List the tools available to generate YANG hash and manual assignment entries

The following initial parameter values are suggested for the registry:

  { "yid-registry" : {
      "name" : "ietf-yid",
      "revision" : TBD,
      "module-bits" : 20,
      "local-bits" : 16
    }
  }

7.3.1. Registry Maintenance

When a YANG module is added to the YANG Module Names Registry, IANA will assign a unique module-id value for the module. At this time an initial YANG Identifier registry entry for the module is created.

When a YANG module update is being published, and IANA actions are being processed for the pending RFC, an update to the YANG Identifier registry for the existing module is created. The registry entry "revision" field is updated to a new value that is greater than the previous value.

No existing entry information is allowed to change. Only new manually numbered local-id values can be added. For local mappings, new "mapping" list entries can be added. For remote mappings, the "mapping-url" and "mapping-type" fields can be updated to identify the new mapping information.

For YANG modules that use YANG hash based local-id values, there is no need to update the registry entry for the module unless any hash collisions would be introduced by the new module revision. For YANG modules that use manually numbered local-id values, there is no need to update the registry unless new data nodes have been added to the YANG module.

8. Acknowledgements

We are very grateful to Bert Greevenbosch who suggested the use of hashes and specified the use of murmur3.

9. References

9.1. Normative References

, ", "
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.
[RFC6020] Bjorklund, M., YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)", RFC 6020, DOI 10.17487/RFC6020, October 2010.
[murmur3]murmurhash family", Web http://en.wikipedia.org/wiki/MurmurHash
[murmur-imp]murmurhash implementation", Web https://code.google.com/p/smhasher/

9.2. Informative References

[RFC2578] McCloghrie, K., Perkins, D. and J. Schoenwaelder, "Structure of Management Information Version 2 (SMIv2)", STD 58, RFC 2578, DOI 10.17487/RFC2578, April 1999.
[RFC4293] Routhier, S., "Management Information Base for the Internet Protocol (IP)", RFC 4293, DOI 10.17487/RFC4293, April 2006.
[RFC6241] Enns, R., Bjorklund, M., Schoenwaelder, J. and A. Bierman, "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011.
[I-D.ietf-netconf-restconf] Bierman, A., Bjorklund, M. and K. Watsen, "RESTCONF Protocol", Internet-Draft draft-ietf-netconf-restconf-16, August 2016.
[I-D.bierman-core-yang-hash] Bierman, A. and P. Stok, "YANG Hash", Internet-Draft draft-bierman-core-yang-hash-00, February 2016.

Appendix A. Open Issues

There are some advanced features not specified in this document. These can be left as future work.

When to assign a module-id:
There may be significant delay between the time a module is used in development and when it is published and added to a registry. Once a module-id has been assigned, it cannot be undone or changed. When should a module-id be assigned in a YANG Identifier registry? Should it be early in development, late in development, or after the module approval process?
How to support private numbering:
Should it be possible for a registry used by a server to contain module-id values that have not been globally assigned in the registry? Should a numeric range be reserved for this purpose? If so, is this range globally reserved or configured into each registry?
How to combine registries:
What if a server vendor wants to support modules not listed in a single registry? How can this be supported? E.g., increase the size of the identifier and allocate some bits for a registry identifier?
Should more YANG statements be numbered:
Should "rpc" and "notification" statements be numbered? Should YANG 1.1 "action" statements be numbered?
How to support anyxml and anydata subtrees:
A YANG data node that is defined with the "anyxml" ar "anydata" statement has an identifier for the top of the specified subtree in instance documents. XML and JSON representations of these YANG statements uses a QName to identify each descendant node within the anyxml or anydata subtree. These descendant nodes do not exist in the YANG module schema tree. They only exist in instance documents conforming to the YANG module. Therefore it is not possible to give these nodes numeric assignments. Use of string (QName) identifiers within anyxml or anydata subtrees may be required.

Appendix B. YANG Identifier Examples

The following simple YANG module is used throughout these examples:

  module example-address {
    namespace "http://example.com/ns/example-address";
    prefix "exaddr";
    revision 2016-08-05;

    container addresses {
      list address {
        key "last first";
        leaf last { type string; }
        leaf first { type string; }
        leaf street { type string; }
        leaf city { type string; }
        leaf zipcode { type string; }
      }
    }
  }

B.1. Hash Collision Repair Example

The following example shows how a hash collision can be avoided using the YANG Hash "rehash" bit.

The following path identifiers and conceptual hash values are assigned for the "example-address" module:

  H1 /example-address:addresses
  H2 /example-address:addresses/address
  H3 /example-address:addresses/address/last
  H4 /example-address:addresses/address/first
  H5 /example-address:addresses/address/street
  H6 /example-address:addresses/address/city
  H7 /example-address:addresses/address/zipcode

The following YANG Identifier values could be assigned using if there are no hash collisions.

  module-id = M
  local-id = Hn

However, if 2 nodes have the same hash value (say H2 and H5) then one of the nodes needs to be renumbered using an manual assignment in the YANG registry. If 16 bit local identifiers are used then values 1 - 32767 are reserved for hash values, and values 32768 - 65535 are reserved for manually assigned rehash mappings.

     { "module" : [
         {
           "module-id" : 17,
           "name" : "example-address",
           "revision" : 0x7e00801,
           "local-type" : "hash",
           "mapping" : [
             {
               "local-id" : 32768,
               "path" : "/example-address:addresses/address/street"
             }
           ]
         }
       ]
     }

B.2. Manual Numbering Example

In this example, manually assigned local-id values are given to the example-address module. Instead of hash values, simple ascending integer values are given.

  1 /example-address:addresses
  2 /example-address:addresses/address
  3 /example-address:addresses/address/last
  4 /example-address:addresses/address/first
  5 /example-address:addresses/address/street
  6 /example-address:addresses/address/city
  7 /example-address:addresses/address/zipcode

If the 'module-id' value '17' is assigned, and the 'local-bits' value is '16', then the following YANG Identifier values would be assigned for these local-id values:

  0x110001 /example-address:addresses
  0x110002 /example-address:addresses/address
  0x110003 /example-address:addresses/address/last
  0x110004 /example-address:addresses/address/first
  0x110005 /example-address:addresses/address/street
  0x110006 /example-address:addresses/address/city
  0x110007 /example-address:addresses/address/zipcode

The example YANG Identifier (YID) file contents:

     { "module" : [
         {
           "module-id" : 17,
           "name" : "example-address",
           "revision" : 0x7e00801,
           "local-type" : "manual",
           "mapping" : [
             {
               "local-id" : 1,
               "path" : "/example-address:addresses"
             },
             {
               "local-id" : 2,
               "path" : "/example-address:addresses/address"
             },
             {
               "local-id" : 3,
               "path" : "/example-address:addresses/address/last"
             },
             {
               "local-id" : 4,
               "path" : "/example-address:addresses/address/first"
             },
             {
               "local-id" : 5,
               "path" : "/example-address:addresses/address/street"
             },
             {
               "local-id" : 6,
               "path" : "/example-address:addresses/address/city"
             },
             {
               "local-id" : 7,
               "path" : "/example-address:addresses/address/zipcode"
             }
           ]
         }
       }
     }

B.3. Augment Numbering Example

In this example, manually assigned local-id values are given to the following "example-phone" module:

  module example-phone {
    namespace "http://example.com/ns/example-phone";
    prefix "exphone";
    import example-address { prefix addr; }
    revision 2016-08-05;

    augment "/addr:addresses/addr:address" {
      container phones {
        list phone {
          key "prefix number";
          leaf prefix { type string; }
          leaf number { type string; }
          leaf type { type phone-type; }
        }
      }
    }
  }

The following local-id mappings are defined in this example.

  1 /example-address:addresses/address/example-phone:phones
  2 /example-address:addresses/address/example-phone:phones\
      /phone
  3 /example-address:addresses/address/example-phone:phones\
      /phone/prefix
  4 /example-address:addresses/address/example-phone:phones\
      /phone/number
  5 /example-address:addresses/address/example-phone:phones\
      /phone/type

If the 'module-id' value '24' is assigned, and the 'local-bits' value is '16', then the following YANG Identifier values would be assigned for these local-id values:

  0x180001 /example-address:addresses/address/example-phone:phones
  0x180002 /example-address:addresses/address/example-phone:phones\
              /phone
  0x180003 /example-address:addresses/address/example-phone:phones\
              /phone/prefix
  0x180004 /example-address:addresses/address/example-phone:phones\
              /phone/number
  0x180005 /example-address:addresses/address/example-phone:phones\
              /phone/type

The example YANG Identifier (YID) file contents are shown below.

     { "module" : [
         {
           "module-id" : 24,
           "name" : "example-phone",
           "revision" : 0x7e00801,
           "local-type" : "manual",
           "mapping" : [
             {
               "local-id" : 1,
               "path" : "/example-address:addresses/address\
                  /example-phone:phones"
             },
             {
               "local-id" : 2,
               "path" : "/example-address:addresses/address\
                  /example-phone:phones/phone"
             },
             {
               "local-id" : 3,
               "path" : "/example-address:addresses/address\
                  /example-phone:phones/phone/prefix"
             },
             {
               "local-id" : 4,
               "path" : "/example-address:addresses/address\
                  /example-phone:phones/phone/number"
             },
             {
               "local-id" : 5,
               "path" : "/example-address:addresses/address\
                  /example-phone:phones/phone/type"
             }
           ]
         }
       }
     }

Appendix C. YANG Hash Examples

The YANG translation of the SMIv2 module specifying the ipNetToMediaTable [RFC4293] yields:


container IP-MIB {
  container ipNetToPhysicalTable {
    list ipNetToPhysicalEntry {
       key "ipNetToPhysicalIfIndex
            ipNetToPhysicalNetAddressType
            ipNetToPhysicalNetAddress";
       leaf ipNetToMediaIfIndex {
          type: int32;
       }
       leaf ipNetToPhysicalIfIndex {
         type if-mib:InterfaceIndex;
       }
       leaf ipNetToPhysicalNetAddressType {
         type inet-address:InetAddressType;
       }
       leaf ipNetToPhysicalNetAddress {
         type inet-address:InetAddress;
       }
       leaf ipNetToPhysicalPhysAddress {
         type yang:phys-address {
            length "0..65535";
         }
       }
       leaf ipNetToPhysicalLastUpdated {
         type yang:timestamp;
       }
       leaf ipNetToPhysicalType {
         type enumeration { ... }
       }
       leaf ipNetToPhysicalState {
         type enumeration { ... }
       }
       leaf ipNetToPhysicalRowStatus {
         type snmpv2-tc:RowStatus;
       }
    }
 }

The YANG hash values for 'ipNetToPhysicalEntry' and its child nodes are calculated by constructing the schema node identifier for the objects, and then calculating the 30 bit murmur3 hash values (shown in parenthesis):

   /IP-MIB:IP-MIB/ipNetToPhysicalTable (0x0aba15cc)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry
      (0xo6aaddbc)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalIfIndex (0x346b3071)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalNetAddressType (0x3650bb64)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalNetAddress (0x06fd4d91)
   /IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalPhysAddress (0x26180bcb)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalLastUpdated (0x3d6bbe90)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalType (0x35ecbb3d)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalState (0x13038bb5)
   /IP-MIB:IP-MIB/ipNetToPhysicalTable/ipNetToPhysicalEntry\
      /ipNetToPhysicalRowStatus (0x09e1fa37)

The example YANG Identifier (YID) file contents are shown below. Note that there are no hash collisions so there are no local identifiers assigned. This is expected to be the typical case.

     { "module" : [
         {
           "module-id" : 25,
           "name" : "IP-MIB",
           "revision" : 0x7d60202,
           "local-type" : "hash"
         }
       ]
     }

If the 'module-id' value '25' is assigned, and the 'local-bits' value is '16', then the following YANG Identifier values would be assigned for these local-id values (only the terminal node is shown since they are unique in SMIv2). Note that only 15 bits of the 30 bit hash value are used. The 16th bit is reserved to identify manually assigned rehash entries.

   0x1915cc  ipNetToPhysicalTable 
   0x195dbc  ipNetToPhysicalEntry
   0x193071  ipNetToPhysicalIfIndex 
   0x193b64  ipNetToPhysicalNetAddressType
   0x194d91  ipNetToPhysicalNetAddress 
   0x190bcb  ipNetToPhysicalPhysAddress 
   0x193e90  ipNetToPhysicalLastUpdated 
   0x193b3d  ipNetToPhysicalType 
   0x190bb5  ipNetToPhysicalState
   0x197a37  ipNetToPhysicalRowStatus 

Authors' Addresses

Andy Bierman YumaWorks, Inc. 685 Cochran St. Suite #160 Simi Valley, CA 93065 USA EMail: andy@yumaworks.com
Peter van der Stok consultant Phone: +31-492474673 (Netherlands), +33-966015248 (France) EMail: consultancy@vanderstok.org URI: www.vanderstok.org