| Network Working Group | B. Claise |
| Internet-Draft | J. Clarke |
| Updates: 7950 (if approved) | Cisco Systems, Inc. |
| Intended status: Standards Track | B. Lengyel |
| Expires: June 18, 2018 | Ericsson |
| K. D'Souza | |
| AT&T | |
| December 15, 2017 |
New YANG Module Update Procedure
draft-clacla-netmod-yang-model-update-03
This document specifies a new YANG module update procedure in case of backward-incompatible changes, as an alternative proposal to the YANG 1.1 specifications. This document updates RFC 7950.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 18, 2018.
Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
The YANG data modeling language [RFC7950] specifies strict rules for updating YANG modules (see section 11 "Updating a Module"). Citing a few of the relevant rules:
What are the consequences?
This section lists a series of problems, hopefully listed in a logical order, which leads to the solution in the next section.
The conclusions drawn in the introduction lead to the logical conclusion that the standardized YANG modules have to be perfect on day one (at least the structure), which in turn might explain why all the IETF YANG modules take so long to standardize. Shooting for perfection (at least in structure) is obviously a noble goal, but if the perfect standard comes too late, it doesn't help the industry.
As we learn from our mistakes, we're going to face more and more backward-incompatible YANG modules. An example is the YANG data model for L3VPN service delivery [RFC8049], which, based on implementation experience, must be updated in a backward-incompatible way with draft-wu-l3sm-rfc8049bis [I-D.wu-l3sm-rfc8049bis].
While Standards Development Organization (SDO) YANG modules are obviously better for the industry, we must recognize that many YANG modules are actually generated YANG modules (for example, from internal databases), also known as native YANG modules, or vendor modules [RFC8199]. From time to time, the new YANG modules are not backward-compatible.
In such cases, it would be better to indicate how backward-compatible a given YANG module actually is.
Sometimes small errors force us to make non-backward compatible updates. As an example imagine that we have a string with a complex pattern (e.g., an IP address). Let's assume the initial pattern incorrectly allows IP addresses to start with 355. In the next version this is corrected to disallow addresses starting with 355. Formally this is an non-backward compatible change as the value space of the string is decreased. In reality an IP address and the implementation behind it was never capable of handling an address starting with 355. So practically this is a backward compatible change, just like a correction of the description statement. Still current YANG rules would force a module name change.
Even if we focus on the IETF, we have to observe that many SDOs, opensource fora, and vendors develop YANG modules. This should be considered a success for an IETF developed technology. However, the operators are faced with this problem: how to select the YANG modules to take into account for their service developments.
The site <https://www.yangcatalog.org> (and the YANG catalog that it provides: YANG module for yangcatalog.org, [I-D.clacla-netmod-model-catalog]) is an attempt to become a reference for all YANG modules available in the industry, for both YANG developers to search on what exists already) and for operators (to discover the more mature YANG models to automate services). This YANG catalog should not only contain pointers to the YANG modules themselves, but also contain metadata related to those YANG modules: What is the module type (service model or not?); what is the maturity level? (e.g., for the IETF: is this an RFC, a working group document or an individual draft?); is this module implemented?; who is the contact?; is there open-source code available? And we expect many more in the future. The industry has begun to understand that the metadata related to YANG models become equally important as the YANG models themselves.
The yangcatalog.org instantiation of the catalog provides a means for module authors and vendors implementing modules to upload their metadata, which is then searchable via an API, as well as using a variety of web-based tools. The instructions for contributing and searching for metadata can be found at <https://www.yangcatalog.org/contribute.php>.
The issue is actually the number of YANG modules the operators are offered. At the time of writing this document, the number of unique YANG modules in the catalog is exactly 2596 (and that number keeps growing), while the IETF has standardized or is busy standardizing a small subset of those. Therefore, it's important to distinguish the relevant YANG modules with the pack and to understand the relationship between the YANG modules.
So the operators use the yangcatalog.org to discover which YANG modules they can use NOW. They base their selection not only on the YANG module content, but also on the related metadata. When faced with the zoo of the YANG modules, it's difficult to understand the relationship between YANG modules. As an example: how could an operator discover that YANG-MODULE-B obsoletes YANG-MODULE-A? Indeed, both have different YANG module names. The only available information is an "obsolete" tag in the published RFC containing YANG-MODULE-B: this tag would point to YANG-MODULE-A. In the world of automation, going through a published RFC as a level of indirection to understand the YANG module obsolete relationship is a non-starter. Food for thought: the IETF should stop thinking that the metric for success is an RFC number, as opposed to the contained YANG module(s).
We need an automatic way to discover that a YANG-MODULE-B obsoletes YANG-MODULE-A, so that YANG-MODULE-A should not be given any attention.
The following example is not an automatic way.
description
"This YANG module defines a generic service configuration
model for Layer 3 VPNs. This model is common across all
vendor implementations. This obsoletes the RFC8049 YANG
module, ietf-l3vpn-svc@2017-01-2";
revision 2017-09-14 {
description
"First revision of RFC8049.";
reference
"RFC xxxx: YANG Data Model for L3VPN Service Delivery";
Along the same lines, while going through an out-of-band tool such as the yangcatalog.org in order to discover the obsolete relationship is a possible automatic way, it is not ideal.
Let's assume for a moment that we change the YANG module, with the specific example of ietf-routing, which some propose to update to ietf-routing-2.
Here are all the ietf-routing dependent YANG modules (those modules that depend on ietf-routing) <https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-routing&recurse=0&rfcs=1&show_subm=1&show_dir=dependents>. So many YANG modules.
Let's look at the difference for ietf-routing-2: <https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-routing-2&recurse=0&rfcs=1&show_subm=1&show_dir=dependents>.
Changing the module name from ietf-routing to ietf-routing-2 implies that the we have to warn all draft authors of ietf-routing YANG dependent modules. First, to make sure they are aware of ietf-routing-2 (publishing a RFC8022bis mentioning in the module description that this module is not compatible with the NMDA architecture, and providing a pointer to ietf-routing-2 ... is not an automatic way... so barely useful). And second, to ask them to change their import (or service composition) to ietf-routing-2. Hopefully, in the ietf-routing case, most dependent YANG modules are part of the IETF, so the communication is a manageable. For the already existing dependent vendor modules the problem is worse.
Changing the ietf-interfaces YANG module name would be a different challenge, as it's used throughout the industry: <https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-interfaces&recurse=0&rfcs=1&show_subm=1&show_dir=dependents>
As described in the previous sections, there is a need to allow non-backward compatible changes without changing a module's name. This would avoid many of the above problems. In most cases even after non-backward compatible updates a module should keep its name. However, for really major changes renaming the module is still the proper way to go:
Allowing non-backward compatible changes to happen without a module name change will decrease the number of separate modules to handle and will make it a trivial task to track these non-backward compatible changes.
The current definition of deprecated and obsolete in [RFC7950] (as quoted below) is problematic and should be corrected.
YANG is considered an interface contract between the server and the client. The current definitions of deprecated and obsolete mean that a schema node that is either deprecated or obsolete may or may not be implemented. The client has no way to find out which is the case except for by trying to write or read data at the leaf in question. This probing would need to be done for each separate data-node, which not a trivial thing to do. This "may or may not" is unacceptable in a contract. In effect, this works as if there would be an if-feature statement on each deprecated schema node where the server does not advertise whether the feature is supported or not. Why is it not advertised?
A management system, SDN controller or any other user of a module should be capable of easily determining the compatibility between two module versions. Higher level logic for a network function, something that can not be implemented in a purely model driven way, is always dependent on a specific version of the module. If the client finds that the module has been updated on the network node, it has to decide if it tries to handle it as it handled the previous version of the model or if it just stops, to avoid problems. To make this decision the client needs to know if the module was updated in a backward compatible way or not.
This is not possible to decide today because of the following:
Finding status changes or violations of update rules need a line by line comparision of the old and new modules, no easy task.
If a schema part is considered old/bad we need to be able to give advance warning that it will be removed. As this is an advance warning the part shall still be present and usable in the current revision; however, it will be removed in one of the next revisions. We need the advance warning to allow users of the module time enough to plan/execute migration away from the deprecated functionality. Often deprecation will be accompanied by information whether the functionality will just disappear or that there is an alternative, possibly more advanced solution that should be used.
Vendors use such warnings often, but the NMDA related redesign of IETF modules is also an example where it would be useful. (As another example see the usage of deprecated in the Java programing language.) The current definition of the deprecated status does not serve this purpose as described in Section 2.8. The definition of "deprecated" in the status statement shall be changed to address this issue.
The solution is composed of four parts, a semantic versioning YANG extension, updates to the YANG 1.1. status statement and module update rules and the import by version statement. An optional additional check, validating the semantic versioning from a syntactic point of view, can either assist in determining the correct semantic versioning values, or can help in determining the values for YANG modules that don't support this extension.
The semantic versioning solution proposed here has already been proposed in [I-D.openconfig-netmod-model-catalog] (included here with the authors permission)which itself is based on [openconfigsemver]. The goal is to indicate the YANG module backwards (in)compatibility, following semver.org semantic versioning [semver]:
"The SEMVER version number for the module is introduced. This is expressed as a semantic version number of the form: x.y.z
Along these lines, we propose the following YANG 1.1 extension for a more generic semantic version. The formal definition is found at the end of this document.
extension module-version {
argument "semver" {
yin-element false;
}
}
The extension would typically be used this way:
module yang-module-name {
namespace "name-space";
prefix "prefix-name";
import ietf-semver { prefix "semver"; }
description
"to be completed";
revision 2017-10-30 {
description
"Change the module structure";
semver:module-version "2.0.0";
}
revision 2017-07-30 {
description
"Added new feature XXX";
semver:module-version "1.2.0";";
}
revision 2017-04-03 {
description
"Update copyright notice.";
semver:module-version "1.0.1";;
}
revision 2017-04-03 {
description
"First release version.";
semver:module-version "1.0.0";;
}
revision 2017-01-26 {
description
"Initial module for inet types";
semver:module-version "0.1.0";;
}
//YANG module definition starts here
See also "Semantic Versioning and Structure for IETF Specifications" [I-D.claise-semver] for a mechanism to combine the semantic versioning, the github tools, and a potential change to the IETF process.
RFC 7950 section 11, must be updated to change the definition of deprecated and obsolete. In both cases the client must be able to determine whether the relevant parts are implemented or not without probing. The following definition is proposed:
If there is a need to allow the server to decide whether a deprecated/obsolete schema part is implemented YANG already has a facility for that: the feature statement. Use it!
RFC 7950 section 11, must be updated to express:
"As experience is gained with a module, it may be desirable to revise that module. Changes to published modules are allowed, even if they have some potential to cause interoperability problems, if the module-version YANG extension is used in the revision statement to clearly indicate the nature of the change."
The YANG catalog contains not only the most up-to-date YANG module revision of a given module, but keeps all previous revisions as well. With APIs in mind, it's important to understand whether different YANG module revisions are backward compatible (this is specifically imported for native YANG modules, i.e. the ones where generated-from = native), even for the YANG modules that don't support the YANG extension specified in this document.
Two distinct leaves in the YANG module [I-D.clacla-netmod-model-catalog] contain this semver notation:
Note that the absolute numbers in the semantic-version and derived-semantic-version are actually meaningless by themselves. That is, one must compare two different semver values for a given module to understand the compatibility between them.
If a module is imported by another one, it is usually not specified which version of the imported module should be used. However not all versions may be acceptable. Today YANG 1.1 allows us to specify the revision date of the imported module, but that is too specific, as even a small spelling correction of the imported module results in a change to its revision date, thus making the module revision ineligible for import.
Using semantic versioning to indicate the acceptable imported module versions is much more flexible.
The module-version statement SHOULD be a substatement of the import statement. An import statement MUST NOT contain both a module-version and a revision substatement. The use of the revision substatement of import should be discouraged/deprecated.
There are a number of open issues to be disccused. These include the following:
The extension described in this module is defined in the YANG module below.
<CODE BEGINS> file "ietf-semver@2017-12-15.yang"
module ietf-semver {
yang-version 1.1;
namespace "urn:ietf:params:xml:ns:yang:ietf-semver";
prefix semver;
organization
"IETF NETMOD (Network Modeling) Working Group";
contact
"WG Web: <https://datatracker.ietf.org/wg/netmod/>
WG List: <mailto:netmod@ietf.org>
Author: Benoit Claise
<mailto:bclaise@cisco.com>
Author: Joe Clarke
<mailto:jclarke@cisco.com>
Author: Kevin D'Souza
<mailto:kd6913@att.com>
Author: Balazs Lengyel
<mailto:balazs.lengyel@ericsson.com>";
description
"This module contains a definition for a YANG 1.1 extension to
express the semantic version of YANG modules.";
revision 2017-12-15 {
description
"Initial revision.";
reference "draft-clacla-netmod-yang-model-update:
New YANG Module Update Procedure";
semver:module-semver 0.1.1;
}
extension module-version {
argument semver;
description
"The version number for the module revision it is used in.
This is expressed as a semantic version string in the form:
x.y.z
where:
* x corresponds to the major version,
* y corresponds to a minor version,
* z corresponds to a patch version.
A major version number of 0 indicates that this model is still
in development, and is potentially subject to change.
Following a release of major version 1, all modules will
increment major revision number where backwards incompatible
changes to the model are made.
The minor version is changed when features are added to the
model that do not impact current clients use of the model.
When major version is stepped, the minor version is reset to 0.
The patch-level version is incremented when non-feature changes
(such as bugfixes or clarifications to human-readable
descriptions that do not impact model functionality) are made
that maintain backwards compatibility.
When major or minor version is stepped, the patch-level is
reset to 0.
The version number is stored in the module meta-data.
By comparing the module-version between two revisions of a
given module, one can know if revision N+1 is backwards
compatible or not relative to revision N, as well as
whether or not new features have been added to revision N+1.
If a module contains this extension it indicates that for this
module the updated status and update rules as this described in
RFC XXXX are used.
The statement MUST only be a substatement of the revision,
import or include statements.
Zero or One module-version statement is allowed per parent
statement. NO substatements are allowed.
";
reference "http://semver.org/ : Semantic Versioning 2.0.0";
}
augment /yanglib:modules-state/yanglib:module {
leaf module-version {
type string {
pattern "[0-9]+.[0-9]+.[0-9]+";
}
}
}
augment /yanglib:modules-state/yanglib:module/yanglib:submodule {
leaf submodule-version {
type string {
pattern "[0-9]+.[0-9]+.[0-9]+";
}
}
}
}
<CODE ENDS>
The document does not define any new protocol or data model. There are no security impacts.
No IANA action is requested.
| [RFC7950] | Bjorklund, M., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016. |
| [I-D.clacla-netmod-model-catalog] | Clarke, J. and B. Claise, "YANG module for yangcatalog.org", Internet-Draft draft-clacla-netmod-model-catalog-02, October 2017. |
| [I-D.claise-semver] | Claise, B., Barnes, R. and J. Clarke, "Semantic Versioning and Structure for IETF Specifications", Internet-Draft draft-claise-semver-01, July 2017. |
| [I-D.openconfig-netmod-model-catalog] | Shaikh, A., Shakir, R. and K. D'Souza, "Catalog and registry for YANG models", Internet-Draft draft-openconfig-netmod-model-catalog-02, March 2017. |
| [I-D.wu-l3sm-rfc8049bis] | Wu, Q., Litkowski, S., Tomotaki, L. and K. Ogaki, "YANG Data Model for L3VPN Service Delivery", Internet-Draft draft-wu-l3sm-rfc8049bis-11, December 2017. |
| [openconfigsemver] | "Semantic Versioning for Openconfig Models" |
| [RFC8049] | Litkowski, S., Tomotaki, L. and K. Ogaki, "YANG Data Model for L3VPN Service Delivery", RFC 8049, DOI 10.17487/RFC8049, February 2017. |
| [RFC8199] | Bogdanovic, D., Claise, B. and C. Moberg, "YANG Module Classification", RFC 8199, DOI 10.17487/RFC8199, July 2017. |
| [semver] | "Semantic Versioning 2.0.0" |