| Interface to the Routing System (i2rs) | E. Voit |
| Internet-Draft | A. Clemm |
| Intended status: Informational | A. Gonzalez Prieto |
| Expires: March 31, 2016 | Cisco Systems |
| September 28, 2015 |
Requirements for Subscription to YANG Datastores
draft-ietf-i2rs-pub-sub-requirements-03
This document provides requirements for a service that allows client applications to subscribe to updates of a YANG datastore. Based on criteria negotiated as part of a subscription, updates will be pushed to targeted recipients. Such a capability eliminates the need for periodic polling of YANG datastores by applications and fills a functional gap in existing YANG transports (i.e. Netconf and Restconf). Such a service can be summarized as a “pub/sub” service for YANG datastore updates. Beyond a set of basic requirements for the service, various refinements are addressed. These refinements include: periodicity of object updates, filtering out of objects underneath a requested a subtree, and delivery QoS guarantees.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 31, 2016.
Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
YANG has gained acceptance as the data definition language of choice for control and management related information. Applications that interact with YANG datastores are extending beyond traditional configuration of network elements. In many cases these applications are aimed at service-assurance, which involves monitoring of operational data and state. The existing YANG technology ecosystem is proving insufficient for those applications due to:
Put simply, periodic fetching of data is not an adequate solution for applications requiring frequent or prompt updates of remote object state. Trying to impose a polling based solution to this problem imposes load on networks, devices, and applications. Additionally, polling solutions are brittle in the face of communication glitches, and they have limitations in their ability to synchronize and calibrate retrieval intervals across a network.
I2RS WG documents have expressed a need for more robust YANG object subscriptions. Similar discussions are underway in NETMOD and NETCONF. With the support of standards bodies such as OMG (DDS), XMPP.org standard, generic Pub/Sub mechanisms to communicate data updates have been defined and proven themselves in a wide variety of deployments.
It is time to incorporate such generic object subscription mechanisms as part of Network Elements, and allow these mechanisms to be applied in the context of data that is conceptually contained in YANG datastores. With such mechanisms, both controller and local Network Element based applications can have access to a set of consistent network information driven via push from peer Network Elements which host authoritative information.
There are some valid IETF starting points and contexts for these mechanisms. For example NETCONF Event Notifications [RFC5277] provides a useful tool for an end-to-end solution. However RFC5277 does not follow the Pub/Sub paradigm, does not allow the explicit deletion of subscriptions, and predates YANG. Predating YANG is an issue, as monitoring and filtering based on YANG subtrees becomes problematic . [RFC6470] defines configuration change notifications, but doesn’t provide the actual configuration change.
Because of this, the authors have put forward this requirements document as well as [datastore-push]. We believe these provide a context upon which to create new solution. It is intended that these documents include requirements and provide technologies applicable beyond I2RS.
For decades, information delivery of current network state has been accomplished either by fetching from operations interfaces, or via dedicated, customized networking protocols. With the growth of SDN, imperative policy distribution, and YANG's ascent as a dominant programmatic interface to network elements, this mixture of fetch plus custom networking protocols is no longer sufficient. What is needed is a push mechanism that is able to deliver objects and object changes as they happen.
These push distribution mechanisms will not replace existing networking protocols. Instead they will supplement these protocols, providing different response time, peering, scale, and security characteristics.
At the same time, SNMP and MIBs are still widely deployed and the defacto choice for many monitoring solutions. Those solutions do not require support for configuration transactions and the need to validate and maintain configuration consistency, hence there is less pressure to abandon SNMP and MIBs. Arguably the biggest shortcoming of SNMP for those applications concerns the need to rely on periodic polling, because it introduces additional load on the network and devices, is brittle in case polling cycles are missed, and is hard to synchronize and calibrate across a network, making data obtained from multiple devices less comparable. If applications need to apply those same interaction patterns for YANG datastores, similar issues can be expected. Migration to YANG datastores by applications that do not have to worry about transactional integrity becomes a lot more compelling if those issues are addressed.
Various I2RS documents highlight the need to provide Pub/Sub capabilities between network elements. From [i2rs-arch], there are references throughout the document beginning in section 6.2. Some specific examples include:
In addition [i2rs-usecase]has relevant requirements. A small subset includes:
And [i2rs-traceability] has Pub/Sub requirements listed in Section 7.4.3.
There are additional individual drafts such as [i2rs-pubsub-security] documenting the Pub/Sub needs for: time delivery sensitivity, support for multiple transport protocols, secure/authorized communications, and support for a range specification of subscribed data delivery content. So the list above should not be considered exhaustive.
This document is intended to cover requirements beyond I2RS. Looking at history, there are many examples of switching and routing protocols which have done explicit or implicit pub/sub in the past. In addition, new policy notification mechanisms which operate on Switches and Routers are being specified now. A very small subset of these includes:
Worthy of note in the list above is the wide variety of broadcast, multicast, and unicast transports used. In addition some transports are at L3, and some at L2. Therefore if we are going to attempt a generic Pub/Sub mechanism, it will need to be structured so that it may support alternative transports. Looking at the nearer term based on current I2RS requirements, NETCONF should be our transport starting point as it supports connection oriented/Unicast communication. But we need to be prepared to decouple where viable to support Multicast and Broadcast distribution as well.
TIBCO, RSS, CORBA, and other technologies all show precursor Pub/Sub technologies. However there are new needs described in Section 4 below which these technologies do not serve. We need a technology.
There are at least two widely deployed generalized pub/sub implementations which come close to current needs: XMPP[XEP-0060] and DDS[OMG-DDS]. Both serve as proof-points that a highly scalable distributed datastore implementation connecting millions of edge devices is possible.
Because of these proof points, we can be comfortable that the underlying technologies can enable reusable generalized YANG object distribution. Analysis will need to fully dimension the speed and scale of such object distribution for various subtree sizes and transport types.
A Subscriber makes requests for set(s) of YANG object data. The Subscriber is the owner of the Subscription.
A Publisher is responsible for distributing subscribed YANG object data per the terms of a Subscription. In general, a Publisher is the owner of the YANG datastore that is subjected to the Subscription.
A Receiver is the target where a Publisher pushes updates. In general, the Receiver and Subscriber will be the same entity. A Subscription Service provides Subscriptions to Subscribers of YANG data.
A Subscription Service interacts with the Publisher of the YANG data as needed to provide the data per the terms of the Subscription.
A Subscription Request for one or more YANG subtrees (including single leafs) made by the Subscriber of a Publisher and targeted to a Receiver. A Subscription may include constraints which dictate how often or under what conditions YANG information updates might be sent.
A Subscription is a contract between a Subscription Service and a Subscriber that stipulates the data to be pushed and the associated terms.
A YANG datastore is a conceptual datastore that contains hierarchical data defined in YANG data models. It is what is referred in existing RFCs as “NETCONF datastore”. However, as the same datastore is no longer tied to NETCONF as a specific transport, the term “YANG datastore” is deemed more appropriate.
An Update provides object changes which have occurred within subscribed YANG subtree(s). An Update must include the current status of (data) node instances which according to any filtering are reportably different from the previously provided state. An Update may include a bundled set of ordered/sequential changes for a given object which have been made since the last update.
A Filter contains evaluation criteria which are evaluated against YANG object(s) within a Subscription. There are two types of Filters: Subtree Filters which identify selected objects/nodes published under a target data node, and object Property Filters where an object should only be published if it has propert(ies) meeting specified Filter criteria. For “on-change” notifications, passing through the Filter requires that a subscribed object is now different that from the previous Push, AND at least one of the YANG objects being evaluated has changed since the last Update.
Many of the requirements within this section have been morphed from OMG’s DDS and XMPP.org’s requirements specifications.
This document provides requirements for the Subscription Service. It does not define all the requirements for the Subscriber/Receiver. However In order to frame the desired behavior of the Subscription Service, it is important to specify key input constraints.
A Subscriber should avoid attempting to establish multiple Subscriptions pertaining to the same information, i.e. referring to the same datastore YANG subtrees.
A Subscriber may provide Subscription QoS criteria to the Subscription Service such that if the Subscription Service is unable to meet those criteria, the Subscription should not be established.
When a Subscriber needs to restart, it is acceptable for the Subscriber to have to resubscribe. There is no requirement for the life span of the Subscription to extend beyond the life span of the Subscriber.
A Subscriber must be able to infer when a Subscription Service is no longer active and when no more updates are being sent.
A Subscriber may check with a Subscription Service to validate the existence and monitored subtrees of a Subscription.
A Subscriber must be able to periodically lease and re-lease a Subscription from a Subscription Service.
A Subscription Service must support the ability to create, renew, timeout, and terminate a Subscription.
A Subscription Service must be able to support and independently track one or more Subscription Requests by the same Subscriber.
A Subscription Service must be able to support an add/change/delete of one or more YANG subtrees as part of the same Subscription Request.
A Subscription Service must support Subscriptions against operational datastores, configuration datastores, or both.
A Subscription Service must be able support a Subtree Filter so that subscribed updates under a target node might publish only operational data, only configuration data, or both.
A Subscription may include filters as defined within a Subscription Request, the Subscription Service must publish only data nodes that meet the filter criteria.
A Subscription Service must support the ability to subscribe to periodic updates. The subscription period must be configurable as part of the subscription request.
A Subscription Service should support the ability to subscribe to updates “on-change”, i.e. whenever values of subscribed data objects change.
For “on-change” updates, the Subscription Service must support a dampening period that needs to pass before the first or subsequent “on-change” updates are sent. The dampening period should be configurable as part of the subscription request.
A Subscription Service must allow Subscriptions to be monitored. Specifically, a Subscription Service must at a minimum maintain information about which Subscriptions are being serviced, the terms of those subscriptions (e.g. what data is being subscribed, associated filters, update policy – on change, periodic), and the overall status of the Subscription – e.g. active or suspended.
A Subscription Service should be able to interpret Subscription QoS parameters, and only establish a Subscription if it is possible to meet the QoS needs of the provided QoS parameters.
A Subscription Service must support terminating of a Subscription when requested by the Subscriber.
A Subscription Service should support the ability to suspend and to resume a Subscription on request of a client.
A Subscription Service may at its discretion revoke or suspend an existing subscription. Reasons may include transitory resource limitation, credential expiry, failure to reconfirm a subscription, loss of connectivity with the Receiver, operator CLI, and/or others. When this occurs, the Subscription Service must notify the Subscriber and update subscription status.
A Subscription Service may offer the ability to modify a subscription filter. If such an ability is offered, the service must provide subscribers with an indication at what point the modified subscription goes into effect.
A Subscription Service must be able to negotiate the following terms of a Subscription:
A Subscription Service should be able to negotiate QoS criteria for a Subscription. Examples of Subscription QoS criteria may include reliability of the Subscription Service, reaction time between a monitored YANG subtree/object change and a corresponding notification push, and the Subscription Service's ability to support certain levels of object liveliness.
In cases where a Subscription Request cannot be fulfilled, the Subscription Service must include in its decline a set of criteria that would have been acceptable when the Subscription Request was made. For example, if periodic updates were requested with too short update intervals for the specified data set, the minimum acceptable interval period should be included. If on-change updates were requested with a dampening period, the minimum acceptable dampening period should be included, or an indication whether only periodic updates are supported along with the minimum acceptable interval period for the data set being subscribed to.
For “on-change” updates, the Subscription Service must only send deltas to the object data for which a change occurred. [Otherwise the subscriber will not know what has actually undergone change.] The updates for each object needs to include an indication whether it was removed, added, or changed.
When a Subscription Service is not able to send updates per its subscription contract, the Subscription must notify subscribers and put the subscription into a state of indicating the Subscription was suspended by the service. When able to resume service, subscribers need to be notified as well. If unable to resume service, the Subscription Service may terminate the subscription and notify Subscribers accordingly.
When a Subscription with “on-change” updates is suspended and then resumed, the first update should include updates of any changes that occurred while the Subscription was suspended, with the current value. The Subscription Service must provide a clear indication when this capability is not supported (because in this case a client application may have to synchronize state separately).
Multiple objects being pushed to a Subscriber, perhaps from different Subscriptions, should be bundled together into a single Update.
The sending of an Update must not be delayed beyond the Push Latency of any enclosed object changes.
The sending of an Update must not be delayed beyond the dampening period of any enclosed object changes.
The sending of an Update must not occur before the dampening period expires for any enclosed object changes.
A Subscription Service may, as an option, support a persistence/replay capability.
A Subscription Service should support different transports.
A Subscription Service should support different encodings of payload.
It must be possible for Receivers to associate the update with a specific Subscription.
In the case of connection-oriented transport, when a transport connection drops, the associated Subscription should be terminated. It is up the Subscriber to request a new Subscription.
As part of the Subscription establishment, there must be mutual authentication between the Subscriber and the Subscription Service.
When there are multiple Subscribers, it should be possible to provide cryptographic authentication in such a way that no Subscriber can pose as the original Subscription Service.
Versioning must be supported.
A Subscription could be used to attempt to retrieve information that a client has not authorized access to. Therefore it is important that data pushed based on Subscriptions is authorized in the same way that regular data retrieval operations are. Data being pushed to a client must be filtered accordingly, just like if the data were being retrieved on-demand. For Unicast transports, the NETCONF Authorization Control Model applies.
Additions or changes within a subscribed subtree structure must be validated against authorization methods before Subscription Updates including new subtree information are pushed.
A loss of authenticated access to subtree or node SHOULD be communicated to the Subscriber
Subscription requests, including requests to create, terminate, suspend, and resume Subscriptions must be properly authorized.
When the Subscriber and Receiver are different, the Receiver must be able to terminate any Subscription to it where objects are being delivered over a Unicast transport.
A Subscription Service should decline a Subscription Request if it would deplete its resources. It is preferable to decline a Subscription when originally requested, rather than having to terminate it prematurely later.
A Subscription Service should be able to negotiate the following Subscription QoS parameters with a Subscriber: Dampening, Reliability, Deadline, Bundling.
A Subscription Service must be able to respond to requests to verify the Liveliness of a subscription.
A Subscription Service must be able to report the currently monitored Nodes of a Subscription.
A Subscription Service must be able to negotiate the minimum time separation since the previous update before transmitting a subsequent update for Subscription. (Note: this is intended to confine the visibility of volatility into something digestible by the receiver.)
A Subscription Service may send Updates over Best Effort and Reliable transports.
Every update to a subscribed object must be sent to the Receiver in sequential order.
The Subscription Service should have the ability to bundle a set of discrete object notifications into a single publishable update for a Subscription. A bundle may include information on different Data Nodes and/or multiple updates about a single Data Node.
For any bundled updates, the Subscription Service must provide information for a Receiver to reconstruct the order and timing of updates.
The Subscription Service must be able to push updates at a regular cadence that corresponds with Subscriber specified start and end timestamps. (Note: the regular cadence can drive one, a discrete quantity, or an unbounded set of periodic updates.)
The Subscription Service should be able to delay Updates on object push for a configurable period per Subscriber.
It must be possible for an administrative entity to determine the Push latency between object change in a monitored subtree and the Subscription Service Push of the update transmission.
If no filtering criteria are provided, or if filtering criteria are met, updates for a subscribed object must be pushed, subject to the QoS limits established for the subscription.
It must be possible for the Subscription Service to receive Filter(s) from a Subscriber and apply them to corresponding object(s) within a Subscription.
It must be possible to attach one or more Subtree and/or Property Filters to a subscription. Mandatory Property Filter types include:
It should be possible for Property Filtering criteria to evaluate more than one property of a particular subscribed object as well as apply multiple filters against a single property.
It should be possible to establish query match criteria on additional objects to be used in conjunction with Property Filtering criteria on a subscribed object. (For example: if A has changed AND B=1, then Push A.) (Note: Query match capability may be done on objects within the datastore even if those objects are not included within the subscription. This of course assumes the subscriber has read access to those objects.)
It must be possible to fetch the state of a single subscription from a Subscription Service.
It must be possible to fetch the state of all subscriptions of a particular Subscriber.
It must be possible to fetch a list and status of all Subscription Requests over a period of time. If there us a failure, some failure reasons might include:
We wish to acknowledge the helpful contributions, comments, and suggestions that were received from Ambika Tripathy and Prabhakara Yellai as well as the helpfulness of related end-to-end system context from [i2rs-pubsub-security] from Nancy Cam Winget, Ken Beck, and David McGrew.
| [RFC2328] | Moy, J., "OSPF Version 2", STD 54, RFC 2328, DOI 10.17487/RFC2328, April 1998. |
| [RFC5277] | Chisholm, S. and H. Trevino, "NETCONF Event Notifications", RFC 5277, DOI 10.17487/RFC5277, July 2008. |
| [RFC6470] | Bierman, A., "Network Configuration Protocol (NETCONF) Base Notifications", RFC 6470, DOI 10.17487/RFC6470, February 2012. |
| [RFC6513] | Rosen, E. and R. Aggarwal, "Multicast in MPLS/BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February 2012. |