NFSv4 D. Noveck
Internet-Draft March 06, 2014
Intended status: Informational
Expires: September 07, 2014

NFS Protocol Extension: Retrospect and Prospect
draft-dnoveck-nfs-extension-01

Abstract

This document surveys the processes by which the NFS protocol has been extended in the past and considers how the mechanisms by which the protocol is extended might best be modified in the future.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on September 07, 2014.

Copyright Notice

Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

This document examines the subject of protocol extension within the NFS family of protocols. In order to better understand the issues that exist going forward with NFSv4, we examine the history of protocol extension throughout the development of NFS including both the pre-IETF period and the development of successive NFSv4 minor versions.

We then use this history as a basis upon which to explore the issues involved in providing a modified extension paradigm that builds on the work already done, but is more flexible.

2. Protocol Extension

Often, protocols require means by which they can be extended. Such extension may be needed to meet new requirements, to correct protocol weaknesses exposed by experience, or even to correct protocol bugs (as can happen when protocols are published as RFC's without fully fleshed-out implementations).

We need to distinguish here between protocol "extension" and "versioning". Versioning is a form of protocol extension but not every form of protocol extension can be accommodated within a versioning paradigm.

When a versioning paradigm is in place, groups of extensions are conceived of as ordered, allowing extensions in subsequent versions to build upon those in previous versions. When multiple extensions are combined into a single version, each of the extensions may be built assuming that the others will be present as well. In such cases, there can be the opportunity to make design changes in the protocol, allowing elements of the protocol to be restructured, sometimes in major ways.

When a versioning paradigm is in effect and extensions are optional, extensions cannot build upon one another, since the presence of any particular extension cannot be assumed. In such cases, the ability to restructure the protocol is reduced, but smaller changes may be introduced more easily.

In this latter case, it is not clear that the word "versioning" is appropriate. Nevertheless, in this document, we will, as in the phrase "NFSv4 minor versioning" use the existing terminology without necessarily subscribing to the view that "versioning" is the appropriate description.

3. Protocol Extension Mechanisms

Some factors that are often relevant in deciding on the means by which a protocol will be extended.

While it is possible to use different sorts of extension mechanisms for different sorts of extensions, protocols typically do not take advantage of that flexibility.

On the other hand, protocols do, as NFS has done, change their preferred extension mechanisms in response to long-term changes in requirements. However, once having done so, they rarely switch back. Changing extension mechanisms is a big step, both conceptually and in implementation terms, and is not frequently repeated.

3.1. Specific Protocol Mechanisms Designed for Extension

Often, protocols will be designed with specific mechanisms, designed to allow protocol extension. An example is the provision for TCP options (see [RFC0793] and [RFC2780].) Most often, such mechanisms are designed to allow individual extensions to be designed and implemented independently, with any dependency relations between extensions specified separately and not enforced by the extension mechanism itself.

3.2. Protocol Extension by XDR Replacement

RPC-based protocols may, and often do, provide for protocol extension by replacing the XDR for one version with that for another and using the RPC versioning mechanism to manage selection of the proper protocol variant. The use of the RPC versioning mechanism enforces a versioning paradigm of this sort on protocols using this extension mechanism.

This extension mechanism allows very extensive protocol changes, up to and including the replacement of one protocol by an entirely different one. For some kinds of protocol extensions, this seems the only way to effect the change.

3.3. Protocol Extension by XDR Extension

It is possible to replace an XDR definition by one which is an extension in the sense that

Within an XDR/RPC framework, extensions can be arrived at by:

Such an extension relation between XDR descriptions is reflexive and transitive and thus defines a partial order. In practice, provisions have to be made to make sure that two extensions of the same description are compatible (i.e. either one is an extension of the other, or there is a another description that is a valid extension of both).

To put things in concrete terms, such compatibility can be assured if measures are taken to ensure:

3.4. Combination of Protocol Extension Mechanisms

It is possible to use multiple of these means of protocol extension simultaneously. When this is done, the result is a composite extension mechanism. For example, if the XDR replacement or XDR extension mechanism is adopted, a protocol-specific mechanism can be added to it, if the protocol-specific mechanism is built on objects whose XDR definition is sufficiently generic. (e.g. opaque arrays or feature bitmasks).

It can be argued that the NFSv4 attribute model provides such an embedded protocol-specific extension mechanism, since sets of attribute values are specified as XDR opaque arrays and attribute sets are specified as variable-length arrays of 32-bit integers allowing new attribute bits to be defined outside of the XDR definition framework.

Note that there exists specification text that suggests that attributes are part of the XDR specification, making it hard to reach a firm conclusion on the matter. However, the resolution of this question does not affect the other matters discussed below, since, in either case, we have an extension mechanism that allows optional extensions.

4. Pre-IETF NFS Versioning

4.1. The Pre-IETF NFS Environment

NFSv2 and NFSv3 were described by the informational RFC's [RFC1094] and [RFC1813]. These documents each described existing interoperating client and server implementations. Thus they started with running code. If there was a rough consensus in effect, it was that these were useful protocols to use and thus that someone building a client or server had to interoperate with the existing implementations.

The following characteristics of protocol development during that period are noteworthy.

As a result of these commonalities, specifications tended to avoid a lot of detail that would have been required in a more diverse environment. New features were thought of in terms of generally understood client and server implementation frameworks and it was generally clear which of those could be implemented without markedly changing that framework.

4.2. Transition from NFSv2 to NFSv3

There were a number of significant changes involved, but only the first two were of major importance.

Of these only the first actually needed something as drastic as the XDR replacement model. The others could have been handled simply by adding new RPC requests and an enum value to an existing NFSv2 XDR. Since, NFS's extension mechanism was then XDR replacement, such choices were not available.

5. NFS Versioning (so far) Within IETF

5.1. Transition from NFSv3 to NFSv4

NFSv4 was the first NFS version published as a Standards track document. Although an initial [RFC3010], entitled "NFS version 4 protocol" was published as a Proposed Standard, it was never implemented due to issues with the design of locking.

Subsequently, [RFC3530], entitled "Network File System (NFS) version 4 Protocol" was published as a Proposed Standard, obsoleting [RFC3010]. Currently, there are bis documents, [RFC3530bis] and [RFC3530bis-dotx], nearing publication. For more information about these documents and general discussion of issues regarding documents obsoleting or updating minor version protocol specifications, see Section 7.1.

The set of changes made to create NFSv4 was larger by far than that for NFSv3. A partial list follows.

These features/extensions were implemented via the XDR replacement model. This was the only realistic alternative, not only because of the size of the list, but also because some of the changes undercut some central design elements of the pre-IETF NFS protocol.

5.2. Initial Minor Versioning Model for NFSv4

The minor versioning model for NFSv4 is an XDR extension model. It was presented within a versioning paradigm but the fact that all the added features were to be (at least initially) optional indicated that features were intended to be built independently, and that clients were expected to deal with their presence or absence. Note that the term "features" is not explicitly defined. We assume that the definition includes operations within COMPOUND or CB_COMPOUND, attributes, aded flag bits and enum values, and new cases of XDR switch definitions.

Now let's look at some specifics of the minor version rules established for NFSv4 in [RFC3530]. Note that some of these were significantly modified by [RFC5661] and [NFSv42], as discussed in Section 5.6.

This model was subsequently modified in [RFC5661] and in [NFSv42]. See Section 5.3 and Section 5.4 for details.

Many of the events anticipated in the model presented above have never been realized and it may be that they never will be realized. See Section 5.5 for some details. Examples are:

5.3. Transition from NFSv4.0 to NFSv4.1

NFSv4.1 made a major change to NFSv4.0. It was able to do so using an XDR extension model although it did not follow the rules laid out in Section 5.2. Specifically, some features were declared "infrastructural" and thus mandatory upon introduction.

Note that at the same time, the requirement that clients and servers support previous minor versions changed from a "must" to a "SHOULD". Presumably, this change reflects the fact that a minor version with substantial infrastructural changes is essentially a new protocol, making the "must" seem dubious. Whether the "SHOULD" here meets the requirements of [RFC2119] needs to be explored.

NFSv4.1 was described in [RFC5661] and [RFC5662], each of which was published as a Proposed Standard.

The following features were added as infrastructural features.

There also a number of optional features.

Note that there has been little implementation work on the last two of these.

Parallel NFS created an alternate protocol extension mechanism for NFS. New pNFS mapping types could be added. Existing mapping types might have their own extension mechanisms. There also exists the possibility that features might be added within the NFSv4 protocol proper, designed to, or capable of, interacting with particular mapping types. This document will not address these issues but eventually, the NFSv4 Protocol will have to deal with them.

5.4. Transition from NFSv4.1 to NFSv4.2

While NFSv4.2 has not been defined in an RFC, its development has been going on for some time and it is unlikely to experience additions to or deletions from its feature set before completion. The descriptions in [NFSv42] and [NFSv42-dotx] can serve as useful references for this analysis. This is despite the fact that some of the features may later experience significant change.

The following features (all optional) are provided for in NFSv4.2:

Note that there are two pieces of infrastructure that are used by multiple features above. These are not "infrastructural" in the sense mentioned in Section 5.3 (i.e. they are not mandatory), but they do serve an infrastructural role in that are required to be present if one of the optional features that use them are supported.

5.5. Evolution of Minor Versioning Model within NFSv4

As noted above, there have been changes made by [RFC5661] and [NFSv42] in the NFSV4 minor versioning model.

With these changes, we can classify potential minor versions, starting with those that currently exist.

We term versions in the first two categories "infrastructure-level versions" Such versions form an ascending sequence in which the difference between, for example, NFSv4.0 and NFSv4.1, is very similar to the difference between NFSv2 and NFSv3. Clients may be designed for one or the other, or for both but a client capable of interacting with both is really choosing between two different protocols.

We term versions in the last category "optional-feature-only versions".

Note that although the concept of optional features being upgraded to mandatory status remains, it is likely that it will not be used very much, if at all, in the future. The situation is similar for the case of features being downgraded to mandatory-to-not-implement.

Given the diversity of NFS clients and servers, it is highly unlikely that a new non-infrastructural feature will be so broadly necessary/desirable that a consensus to make it mandatory would be likely to arise. Such a decision would prevent servers not implementing such a feature from incorporating other later-developed features. It is only when a feature is judged so useful by users that people will not use servers without it, that adoption will become universal. At this point, a decision to make it mandatory would merely ratify what had already happened on its own.

Except in the case of a universally recognized mistake, any downgrading to mandatory-to-not-implement, would only happen when a replacement becomes mandatory so the considerations above make that situation equally unlikely to occur.

Still, it is possible that versions making such feature status changes will be created in the future. We will call any such "mandatory-feature-change" versions.

5.6. Current Minor Versioning Model for NFSv4

Minor versions which are infrastructure-level or which are, mandatory-feature-change versions form an ascending sequence in which we also have a versioning paradigm, implemented using XDR extension.

Optional-feature-only versions are fundamentally different. Each NFSv4.2 server implements the same protocol as NFSv4.1 with a particular set of optional features beyond those that are mandatory. This set may range from the null set all the way to all of the optional features. Here, it appears that the versioning paradigm is not appropriate to the reality of the extension mechanism.

As a way of illustrating the basic point here, let us consider two servers each of which only supports operations within NFSv4.1:

Although this obeys the rules as they stand, there is no real value for the client, the server, or the protocol in making these artificial distinctions. Optional-feature-only minor versions such as NFSv4.2 are not minor versions in the same sense that NFSv4.0 and NFSv4.1 are. In this case the minorversion field is not providing any information, while the set of operations supported is the important thing that the server implementer chooses and the client needs to know.

In later sections we will discuss how this mismatch might be best addressed as NFSv4 development proceeds.

5.7. Review of NFSv4 Versioning so far

To summarize protocol extension as it applies to the NFSV4 protocols:

6. NFSv4 Versioning Now

6.1. Current NFS Versioning Practices

The following pattern was followed for NFSv4.2, and, unless changes are made, seems likely to persist.

This pattern of development is not a good fit for the kind of minor version that NFSv4.2 is and many future such minor versions will be. Such versions consist of a set of mostly unrelated features, each individually selectable or not by implementers, artificially yoked together. In essence, we have a "feature batch" rather than a minor version.

6.2. Problems with Current NFS Versioning Approach

A number of issues have been noted with the current process for NFSv4.2, leading to the conclusion that the process needs to be revised in some way for future minor versions, of the same sort.

Some instances of problems/issues ascribable to a lack of searching document review are described below. Rather than requiring the necessary review prior to feature acceptance, a common pattern has been that important issues are only discovered on those occasions in which it appears that work on the minor version is coming to a close and that there are issues that have to be addressed before they create difficulties for prospective implementers.

If we look at the problems above, we can understand better how such problems can arise. In short, the decision as to what features to include within a minor version, is not a good use of the rough consensus model and in proceeding on that basis, the group created a set of perverse incentives that undercut the process. Also, as the process goes on for a long time, as is likely, these perverse incentives are intensified. Consider the following points:

7. Going Forward with a New NFSv4 Extension Approach

As we work to correct the issues noted above, and fill out the details of a modified extension paradigm, we will have to take note of the design considerations put forth in [RFC6709].

7.1. Protocol Specification Updates

In some cases it is necessary to update the specification of an NFSv4 minor version, to deal with cases where the existing specification is unclear or incorrect and the changes are too large to address via the errata mechanism.

In the case of NFSv4.0, the following documents will update or obsolete [RFC3530], if approved.

Although these documents change the specification of the protocol, they do not make any substantive changes in the XDR definition.

It is generally held to be the case, that a document updating a minor version RFC, is not allowed to extend the XDR. Sometimes typedefs are added to make it clearer how particular fields are used. Also comments have been added and minimal reformatting done, but even addition of new error codes, as opposed to adding existing error codes to the list of those allowed to be returned by a given operation, has not been allowed.

7.2. Extension Mechanisms used for Protocol Updates

Given that we have an XDR extension paradigm in place, it may not make sense to prohibit XDR extensions to be made in these update documents, as it may be necessary to fix protocol (as opposed to specification) bugs. Although the working group has been able to avoid XDR extension in fixing protocol bugs, that sort of luck might not continue.

While the working group was able to compatibly correct the state migration issues without an XDR change, doing so made the process more complicated than it would be otherwise, since client implementations needed to infer trunking patterns from server behavior, rather than having this information directly provided to them.

It should be noted that the prohibition of such XDR changes is not explicitly mentioned anywhere, to my knowledge. Rather, it seems to be a piece of NFSv4 versioning folklore which needs to be either justified or discarded.

Acknowledging this change would enable the working group to do the following sorts of things (in addition to the correction of the kinds of specification bug fixes now recognized as allowed) in bis documents and other documents which update minor version definition RFC's. While it would be theoretically possible to add entirely new features, working group and IESG review should keep additions limited to the following two classes of items.

Doing things this way would address the issues that have given rise to the perceived need for "micro-versioning". Note that the sorts of changes that would be made would not require any change in the minorversion field at all.

7.3. Requirements for a New NFSv4 Extension Approach

The following requirements will govern construction of a possible new protocol extension approach for NFSv4.

7.4. Principles upon which to Base a New NFSv4 Extension Approach

The following principles seem the best way to meet these requirements without a disruptive major-version-scale shift in the NFSv4 definition (i.e. something as big as the shift from NFSv3 to NFSv4 or from NFSv4.0 to NFSv4.1)

7.5. Work Going Forward in Creating a New NFSv4 Extension Approach

The following set of steps are necessary in many possible ways of proceeding along the way to a new extension approach for NFSv4. Details and actual sequencing will reflect choices that the working group makes.

As far as the document specifying the NFSv4 extension and versioning framework, the following are important elements:

8. Security Considerations

Since no substantive protocol changes are proposed here, no security considerations apply.

As extensions are designed and specified, their security issues will be addressed and each extension will receive the appropriate security review from the NFSv4 working group and IESG.

9. IANA Considerations

The current document does not require any actions by IANA.

Depending on decisions that the working group makes about how to address the issues raised here, future documents may require actions by IANA.

10. Acknowledgements

The author wishes to thank Chuck Lever of Oracle for his helpful document review and many important suggestions.

11. References

11.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

11.2. Informative References

[RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.
[RFC1094] Nowicki, B., "NFS: Network File System Protocol specification", RFC 1094, March 1989.
[RFC1813] Callaghan, B., Pawlowski, B. and P. Staubach, "NFS Version 3 Protocol Specification", RFC 1813, June 1995.
[RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For Values In the Internet Protocol and Related Headers", BCP 37, RFC 2780, March 2000.
[RFC3010] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M. and D. Noveck, "NFS version 4 Protocol", RFC 3010, December 2000.
[RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M. and D. Noveck, "Network File System (NFS) version 4 Protocol", RFC 3530, April 2003.
[RFC3530bis] Haynes, T. and D. Noveck, "Network File System (NFS) Version 4 Protocol ", February 2014.

Work in progress.

[RFC3530bis-dotx] Haynes, T. and D. Noveck, "Network File System (NFS) Version 4 Protocol External Data Representation Standard (XDR) Description ", February 2014.

Work in progress.

[RFC5661] Shepler, S., Eisler, M. and D. Noveck, "Network File System (NFS) Version 4 Minor Version 1 Protocol", RFC 5661, January 2010.
[RFC5662] Shepler, S., Eisler, M. and D. Noveck, "Network File System (NFS) Version 4 Minor Version 1 External Data Representation Standard (XDR) Description", RFC 5662, January 2010.
[RFC6709] Carpenter, B., Aboba, B. and S. Cheshire, "Design Considerations for Protocol Extensions", RFC 6709, September 2012.
[NFSv42] Haynes, T., "NFS Version 4 Minor Version 2 ", February 2014.

Work in progress.

[NFSv42-dotx] Haynes, T., "NFS Version 4 Minor Version 2 External Data Representation Standard (XDR) Description ", February 2014.

Work in progress.

[migr-v4.0-update] Noveck, D., Shivam, P., Lever, C. and B. Baker, "NFSv4.0 migration: Specification Update ", October 2013.

Work in progress.

Author's Address

David Noveck 26 Locust Avenue Lexington, MA 02421 US Phone: +1 781 572 8038 EMail: davenoveck@gmail.com