mzhang-nfsv4-recursively-setting-00.txt

Internet DRAFT - draft-mzhang-nfsv4-recursively-setting

draft-mzhang-nfsv4-recursively-setting

Last Version:	draft-mzhang-nfsv4-recursively-setting-00.txt	Tracker Entry
Date:	`27-Dec-2023`
Disposition:	current

Network File System Version 4 M. Zhang
Internet-Draft R. Kunhi Parambattu
Intended status: Standards Track J. Yang
Expires: 27 June 2024 D. Geng
Z. Jiang
Y. Du
Huawei Technologies
25 December 2023

Recursively Setting Directories and Subitems
draft-mzhang-nfsv4-recursively-setting-00

Abstract

In recent years, the concept of near-data computing has been widely
recognized in storage architectures. The core idea is to process
data nearby, reduce the overhead of network transmission, and utilize
the computing capability of smart devices (such as intelligent NICs,
smart SSDs, and DPUs). This reduces CPU and memory usage of clients
(computing nodes) and improves data processing efficiency. This
design idea is applied in NFSv4.2 or future NFS verions, such as
Server-Side Copy, in which client sends the control command and the
storage server copies data without passing through the data between
the client and storage server. Compared with traditional copy
operations, data is read from the source storage server and then
written to the target storage server after two network transmissions.
Data transmission on the network is reduced, and bandwidth resources
are greatly released. In addition, the client changes from an
original data copy executor to a data copy controller, and a specific
execution action is executed by the storage server. Therefore, a
large amount of computing resources and memory resources are saved on
the client side.

Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.

Zhang, et al. Expires 27 June 2024 [Page 1]

Internet-Draft Recursively Setting Directoriestle December 2023

Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."

This Internet-Draft will expire on 27 June 2024.

This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.

Table of Contents

1. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 2
2. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . 3
3. Implementation Considerations . . . . . . . . . . . . . . . . 4
4. Recursive Set Operations . . . . . . . . . . . . . . . . . . 7
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.1. Normative References . . . . . . . . . . . . . . . . . . 12
5.2. Informative References . . . . . . . . . . . . . . . . . 12
Appendix A. An Appendix . . . . . . . . . . . . . . . . . . . . 12
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12

1. Problem Statement

In actual storage applications, users often recursively set the
attributes of directories and subitems(their subfiles and
subdirectories). Message interaction between client and server is
complex, and the client consumes a lot of resources, which does not
match the concept of near-data computing. FIG. 1 shows the existing
flowchart of recursively setting the attributes of all files under
dir1.

Zhang, et al. Expires 27 June 2024 [Page 2]

Internet-Draft Recursively Setting Directoriestle December 2023

Step 1: The client sends the readdir command to obtain the list of
all files in dir1.

Step 2: The storage server responds to the readdir operation. If the
directory contains many subitems, the client needs to run the readdir
operation for multiple times.

Step 3: The client obtains the file list based on the response
information, executes the getattr request for each file, and obtains
the file attributes.

Step 4: The storage server responds to the getattr request.

Step 5: The client sends a setattr request.

Step 6: The storage server responds to the setattr request.

Repeat step 1 to step 6 to traverse all files. If the dir1 directory
contains 100,000 files, repeat step 1 to step 6 for 100,000 times to
recursively set the dir1 attribute. It can be learned that the
process consumes CPU resources and memory resources of the client,
and a large number of messages are exchanged between the client and
the storage server. As a result, an end-to-end time for the
attribute setting operation is relatively long.

preamble to the figure.

Client Server
+ +
| |
|------ Readdir ----------------------->|
|<--------------------------------------|
|------ Getattr ----------------------->|
|<--------------------------------------|
|------ Setattr ----------------------->|
|<--------------------------------------|
| .... |
| |

Figure 1: Existing flowchart for recursive set operation

As you can see, this figure doodled and dawdled.

2. Protocol Overview

After adopting the concept of near data calculation, the above
scenario can be optimized.

Zhang, et al. Expires 27 June 2024 [Page 3]

Internet-Draft Recursively Setting Directoriestle December 2023

Step 1: The client identifies that the operation object of the
attribute setting is a directory and the attribute setting is
recursive, and invokes a new operation (for example, RECURSIVE_SET).

Step 2: The storage server recursively queries all files in the
directory, sets attributes for each file, and sends a RECURSIVE_SET
response.

Compared with the original process, this process not only saves the
CPU and memory usage of the client, but also significantly reduces
the number of messages exchanged between the client and server. This
greatly improves the efficiency in recursive attribute setting
scenarios.

Similar to the design of Server-Side Copy, the new operation word
RECURSIVE_SET is used to recursively set the attributes of a
directory and its subitems. This operation can be in synchronous or
asynchronous mode, which is determined by the input parameter:

o If no back channel is created when the client and server establish
a connection, the client can only use the synchronous mode in the
RECURSIVE_SET request. If the client uses the asynchronous mode, the
server returns the error code NFS4ERR_CB_PATH_DOWN.

o If a back channel is created when the client and server establish a
connection, the RECURSIVE_SET request can be in synchronous or
asynchronous mode.

Note to RFC Editor: this section may be removed on publication as an
RFC.

3. Implementation Considerations

A recommended Recursive Set operation in an synchronous mode is shown
in Figure 2.

Step 1: The client sends a RECURSIVE_SET request. In the request,
select the synchronization setting mode and set rsa_sync to true.

Step 2: If the storage server completes the setting within the
timeout period, the storage server returns the request result to the
client. If the setting is not complete within the timeout period,
generate rsr_callback_id and rsr_recursiveverf. In addition, the
rsr_callback_id, rsr_recursiveverf, and error code are sent to the
client.

Zhang, et al. Expires 27 June 2024 [Page 4]

Internet-Draft Recursively Setting Directoriestle December 2023

Step 3: The client sends a RECURSIVE_SET_STATUS query request. The
request contains the information of rss_stateid. The value of
rss_stateid is obtained from the response packet of RECURSIVE_SET
which value is the value rsr_callback_id. If the value of
rss_stateid is the same as the value of rsr_callback_id cached on the
storage server, the storage server returns the setting result.
Storage server return NFS4_OK if the server successfully finish the
recursive setting or NFS4_Pending if the setting is till on going, or
else set failure error code. If the value of rss_stateid in the
request is different from the value cached on the server, the storage
server returns the error code NFS4ERR_BAD_STATEID.

Step 4: After receiving the RECURSIVE_SET_STATUS response, the client
delays sending the RECURSIVE_SET_STATUS request if the error code is
NFS4_PENDING. The delay setting request must contain rss_stateid.
If the error code returned by the server is NFS4_OK, the recursive
attribute setting is successful. If the error code is the error code
of setattr defined in NFSv4.2, the recursive attribute setting fails.
In this case, the client returns a response to the application.

preamble to the figure.

Client Server
+ +
| |
|------ RECURSIVE_SET(rsa_sync = 1) --------------------->|
| |
|<-----Response(rsr_callback_id=0, rsr_recursiveverf=0)---| within the timeout period
| |
| |
|<----Response(rsr_callback_id=1, rsr_recursiveverf=1)----| beyond the timeout period
| |
| |
| |
|-------RECURSIVE_SET_STATUS(rss_stateid=1)-------------->|
| |
|<------Response------------------------------------------|
| |
| |

Figure 2: A synchronous Recursive Set

An alternative Recursive Set operation in an synchronous mode is also
given in Figure 3.

As you can see, this figure doodled and dawdled.

Zhang, et al. Expires 27 June 2024 [Page 5]

Internet-Draft Recursively Setting Directoriestle December 2023

Step 1: The client sends a RECURSIVE_SET request. In the request,
select the synchronization setting mode and set rsa_sync to true.

Step 2: If the storage server completes the setting within the
timeout period, the storage server returns the request result to the
client. If the setting is not complete within the timeout period,
generate rsr_callback_id and rsr_recursiveverf, and set the error
code to NFS4_PENDING. In addition, the rsr_callback_id,
rsr_recursiveverf, and error code are sent to the client.

Step 3: The client sends a RECURSIVE_SET_STATUS query request. The
request contains the information of rss_stateid. The value of
rss_stateid is obtained from the response packet of RECURSIVE_SET
which value is the value rsr_callback_id. If the value of
rss_stateid is the same as the value of rsr_callback_id cached on the
storage server, the storage server returns the setting result. (if
Success, set NFS4_Pending, or else set failure error code). If the
value of rss_stateid in the request is different from the value
cached on the server, the storage server returns the error code
NFS4ERR_BAD_STATEID.

Step 4: After receiving the RECURSIVE_SET response, the client delays
sending the RECURSIVE_SET_STATUS request if the error code is
NFS4_PENDING. The delay setting request must contain rss_stateid.
If the error code returned by the server is NFS4_OK, the recursive
attribute setting is successful. If the error code is the error code
of setattr defined in NFSv4.2, the recursive attribute setting fails.
In this case, the client returns a response to the application.

A recommended Recursive Set operation in an asynchronous mode is
shown in Figure 3.

Step 1: The client sends a RECURSIVE_SET request. In the request,
the asynchronous setting mode is rsa_sync=false.

Step 2: The storage server needs to generate rsr_callback_id and
rsr_recursiveverf, and set the error code to NFS4_OK. In addition,
the rsr_callback_id, rsr_recursiveverf, and error code are sent to
the client. The storage server continues the recursive setting
operation.

Step 3: After receiving the response, the client checks the error
code and starts an asynchronous task to receive the callback message
from the server.

Zhang, et al. Expires 27 June 2024 [Page 6]

Internet-Draft Recursively Setting Directoriestle December 2023

Step 4: The client create an asynchronous listening task and matches
rsr_callback_id and rsr_recursiveverf. If the matching succeeds, the
task is successfully executed. If rsr_callback_id can be matched but
rsr_recursiveverf cannot be matched, client skip the message.

Step 5: After recursively setting attributes, the storage server
sends a message to the client through the backchannel of the NFS4 and
notifies the client of the setting result.

Step 6: If the client does not receive the asynchronous message, the
started task is forcibly terminated when the session is destroyed.
If an error occurs when the storage server recursively sets subitem
attributes, the storage server terminates the task and returns the
error code to the client. All possible errors are subject to the
error codes defined by setattr.

preamble to the figure.

Client Server
+ +
| |
|------ RECURSIVE_SET(rsa_sync = 0) --------------------->|
| |
|<------Response(rsr_callback_id=1, rsr_recursiveverf=1)--|
| |
| |
|<------CB_RECURSIVE_SET----------------------------------|
| |
| |
| |

Figure 3: An asynchronous Recursive Set

As you can see, this figure doodled and dawdled.

4. Recursive Set Operations

4.1 Operation TBD1: RECURSIVE_SET – Recursively sets the attributes
of a directory and its subitems

ARGUMENT

Struct RECURSIVE_SET4args {

Zhang, et al. Expires 27 June 2024 [Page 7]

Internet-Draft Recursively Setting Directoriestle December 2023

bool rsa_sync;

};

RESULT

struct recursive_set_response4 {

stateid4 rsr_callback_id;

verifier4 rsr_recursiversr_recursiveverf;

};

union RECURSIVE_SET4res (nfsstat4 rsr_status) {

case NFS4_OK:

recursive_set_response4 rsr_resok4;

default:

void;

};

DESCRIPTION

The RECURSIVE_SET operation is used by the client to recursively set
the attributes of a directory and all its subitems. The operation
should be placed after setattr in the existing setattr operation.
After the storage server receives the setattr combination operation,
if the setattr operation does not followed by RECURSIVE_SET, the
original process remains unchanged. If the setattr operator is
followed by the RECURSIVE_SET operation, the storage server considers
the attributes of the directory and its subitems to initiate
recursive set mode.

If the storage is successfully executed, the values of
rsr_callback_id and rsr_recursiversr_recursiveverf are 0.

Zhang, et al. Expires 27 June 2024 [Page 8]

Internet-Draft Recursively Setting Directoriestle December 2023

If the storage is not complete, the values of rsr_callback_id and
rsr_recursiversr_recursiveverf are generated.

If rsa_sync is set to true, there are two processing policies:

o According to the NFSv4 protocol, the client must wait for the
response from the server. Therefore, the client can wait for the
processing result from the server. A problem in this mode is that
the current request occupies a slot in a session, resulting in a
decrease in the number of available slots. If multiple tasks of the
same type are being executed, no slot is available on the client in
severe cases.

o The storage server determines the implementation duration. If the
implementation duration is too long, the storage server may return
non-zero values of rsr_callback_id and
rsr_recursiversr_recursiveverf.

After the client receives the request, the client waits for a period
of time and executes RECURSIVE_SET_STATUS to query the execution
progress of the current task. If the server does not finish the
execution , NFS4ERR_PENDING is returned. After receiving the error
code, the client retryes the query after a period of time. If the
execution is complete, NFS4_OK is returned.

4.2 Operation TBD2: RECURSIVE_SET_STATUS – Query the result of the
recursively setting the attributes of directories and their subitems

ARGUMENT

struct RECURSIVE_SET_STATUS4args {

stateid4 rssa_stateid;

};

RESULT

#define NFS4ERR_PENDING 10090

struct RECURSIVE_SET_STATUS4res {

Zhang, et al. Expires 27 June 2024 [Page 9]

Internet-Draft Recursively Setting Directoriestle December 2023

nfsstat4 rssr_status;

};

DESCRIPTION

The RECURSIVE_SET_STATUS operation is used by the client to query the
status of a recursively set task (attributes of a directory and its
subitems). If the task on the storage server is complete, NFS4_OK is
returned. If any error occurs during task execution, a response
error code is returned and the error code is not extended or modified
in this case so the error code is the same as the error code that may
occur during the setattr operation. If the current setting task is
not complete, NFS4_PENDING is returned.

4.3 Operation TBD3: RECURSIVE_SET_CANCEL – Canceling a Running Task
on the Client

ARGUMENT

//The following operation is used to cancel the recursive setting
task that is being executed.

struct RECURSIVE_SET_CANCEL4args {

stateid4 rsca_stateid;

};

RESULT

struct RECURSIVE_SET_CANCEL4res {

nfsstat4 rscr_status;

};

DESCRIPTION

Zhang, et al. Expires 27 June 2024 [Page 10]

Internet-Draft Recursively Setting Directoriestle December 2023

RECURSIVE_SET_CANCEL is used to cancel the task that is being
executed. The request packet contains rsca_stateid. The value of
rsca_stateid is obtained from the response of RECURSIVE_SET. If the
storage server fails to cancel the task, NFS4ERR_DELAY is returned.
When receiving the message, the client delays the retry. If the
current task is complete, NFS4_OK is returned.

4.4 Operation TBD4: CB_RECURSIVE_SET_NOTIFY – Notify the recursively
setting result to client

ARGUMENT

struct CB_RECURSIVE_SET_NOTIFY4args {

nfs_fh4 crsna_fh;

stateid4 crsna_stateid;

verifier4 crsna_recursiveverf;

nfsstat4 crsna_status;

};

RESULT

struct CB_RECURSIVE_SET_NOTIFY4res {

nfsstat4 crsnr_status;

};

DESCRIPTION

Zhang, et al. Expires 27 June 2024 [Page 11]

Internet-Draft Recursively Setting Directoriestle December 2023

CB_RECURSIVE_SET_NOTIFY is used to send the server callback to client
to notify the client of the result of the task of recursively setting
the attributes of directories and their subitems. Client check the
crsna_stateid and crsna_recursiveverf and client will finish the wait
task if they are matched the value received from previous
RECURSIVE_SET response or will skip the notification in case of not
match and return NFS4ERR_BAD_STATEID to server.

5. References

5.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.

[RFC7862] Haynes, T., "Network File System (NFS) Version 4 Minor
Version 2 Protocol", RFC 7862, DOI 10.17487/RFC7862,
November 2016, <https://www.rfc-editor.org/info/rfc7862>.

5.2. Informative References

[InfRef] "", 2004.

Appendix A. An Appendix

Authors' Addresses

Minqian Zhang
Huawei Technologies
1899 Xiyuan
Chengdu
High-tech West District, 611731
China
Phone: +86-13547833949
Email: zhangmingqian.zhang@huawei.com

Rijesh Kunhi Parambattu
Huawei Technologies
Email: rijesh.kunhi.parambattu1@huawei.com

Jing Yang
Huawei Technologies
Email: yangjing8@huawei.com

Zhang, et al. Expires 27 June 2024 [Page 12]

Internet-Draft Recursively Setting Directoriestle December 2023

Dongyu Geng
Huawei Technologies
Email: gengdongyu@huawei.com

Zhongbing Jiang
Huawei Technologies
Email: jiangzhongbing@huawei.com

Yunfei Du
Huawei Technologies
Email: duyunfei1@huawei.com

Zhang, et al. Expires 27 June 2024 [Page 13]