Directory Delegation clarification for Network File System Version 4, Minor Version 2
Internet-Draft draft-rmacklem-nfsv4-directory-delegations-03
Rick Macklem
FreeBSD Project
Canada
rmacklem@uoguelph.ca
Jeffrey Layton
Linux NFS Project
USA
jlayton@poochiereds.net
8 June 2024
Transport
Network File System Version 4
This document describes the addition of bit flags to be used in the request by a client for the GET_DIR_DELEGATION operation to allow the client to specify detailed behavior of CB_NOTIFYs the server will perform on the client. Early implementation experience with directory delegations has demonstrated that clients may not need the full information specified in [RFC8881] for CB_NOTIFYs and may not require the recall of the directory delegation to be done synchronously. Limiting the CB_NOTIFY's requirements can simplify server implementation of directory delegations. These additional bit flags allow the client to request desired behavior. The server's reply will then specify what behavior the client can expect.
Note
This note is to be removed before publishing as an RFC.
Discussion of this draft occurs on the NFSv4 working group mailing list, archived at https://mailarchive.ietf.org/arch/browse/nfsv4/. Working Group information is available at https://datatracker.ietf.org/wg/nfsv4/about/.
Introduction
Implementation experience with directory delegations in NFSv4.2 has identified a need for a client to request specific information be provided in the CB_NOTIFYs performed by the server when a directory delegation has been issued to a client. It has also been discovered that certain aspects of CB_NOTIFYs are underspecified. This document specifies the addition of bit flags to be used in the gdda_notification_types request argument and gddr_notification reply argument of the GET_DIR_DELEGATION operation to allow negotiation of specific information to be provided in the CB_NOTIFYs. Note that these bit flags are used in addition to the bits defined in Section 20.4.1 of [RFC8881] for the notification types and not in place of them. A client will also set bit(s) in gdda_notification_types for the types of notifications requested, if the client is requesting notifications.
The CB_NOTIFY_WANT_VALID bit is used to indicate to the server that this extended behavior is being requested and the server acknowledges that it supports this extended behavior by setting CB_NOTIFY_WANT_VALID in the gddr_notification field in the GET_DIR_DELEGATION reply. It also implies that the specification of behavior for CB_NOTIFYs in this document are expected by the client.
A server should reply with as large a subset of the bits set in gdda_notification_types in gddr_notification as is possible. The client will decide if the set of bit flags set in the gddr_notification field of the GET_DIR_DELEGATION reply provides behavior that is useful for the client. If the directory delegation is deemed not useful, the client SHOULD DELEGRETURN the directory delegation.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
Protocol Extension Considerations
This document presents an extension to minor version 2 of the NFSv4 protocol as described in [RFC8178]. It describes new OPTIONAL features. NFSv4.2 servers and clients implemented without knowledge of this extension will continue to interoperate with clients and servers that are aware of the extension (whether or not they support it).
Note that [RFC7862] does not define NFSv4.2 as non-extensible, so [RFC8178] treats it as an extensible minor version. This Standards Track RFC extends NFSv4.2 but does not update [RFC7862] or [RFC8178].
New Notification Bit Flags
New Bit Flags for Notification Bitmaps

/* New bits for gdda_notification_types and gddr_notification */
const CB_NOTIFY_TYPE_BIT_MASK                 = 0x0000FFFF;
const CB_NOTIFY_WANT_BIT_MASK                 = 0xFFFF0000;
const CB_NOTIFY_WANT_VALID                    = 0x00010000;
const CB_NOTIFY_WANT_OLD_DIR_OFF_COOKIE       = 0x00020000;
const CB_NOTIFY_WANT_NEW_DIR_OFF_COOKIE       = 0x00040000;
const CB_NOTIFY_WANT_ADD_PREV_ENTRY           = 0x00080000;
const CB_NOTIFY_WANT_LAST_ENTRY_BOOL          = 0x00100000;
const CB_NOTIFY_WANT_MONOTONIC_DIR_OFF_COOKIE = 0x00200000;
const CB_NOTIFY_WANT_NOTIFY_SAME_CLIENT       = 0x00400000;
const CB_NOTIFY_WANT_SYNCHRONOUS_RECALL       = 0x00800000;
const CB_NOTIFY_WANT_DIR_ATTRS_RECALL         = 0x01000000;

Definitions of new bit flags for notification bitmaps
CB_NOTIFY_WANT_VALID
This bit flag is set to indicate this extension is being used.
Rationale
This bit is used to by the client to indicate that this extension is being requested and by the server to indicate that it supports this extension. If not set in the server's reply, all bits not in the CB_NOTIFY4_TYPE_BIT_MASK MUST be ignored by the client and the client MUST assume [RFC8881] compliant directory delegation behavior for the server. If a server receives a GET_DIR_DELEGATION request without this bit set in it, the client is requesting a [RFC8881] compliant directory delegation. If the client receives any of NFS4ERR_BADXDR, NFS4ERR_INVAL or NFS4ERR_SERVERFAULT for a GET_DIR_DELEGATION reply, the client MUST assume that the server does not support this extension and can only support [RFC8881] compliant behavior.
CB_NOTIFY_WANT_OLD_DIR_OFF_COOKIE
This bit flag is set to indicate a valid nrm_old_entry_cookie in the NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests.
Rationale
This bit indicates that the nrm_old_entry_cookie field will be filled in correctly. When this bit flag is not set, all nrm_old_entry_cookie field(s) in CB_NOTIFY requests MUST be ignored. A server implementation may be simplified if it does not need to fill this field in correctly.
CB_NOTIFY_WANT_NEW_DIR_OFF_COOKIE
This bit flag is set to indicate a valid nad_new_entry_cookie in the NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests. When this bit flag is not set, the nad_new_entry_cookie array MUST be of 0 length.
Rationale
This bit indicates that the nad_new_entry_cookie array will always be of length 1 with a valid cookie value in it. When this bit flag is not set, the array nad_new_entry_cookie should be of 0 length and any array entry MUST be ignored. A server implementation may be simplified if it does not need to fill this field in correctly.
CB_NOTIFY_WANT_ADD_PREV_ENTRY
This bit flag is set to indicate a valid nad_prev_entry in the NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests. When this bit flag is set, the nad_prev_entry array will be of length 1 with a valid previous entry in it unless the new entry has been added at the beginning of the directory. When this bit flag is not set, the nad_prev_entry array MUST be of 0 length.
Rationale
A server implementation may be simplified if it does not need to fill in a previous entry correctly.
CB_NOTIFY_WANT_MONOTONIC_DIR_OFF_COOKIE
This bit flag is set to indicate that the directory offset cookies in the server's READDIR operation replies are always monotonically increasing values for this directory. If this bit is not set, a client can only assume directory offset cookie values in READDIR operation replies are unique for this directory.
Rationale
If the directory offset cookies in the server's READDIR replies are monotonically increasing, the client can use these cookies to maintain the order of the directory entries in the READDIR reply. It can also use a valid value in nad_new_entry_cookie to determine the ordinal position of the new entry. The client can make good use of this ordinal position information, particularily when it is using CB_NOTIFYs to maintain cached READDIR reply blocks in the client.
CB_NOTIFY_WANT_LAST_ENTRY_BOOL
This bit flag is set to indicate a valid nad_last_entry in the NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests. When this bit flag is set, nad_last_entry will be set true or false to indicate if the new entry is the last entry in the directory. When this bit flag is not set, nad_last_entry should be false and MUST be ignored by the client.
Rationale
A server implementation may be simplified if it does not need to indicate if a new entry is the last entry in a directory.
CB_NOTIFY_WANT_NOTIFY_SAME_CLIENT
This bit flag is set to indicate that NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests will be sent to the client that performed the operation(s) that changed the directory contents. If this bit is not set, the server will only send NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFY requests to other client(s) that hold a directory delegation for the directory whose contents changed. Normally, a server is required to perform a CB_RECALL of the directory delegation if it cannot perform the CB_NOTIFY(s) that a directory change requires. However, if this bit is not set, the CB_RECALL is only required to be done to other client(s) that hold a directory delegation for a directory that has changed and not for the client that performed the operation(s) that caused the directory change.
Rationale
For some client implementations, a client will not require any CB_NOTIFYs for directory content changes that the same client caused via operation(s) performed on the directory. For other clients, the NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY and NOTIFY4_RENAME_ENTRY CB_NOTIFYs are required for directory contents changes caused by the client performing operation(s) on the directory. This is likely because the client requires position information for the removal/addition of new entries from the server.
CB_NOTIFY_WANT_SYNCHRONOUS_RECALL
This bit flag is set to indicate that any required CB_RECALL of the directory delegation will be done synchronously. This behavior is described in Section 10.9.2 of [RFC8881] and requires that the recall be completed before the completion of the directory operation being performed on the server that caused the recall. If this bit is not set, any recall will be performed asynchronously, but with as short a delay as possible after the operation on the directory that caused the recall.
Rationale
A synchronous recall will normally result in a reply of NFS4ERR_DELAY to the directory operation that is causing the directory change that is, in turn, causing the required CB_RECALL. This results in significant delays for clients performing such operations. Since CB_NOTIFYs are done asynchronously, it is likely that required CB_RECALLs can be asynchronous, as well. It seems likely that synchronous behavior may only be required for the pure recall model described in Section 10.9.2 of [RFC8881] and should only be required by a client if needed for correct behavior.
CB_NOTIFY_WANT_DIR_ATTRS_RECALL
This bit flag is set to indicate that any change to the directory's attributes caused by operation(s) performed by other clients will result in a CB_RECALL of the directory delegation. Whether or not the CB_RECALL is done synchronously is determined by the CB_NOTIFY_WANT_SYNCHRONOUS_RECALL bit flag.
Rationale
Some clients could use a CB_RECALL of the directory delegation to indicate that the directory's attributes have changed and that a ACCESS operation is needed to determine if the directory is still readable. Other clients may use NOTIFY4_CHANGE_DIR_ATTRS for handling directory attributes and directory access, while still others may choose to use ACCESS operations to determine if the directory is still readable. This flag avoids a CB_RECALL of the directory delegation for clients that do not require that behavior.
Semantics Clarifications
[RFC8881] does not specify what attributes are to be sent in the nad_new_entry.ne_attrs field of the CB_NOTIFYs. For this extension, the gdda_child_attributes argument to GET_DIR_DELEGATIONS indicates what attributes the client wants to be sent in the nad_new_entry.ne_attrs field of NOTIFY4_ADD_ENTRY or NOTIFY4_RENAME_ENTRY if either of these CB_NOTIFYs are requested in the gdda_notification_types argument.
The XDR specification for CB_NOTIFY found in [RFC8881] allows multiple bits to be set in cna_changes.notify_mask. If NOTIFY4_CHANGE_DIR_ATTRS is set along with any of NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY or NOTIFY4_RENAME_ENTRY, the directory attributes MUST be post operation attributes, for the operation that caused the NOTIFY4_REMOVE_ENTRY, NOTIFY4_ADD_ENTRY or NOTIFY4_RENAME_ENTRY.
Implementation Status
This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft. The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs.
Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist.
FreeBSD NFS server and client
Organization:
FreeBSD Project
URL:
https://www.freebsd.org
Maturity:
Experimental software based on the current document.
Coverage:
The bulk of this specification is implemented, but using only certain bit flags. The current client implementation uses directory delegations and associated CB_NOTIFYs to maintain the client's name lookup cache. A previous experimental implementation used directory delegations to maintain cached READDIR reply blocks in the client.
Licensing:
BSD
Implementation experience:
The FreeBSD NFSv4.2 server implements the generation of monotonically increasing directory offset cookies and provides these cookies in NOTIFY4_REMOVE_ENTRY and NOTIFY4_ADD_ENTRY CB_NOTIFY's. It does not generate a previous entry for NOTIFY4_ADD_ENTRY and always replies with a 0 length nad_prev_entry array. It does not set nad_last_entry correctly and always returns false. When the FreeBSD NFSv4.2 server is required to recall a directory delegation, it performs the CB_RECALL asynchronously. The above is sufficient for the requirements of the FreeBSD NFSv4.2 client implementation at this time. The current client only sets CB_NOTIFY_WANT_VALID along with NOTIFY4_CHANGE_DIR_ATTR, NOTIFY4_REMOVE_ENTRY and NOTIFY4_ADD_ENTRY and uses the CB_NOTIFYs to maintain the client's name lookup cache. A previous client set the CB_NOTIFY_WANT_VALID, CB_NOTIFY_WANT_OLD_DIR_OFF_COOKIE, CB_NOTIFY_WANT_NEW_DIR_OFF_COOKIE, CB_NOTIFY_WANT_MONOTONIC_DIR_OFF_COOKIE and CB_NOTIFY_WANT_NOTIFY_SAME_CLIENT so that it can use the CB_NOTIFYs to maintain up to date cached READDIR reply blocks.
Linux kernel NFS server and client
Organization:
Linux NFS Project
URL:
https://www.linux-nfs.org
Maturity:
Draft implementation in progress
Coverage:
The draft implementation has recallable directory delegation support, but no CB_NOTIFY support as of yet.
Licensing:
GPL
Implementation experience:
The Linux client already aggressively caches directory information, which renders directory delegations without notifications relatively useless. Dealing with directory cookies and ordering is difficult across the broad range of exportable Linux filesystems, so it is likely that the Linux client and server will opt to not provide or request that information.
References
Normative References
[RFC8881]
[RFC2119]
[RFC8174]
[RFC8178]
[RFC7862]
Acknowledgments
XXX