Subject: [nfsv4] layoutsegment type proposal

From: Benny Halevy <[email-masked]>
Date: Mon, 21 May 2007 14:46:05 +0300

Following is a proposal for using a layoutsegment structure for
LAYOUTGET, LAYOUTRETURN, and CB_LAYOUTRECALL.

Please note:
1. I defined a common layoutscope4 enumeration to be used LAYOUTRETURN
   and CB_LAYOUTRECALL (replacing layout{return,recall}_type4).
   Besides being more efficient in type definitions, this makes a lot of sense
   for the final layoutreturn in response to a cb_layoutrecall that should be
   for the same scope, and for comparing incoming layoutreturns with outstanding
   cb_layoutrecalls on the server.

2. the offset and length fields are not in use for LAYOUTSCOPE4_{FSID,ALL}.
   previously they were given only for LAYOUT{RECALL,RETURN}_FILE, now as 
   part of the embedded layoutsegment4 structure they always occupy space.
   I specified setting them to zero while not in use so that in the future
   if we find a used for them (e.g. specifying a range for all layouts (in a
   fsid)) the server can set them to a meaningful value.

3. I haven't defined the use for ls_iomode in LAYOUTCOMMIT since this is another
   pending issue.

The simplified structures look like this:

struct layout_content4 is gone
struct layoutupdate4 is gone
enum layoutreturn_type4 is gone
enum layoutrecall_type4 is gone

    struct layoutsegment4 {
        layouttype4             ls_layouttype;
        iomode4                 ls_iomode;
        offset4                 ls_offset;
        length4                 ls_length;
    };

   struct layout4 {
       struct layoutsegment4   lo_seg;
       opaque                  lo_body<>;
   };

   struct LAYOUTCOMMIT4args {
           /* CURRENT_FH: file */
           layoutsegment4          loca_seg;
           bool                    loca_reclaim;
           newoffset4              loca_last_write_offset;
           newtime4                loca_time_modify;
           newtime4                loca_time_access;
           opaque                  loca_layoutupdate<>;
   };

   enum layoutscope4 {
       LAYOUTSCOPE4_FILE           = 1,
       LAYOUTSCOPE4_FSID           = 2,
       LAYOUTSCOPE4_ALL            = 3,
   };

   struct LAYOUTRETURN4args {
           /* CURRENT_FH: file */
           bool                    lora_reclaim;
           layoutscope4            lora_scope;
           layoutsegment4          lora_seg;
   };

   union layoutrecall4 switch(layoutscope4 clora_scope) {
   case LAYOUTSCOPE4_FILE:
           nfs_fh4            clora_fh;
   case LAYOUTSCOPE4_FSID:
           fsid4              clora_fsid;
   case LAYOUTSCOPE4_ALL:
           void;
   };
 
   struct CB_LAYOUTRECALL4args {
           bool                    clora_changed;
           layoutrecall4           clora_recall;
           layoutsegment4          clora_seg;
   };

The complete diff follows below.

Benny

diff -u draft-ietf-nfsv4-minorversion1-10.txt draft-ietf-nfsv4-minorversion1-10-layoutseg.txt
--- draft-ietf-nfsv4-minorversion1-10.txt	2007-03-14 14:36:21.000000000 +0200
+++ draft-ietf-nfsv4-minorversion1-10-layoutseg.txt	2007-05-21 14:13:08.000000000 +0300
@@ -3897,21 +3897,24 @@
    They define the set of devices associated with a file system for the
    layout type specified in the GETDEVICELIST4args.
 
-3.2.19.  layout_content4
-
-   struct layout_content4 {
-           layouttype4 loc_type;
-           opaque      loc_body<>;
-   };
-
-   The loc_body field must be interpreted based on the layout type
-   (loc_type).  This document defines the loc_body for the NFSv4.1 file
-   layout type is defined; see Section 13.3 for its definition.
-
-
-
-
+3.2.19.  layoutsegment4
 
+   struct layoutsegment4 {
+       layouttype4             ls_layouttype;
+       iomode4                 ls_iomode;
+       offset4                 ls_offset;
+       length4                 ls_length;
+   };
+
+   The layoutsegment4 structure identifies a subset of a file's layout.
+   The layout is given by the filehandle with the operation and the client ID
+   associated with the session used to send the operation.  ls_iomode can be
+   either LAYOUTIOMODE4_READ or LAYOUTIOMODE4_RW to identify a read or read/write
+   layout, or LAYOUTIOMODE4_ANY when recalling or returning both read and
+   read/write layouts.  ls_offset identifies the starting offset of the octet
+   range and ls_length identifies the number of octets in the range. The special
+   value of all ones for ls_length identifies a range spanning from ls_offset
+   to the end of the file.

@@ -3924,33 +3927,18 @@
 3.2.20.  layout4
 
    struct layout4 {
-       offset4                 lo_offset;
-       length4                 lo_length;
-       layoutiomode4           lo_iomode;
-       layout_content4         lo_content;
+       struct layoutsegment4   lo_seg;
+       opaque                  lo_body<>;
    };
 
    The layout4 structure defines a layout for a file.  The layout type
-   specific data is opaque within lo_content.  Since layouts are sub-
+   specific data is opaque within lo_body.  Since layouts are sub-
    dividable, the offset and length together with the file's filehandle,
    the client ID, iomode, and layout type, identifies the layout.
 
-3.2.21.  layoutupdate4
-
-   struct layoutupdate4 {
-       layouttype4             lou_type;
-       opaque                  lou_body<>;
-   };
-
-   The layoutupdate4 structure is used by the client to return 'updated'
-   layout information to the metadata server at LAYOUTCOMMIT time.  This
-   structure provides a channel to pass layout type specific information
-   (in field lou_body) back to the metadata server.  E.g., for block/
-   volume layout types this could include the list of reserved blocks
-   that were written.  The contents of the opaque lou_body argument are
-   determined by the layout type and are defined in their context.  The
-   NFSv4.1 file-based layout does not use this structure, thus the
-   lou_body field should have a zero length.
+   The lo_body field must be interpreted based on the layout type
+   (lo_seg.ls_layouttype).  This document defines the lo_body for the NFSv4.1 file
+   layout type; see Section 13.3 for its definition.
 
 3.2.22.  layouthint4
 
@@ -3988,13 +3976,27 @@
    The iomode specifies whether the client intends to read or write
    (with the possibility of reading) the data represented by the layout.
    The ANY iomode MUST NOT be used for LAYOUTGET, however, it can be
-   used for LAYOUTRETURN and LAYOUTRECALL.  The ANY iomode specifies
+   used for LAYOUTRETURN and CB_LAYOUTRECALL.  The ANY iomode specifies
    that layouts pertaining to both READ and RW iomodes are being
    returned or recalled, respectively.  The metadata server's use of the
    iomode may depend on the layout type being used.  The storage devices
    may validate I/O accesses against the iomode and reject invalid
    accesses.
 
+3.2.XX.  layoutscope4
+
+   enum layoutscope4 {
+       LAYOUTSCOPE4_FILE           = 1,
+       LAYOUTSCOPE4_FSID           = 2,
+       LAYOUTSCOPE4_ALL            = 3,
+   };
+
+   The layoutscope4 enumerated type is used for CB_LAYOUTRECALL and LAYOUTRETURN
+   to identify the scope of the operation.  LAYOUTSCOPE4_FILE specifies a
+   layout associated with the specified file. LAYOUTSCOPE4_FSID specifies all
+   layouts associated with the given filesystem. LAYOUTSCOPE4_ALL specifies
+   all layouts associated with the client ID.
+
 3.2.24.  nfs_impl_id4
 
    struct nfs_impl_id4 {
@@ -12659,7 +12661,7 @@
 
    based protocols, block allocation could occur when a READ/WRITE
    iomode is specified.  A special LAYOUTIOMODE4_ANY iomode is defined
-   and can only be used for LAYOUTRETURN and LAYOUTRECALL, not for
+   and can only be used for LAYOUTRETURN and CB_LAYOUTRECALL, not for
    LAYOUTGET.  It specifies that layouts pertaining to both READ and
    READ/WRITE iomodes are being returned or recalled, respectively.
 
@@ -13051,16 +13053,16 @@
 
 12.5.3.3.  LAYOUTCOMMIT and layoutupdate
 
-   The LAYOUTCOMMIT argument contains a loca_layoutupdate field
-   (Section 17.42.2) of data type layoutupdate4 (Section 3.2.21).  This
+   The LAYOUTCOMMIT argument contains am opaque loca_layoutupdate field
+   (Section 17.42.2).  This
    argument is a layout type-specific structure.  The structure can be
    used to pass arbitrary layout type-specific information from the
    client to the metadata server at LAYOUTCOMMIT time.  For example, if
    using a block/volume layout, the client can indicate to the metadata
    server which reserved or allocated blocks the client used and did not
-   use.  The content of loca_layoutupdate (field lou_body) need not be
+   use.  The content of loca_layoutupdate need not be
    the same the layout type-specific content returned by LAYOUTGET
-   (Section 17.43.3) in the loc_body field of the lo_content field, of
+   (Section 17.43.3) in
    the logr_layout field.  The content of loca_layoutupdate is defined
    by the layout type specification and is opaque to LAYOUTCOMMIT.
 
@@ -13698,9 +13700,9 @@
       of the metadata server.  The metadata server applies some
       consistency checks on the loca_layoutupdate field of the arguments
       to determine whether the client can commit the data written to the
-      data server to the filesystem.  The loca_layoutupdate field is of
-      data type layoutupdate4, and contains layout type-specific content
-      (in the lou_body field of loca_layoutupdate).  The layout type-
+      data server to the filesystem.  The loca_layoutupdate field is
+      opaque, and contains layout type-specific content.
+      The layout type-
       specific information that loca_layoutupdate might have is
       discussed in Section 12.5.3.3.  If the metadata server's
       consistency checks on loca_layoutupdate succeed, then the metadata
@@ -14147,7 +14149,7 @@
            nfs_fh4         nfldf_fh;
    };
 
-   %/* Encoded in the loc_body field of type layout_content4: */
+   %/* Encoded in the loc_body field of type layout4: */
    struct nfsv4_1_file_layout4 {
            stripetype4                     nfl_stripe_type;
            bool                            nfl_commit_through_mds;
@@ -14158,8 +14160,8 @@
    };
 
    %/*
-   % * Encoded in the lou_body field of type layoutupdate4:
-   % *      Nothing. lou_body is a zero length array of octets.
+   % * Encoded in loca_layoutupdate:
+   % *      Nothing. loca_layoutupdate is a zero length array of octets.
    % */

@@ -23166,15 +23168,15 @@
            void;
    };
 
+
    struct LAYOUTCOMMIT4args {
            /* CURRENT_FH: file */
-           offset4                 loca_offset;
-           length4                 loca_length;
+           layoutsegment4          loca_seg;
            bool                    loca_reclaim;
            newoffset4              loca_last_write_offset;
            newtime4                loca_time_modify;
            newtime4                loca_time_access;
-           layoutupdate4           loca_layoutupdate;
+           opaque                  loca_layoutupdate<>;
    };

@@ -23213,7 +23215,7 @@
    SEQUENCE operation), and octet range.  Since layout segments are sub-
    dividable, a smaller portion of a layout segment, retrieved via
    LAYOUTGET, may be committed.  The region being committed is specified
-   through the octet range (loca_offset and loca_length).
+   through the octet range (loca_seg.ls_offset and loca_seg.ls_length).
 
    The LAYOUTCOMMIT operation indicates that the client has completed
    writes using a layout obtained by a previous LAYOUTGET.  The client
@@ -23285,10 +23287,14 @@
 
    The loca_layoutupdate argument to LAYOUTCOMMIT provides a mechanism
    for a client to provide layout specific updates to the metadata
-   server.  For example, the layout update can describe what regions of
+   server.
+   The contents of the opaque loca_layoutupdate field are
+   determined by the layout type and are defined in their context.
+   For example, the layout update can describe what regions of
    the original layout have been used and what regions can be
    deallocated.  There is no NFSv4.1 file layout-specific layoutupdate4
-   structure.
+   structure, thus the
+   loca_layoutupdate field should have a zero length.

@@ -23526,9 +23532,8 @@
    struct LAYOUTRETURN4args {
            /* CURRENT_FH: file */
            bool                    lora_reclaim;
-           layouttype4             lora_layout_type;
-           layoutiomode4           lora_iomode;
-           layoutreturn4           lora_layoutreturn;
+           layoutscope4            lora_scope;
+           layoutsegment4          lora_seg;
    };

@@ -23543,32 +23548,34 @@
 
    Returns one or more layouts or layout segments represented by the
    client ID (derived from the sessionid in the preceding SEQUENCE
-   operation), lora_layout_type, and lora_iomode.  When layoutreturn is
-   LAYOUTRETURN4_FILE the returned layout segment is further identified
-   by the current filehandle, lrf_offset, and lrf_length.  When
-   layoutreturn is LAYOUTRETURN4_FSID the current filehandle is used to
-   identify the file system and all layouts or layout segments matching
-   the client ID, lora_layout_type, and lora_iomode are returned.  When
-   layoutreturn is LAYOUTRETURN4_ALL all layouts or layout segments
-   matching the client ID, lora_layout_type, and lora_iomode are
-   returned and the current filehandle is not used.  After this call,
+   operation), lora_seg.ls_layouttype, and lora_seg.ls_iomode.  When lora_scope is
+   LAYOUTSCOPE4_FILE the returned layout segment is further identified
+   by the current filehandle, ls_offset, and ls_length.  When
+   lora_scope is LAYOUTSCOPE4_FSID the current filehandle is used to
+   identify the file system; all layouts matching
+   the client ID, ls_layouttype, and ls_iomode are returned and ls_offset and ls_length
+   are not used and should be set to zero.
+   When lora_scope is LAYOUTSCOPE4_ALL all layouts 
+   matching the client ID, ls_layouttype, and ls_iomode are
+   returned and the current filehandle, ls_offset, and ls_length are not used;
+   ls_offset and ls_length should be set to zero.  After this call,
    the client MUST NOT use the returned layout segment(s) or layout(s)
    and the associated storage protocol to access the file data.  A
    layout segment being returned may be a subdivision of a layout
    segment previously fetched through LAYOUTGET.  As well, it may be a
    subset or superset of a layout segment specified by CB_LAYOUTRECALL.
    However, if it is a subset, the recall is not complete until the full
-   recalled scope (LAYOUTRETURN4_FILE octet range, LAYOUTRETURN4_FSID,
-   or LAYOUTRETURN4_ALL) has been returned.  It is also permissible, and
+   recalled scope (LAYOUTSCOPE4_FILE octet range, LAYOUTSCOPE4_FSID,
+   or LAYOUTSCOPE4_ALL) has been returned.  It is also permissible, and
    no error should result, for a client to return a octet range covering
-   a layout it does not hold.  If the lrf_length is all 1s, the layout
-   covers the range from lrf_offset to EOF.  An iomode of
+   a layout it does not hold.  If the ls_length is all 1s, the layout
+   covers the range from ls_offset to EOF.  An ls_iomode of
    LAYOUTIOMODE4_ANY specifies that all layouts that match the other
-   arguments to LAYOUTRETURN (i.e., client ID, lora_layout_type, and one
+   arguments to LAYOUTRETURN (i.e., client ID, ls_layouttype, and one
    of current filehandle and range; fsid derived from current
-   filehandle; or LAYOUTRETURN4_ALL) are being returned.
+   filehandle; or LAYOUTSCOPE4_ALL) are being returned.
 
-   When lr_returntype is set to LAYOUTRETURN4_FSID or LAYOUTRETURN4_ALL
+   When lora_scope is set to LAYOUTSCOPE4_FSID or LAYOUTSCOPE4_ALL

@@ -23588,7 +23595,7 @@
    server's grace period.  When returning layouts that were acquired
    during the metadata server's grace period MUST set the lora_reclaim
    field to FALSE.  The lora_reclaim field MUST be set to FALSE also
-   when lr_layoutreturn is LAYOUTRETURN4_FSID or LAYOUTRETURN4_ALL.  See
+   when lora_scope is LAYOUTSCOPE4_FSID or LAYOUTSCOPE4_ALL.  See
    LAYOUTCOMMIT (Section 17.42) for more details.
 
    Layouts may be returned when recalled or voluntarily (i.e., before
@@ -23610,7 +23617,7 @@
    the metadata server's grace period, NFS4ERR_NO_GRACE is returned.
 
    If the LAYOUTRETURN request sets the lora_reclaim field to TRUE and
-   lr_returntype is set to LAYOUTRETURN4_FSID or LAYOUTRETURN4_ALL,
+   lr_returntype is set to LAYOUTSCOPE4_FSID or LAYOUTSCOPE4_ALL,
    NFS4ERR_INVAL is returned.
 
    On success, the current filehandle retains its value.
@@ -23624,7 +23631,7 @@
    callback MUST be serialized with any outstanding, intersecting
    LAYOUTRETURN operations.  Note that it is possible that while a
    client is returning the layout for some recalled range the server may
-   recall a superset of that range (e.g.  LAYOUTRECALL4_ALL); the final
+   recall a superset of that range (e.g.  LAYOUTSCOPE4_ALL); the final

@@ -23637,13 +23644,13 @@
    recall is done - when its corresponding final return operation is
    replied.
 
-   Returning all layouts in a file system using LAYOUTRETURN4_FSID is
+   Returning all layouts in a file system using LAYOUTSCOPE4_FSID is
    typically done in response to a CB_LAYOUTRECALL for that file system
-   as the final return operation.  Similarly, LAYOUTRETURN4_ALL is used
+   as the final return operation.  Similarly, LAYOUTSCOPE4_ALL is used
    in response to a recall callback for all layouts.  It is possible
    that the client already returned some outstanding layouts via
-   individual LAYOUTRETURN calls and the call for LAYOUTRETURN4_FSID or
-   LAYOUTRETURN4_ALL marks the end of the LAYOUTRETURN sequence.  See
+   individual LAYOUTRETURN calls and the call for LAYOUTSCOPE4_FSID or
+   LAYOUTSCOPE4_ALL marks the end of the LAYOUTRETURN sequence.  See
    Section 12.5.4.1 for more details.
 
 17.45.  Operation 52: SECINFO_NO_NAME - Get Security on Unnamed Object
@@ -24709,32 +24716,19 @@
     * NFSv4.1 callback arguments and results
     */
 
-   enum layoutrecall_type4 {
-           LAYOUTRECALL4_FILE = 1,
-           LAYOUTRECALL4_FSID = 2,
-           LAYOUTRECALL4_ALL  = 3
-   };
-
-   struct layoutrecall_file4 {
-           nfs_fh4         lor_fh;
-           offset4         lor_offset;
-           length4         lor_length;
-   };
-
-   union layoutrecall4 switch(layoutrecall_type4 recalltype) {
-   case LAYOUTRECALL4_FILE:
-           layoutrecall_file4 lor_layout;
-   case LAYOUTRECALL4_FSID:
-           fsid4              lor_fsid;
-   case LAYOUTRECALL4_ALL:
+   union layoutrecall4 switch(layoutscope4 clora_scope) {
+   case LAYOUTSCOPE4_FILE:
+           nfs_fh4            clora_fh;
+   case LAYOUTSCOPE4_FSID:
+           fsid4              clora_fsid;
+   case LAYOUTSCOPE4_ALL:
            void;
    };
 
    struct CB_LAYOUTRECALL4args {
-           layouttype4             clora_type;
-           layoutiomode4           clora_iomode;
            bool                    clora_changed;
            layoutrecall4           clora_recall;
+           layoutsegment4          clora_seg;
    };

@@ -24758,18 +24752,18 @@
    The CB_LAYOUTRECALL operation is used to begin the process of
    recalling layout segments, a layout, all layouts pertaining to a
    particular file system (FSID), or layouts in all file systems (ALL).
-   If LAYOUTRECALL4_FILE is specified, the lrf_offset and lrf_length
-   fields specify the layout segments.  If a lrf_length of all ones is
+   If LAYOUTSCOPE4_FILE is specified, the clora_seg.ls_offset and clora_seg.ls_length
+   fields specify the layout segments.  If a ls_length of all ones is
    specified then all layout segments identified by the current file
-   handle, clora_type, clora_iomode, and corresponding to the octet
-   range from lrf_offset to the end-of-file MUST be returned (via
-   LAYOUTRETURN, see Section 17.44).  The clora_iomode specifies the set
-   of layouts to be returned.  An clora_iomode of LAYOUTIOMODE4_ANY
+   handle, ls_layouttype, ls_iomode, and corresponding to the octet
+   range from ls_offset to the end-of-file MUST be returned (via
+   LAYOUTRETURN, see Section 17.44).  The ls_iomode specifies the set
+   of layouts to be returned.  An ls_iomode of LAYOUTIOMODE4_ANY
    specifies that all matching layout segments regardless of iomode,
    must be returned; otherwise, only layout segments that exactly match
-   the iomode must be returned.  If clora_iomode is LAYOUTIOMODE4_ANY,
-   lo_offset is zero, and lo_length is all ones, then the entire layout
-   is to be returned.
+   the iomode must be returned.  If ls_iomode is LAYOUTIOMODE4_ANY,
+   ls_offset is zero, and ls_length is all ones, then the entire layout
+   for the file is to be returned.
 
    If the clora_changed field is TRUE, then the client SHOULD not write
    and commit its modified data to the storage devices specified by the
@@ -24785,17 +24779,19 @@
    overlapping with the requested layout, it returns
    NFS4ERR_NOMATCHING_LAYOUT.
 
-   If LAYOUTRECALL4_FSID is specified, the fsid specifies the file
-   system for which any outstanding layouts MUST be returned.  If
-   LAYOUTRECALL4_ALL is specified, all outstanding layouts MUST be
-   returned.  In addition, LAYOUTRECALL4_FSID and LAYOUTRECALL4_ALL
+   If LAYOUTSCOPE4_FSID is specified, then clora_fsid specifies the file
+   system for which any outstanding layouts MUST be returned; ls_offset and
+   ls_length are not used and should be set to zero.  If
+   LAYOUTSCOPE4_ALL is specified, all outstanding layouts MUST be
+   returned; here too ls_offset and ls_length are not used and should be set to zero.
+   In addition, LAYOUTSCOPE4_FSID and LAYOUTSCOPE4_ALL
    specify that all the storage device ID to storage device address
    mappings in the affected file system(s) are also recalled.  The
-   respective LAYOUTRETURN with either LAYOUTRETURN4_FSID or
-   LAYOUTRETURN4_ALL acknowledges to the server that the client
+   respective LAYOUTRETURN with either LAYOUTSCOPE4_FSID or
+   LAYOUTSCOPE4_ALL acknowledges to the server that the client
    invalidated the said device mappings.  Device mappings are
-   invalidated also when no layouts are found for LAYOUTRECALL4_FSID or
-   LAYOUTRECALL4_ALL and NFS4ERR_NOMATCHING_LAYOUT is returned.
+   invalidated also when no layouts are found for LAYOUTSCOPE4_FSID or
+   LAYOUTSCOPE4_ALL and NFS4ERR_NOMATCHING_LAYOUT is returned.
 
 19.3.5.  IMPLEMENTATION
 
@@ -25633,10 +25629,9 @@
 
           +  the loh_body field of the layouthint4 data type;
 
-          +  the loc_body field of layout_content4 data type (which in
-             turn is the lo_content field of the layout4 data type);
+          +  the lo_body field of the layout4 data type;
 
-          +  the lou_body field of the layoutupdate4 data type;
+          +  the loca_layoutupdate field of the LAYOUTCOMMIT arguments;
 
        *  Describe or define the storage access protocol used to access
           the data servers

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Garth Goodson <[email-masked]>
Date: Mon, 21 May 2007 10:19:18 -0700

> 3. I haven't defined the use for ls_iomode in LAYOUTCOMMIT since this is another

I'm slightly concerned about the varying semantics of some of the fields 
depending on the operation in use (when to set what to zero/ignore). 
Although, I do see some of the additional benefits, so if others don't 
think this is a big deal, I would be ok with it.

I'm also not convinced that iomode is required for layoutcommit.  I'm 
not sure it adds anything over what can already be achieved, although we 
can leave this for another discussion thread.

-Garth

---

From: "Muntz, Daniel" <[email-masked]>
Date: Mon, 21 May 2007 12:21:57 -0700

Hi all,

The startup script for idmapd checks to see if the sunrpc module is
loaded, and if the module is not loaded, idmapd doesn't start.  However,
if you build sunrpc into the kernel statically, the test fails and
idmapd doesn't start.  I'm not sure which distributions share this
startup script, but FC and RHEL both have this problem.

  -Dan