Subject: [nfsv4] Another pNFS issue

From: Dean Hildebrand <[email-masked]>
Date: Tue, 5 Jul 2005 11:26:24 -0400 (EDT)

Another issue that I brought up at the end of our CITI pNFS meeting and we
ran out of time to talk about is the interaction of the layout retrieval
and locks.

The question is: When using mandatory locking, should a server return a
layout for a byte-range to a client that conflicts with a byte-range lock
owned by another client?

I would like to say that layouts and locks are independent, BUT, certain
layout types, e.g., block based, cannot support mandatory locks without
this interaction.  I'm assuming with our support for minor versioning
rules, it would not be ok to break mandatory locking in the protocol.
Example:
1. Client1 locks the range 10-100 in file a.
2. Client2 requests the layout for bytes 0-20 in file a.

Cases:
NFSv4 file- and Object-based layouts:
a) The server could propagate the lock information to the data servers to
ensure correctness.
b) Another option would be that the server recalls all outstanding layouts
that conflict with the lock before granting the mandatory byte-range lock.
All future layout requests that conflict with the lock are rejected.
c) Another option is that we state that is ok for pNFS supported file
systems to not support mandatory locks.

Block-based and other layouts with the inability to propagate lock
information to the servers:
a) Options b) or c) from above are the only options that I can see.  Which
one do we want to support?

Even if you think it should be underlying file system dependent, I think
we should have a statement in the draft.

Dean

On Mon, 20 Jun 2005, Garth Goodson wrote:

> There are a number of open issues and changes that were suggested for
> the pnfs draft ID (draft-welch-pnfs-ops-02.txt).
>
> Please post any feedback or opinions (or feel free to add more).
>
> Thanks,
> -Garth
>
>
> 3.1.1 Device IDs
>
> Proposal: device ids are valid only while a server is up, remappings
> while server is up must use a different device id.  (I prefer this)
>
> Alternative: device ids are attached to leases and may be timed-out
> (probably also need to be recallable or invalidated)
>
> 3.1.2 Aggregation Schemes
>
> Proposal: aggregation scheme (e.g., striping is part of opaque
> layout-type defined structure)
>
> Alternative: make a general striping aggregation that is outside of the
> opaque structure; push striping up a level... (it references layout-type
> opaque specific structures which contain the devices)
>
> 3.2.2 Operation sequencing
>
> Issue: Race condition exists between LAYOUTGET/LAYOUTRECALL passing each
> other on wire.  Sessions does not solve this since they are on different
> channels.  May require a seq-id to be returned in layout ops.
>
> 3.3.1 Identifying layouts
>
> Proposal: layouts are identified by <ClientID, FH, offset, length>
>
> Alternative: may need to distinguish between read vs. read/write layout
> types, e.g., block/obj layouts may have separate read vs. read/write
> layouts and will want to handle them separately.  Propose adding iomode
> to layout identification (and allowing recall to specify the specific
> mode or any mode).
>
> 3.3.3 Copy-on-write
>
> Same as discussion for 3.3.1
>
> 3.4 Recalling a LAYOUT
>
> Addition: Add a recall to recall all layouts pertaining to a specific fsid.
>
> Issue: Long callback recalls (if client has dirty data that needs to be
> flushed).  Mostly wording on how long client should have to write data
> while holding the layout.  It can always revert to sending the data
> through the metadata server if I/Os to data servers fail (due to server
> revoking layout).
>
> 3.5 Committing a layout
>
> Addition: add mtime/atime hints to LAYOUTCOMMIT (server can use them or
> not based on current state -- e.g., server should not allow time to go
> backwards and should not set the mtime if the mtime is already higher
> than that being set).  SETATTR used to specify an exact time and is
> constrained by regular V4 semantics.  The main difference between the
> time in LAYOUTCOMMIT and SETATTR is that the times set by SETATTR are
> mandatory vs. hints.
>
> 3.5.1 LAYOUTCOMMIT and EOF
>
> Change: instead of specifying newEOF with a flag (depending on whether
> client thinks it is setting a new EOF), client will specify the last
> byte to which it wrote.
>
> 5.1 File Striping and Data Access
>
> Change: simplify striping layout -- have enum for SPARSE vs. DENSE
> layout instead of skip and start offset
>
> Issue: think about what error gets returned if a client performs a
> non(READ/WRITE/PUTFH/COMMIT) at a data server; issue: this may be a
> problem if a regular nfsv4 data server is used as it has no way to
> differentiate accesses.
>
> 5.2 Global Stateids
>
> Issue: does not provide for unmodified NFSv4 data servers.  More
> thinking must be done if unmodified V4 servers are to be used as data
> servers.  See section: 5.7 for discussion.
>
> 5.3 I/O mode
>
> Change: Don't restrict I/O mode to be RW.  It may be useful to have
> read-only replicas to which a client can be directed if the iomode is READ.
>
> 5.4.1 Lock State Propagation
>
> Issue: can seq ids in stateids be ignored on the data servers (what
> about if sessions are used?)
>
> 5.4.3 Access State Propagation
>
> Issue: NFSv4 spec says that READs/WRITEs do not require same principal
> as OPEN.  This opens a security hole, but some implementations depend on
> it.  pNFS should probably go with the spec on this and not change the
> semantics.
>
> 6.3 pnfs_devaddr4
>
> Change: switch on layouttype4 instead of devaddrtypes4
>
> Change?: add disk signature to list of types
>
> 7 pnfs File Attributes
>
> Add: PREFERRED_ALIGNMENT, PREFERRED_BLOCKSIZE as FSID level attributes
>
> 9.2 LAYOUTCOMMIT
>
> Add: mtime/atime time attribute hints to args
>
> Change: neweof in result as per Object Internet Draft.  Basically, have
> a specific structure for each layout type that is the layoutcommit
> layout (vs. the layout received by layout get).  This allows extra
> opaque data to be sent on layoutcommit.
>
> General:
>
> IANA - think about whether additional layouttypes go through IANA or
> specification process.
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nfsv4
>

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4