Subject: [nfsv4] Eliminating stateid's from pnfs IO

From: "J. Bruce Fields" <[email-masked]>
Date: Wed, 21 Sep 2005 14:19:49 -0400

Why shouldn't pNFS clients use only special stateid's for all IO?

>From http://www.ietf.org/internet-drafts/draft-welch-pnfs-ops-03.txt:

	"The client MUST use the stateid representing open or lock state
	as returned by an earlier metadata operation (e.g., OPEN, LOCK),
	or a special stateid to perform I/O on the data-servers (as in
	regular NFSv4)."

I don't understand why we're requiring this.

Stateid's perform two purposes that I can think of. First, they provide
protection against replays that cross state-transition boundaries.  (For
example, if a client flushes some write data and then sends an unlock,
there is a risk those writes could be replayed after the unlock.  The
stateid change on unlock allows the server to reject those replays.)

Second, stateid's tie I/O to lockowners, allowing the server to
determine whether a given I/O request conflicts with an existing
mandatory file lock.

I believe sessions solve the replay problem completely.

The second problem should also be solveable using sessions: the session
allows the server to determine the client that sent a given I/O request,
thus allowing it to recognize inter-client conflicts.  If the client
itself can be relied on to handle conflicts between local processes,
then we're done.

What am I missing?

The current draft's stateid requirement necessitates a high-performance
nfsv4-specific server-to-server protocol, which seems likely to
complicate pNFS implementation.  So it would obviously be nice if we
could eliminate that requirement.

At the very least I think the draft needs a clear statement of the
problem that stateid's are meant to solve.  Maybe I'm missing it....

--b.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Halevy, Benny" <[email-masked]>
Date: Fri, 23 Sep 2005 16:21:29 -0400

> One remaining thing I'm confused about, that effects the 
> requirements on
> servers, is whether in 4.0 servers are *required* to use 
> stateid's like
> file descriptors: for example, is the server required to use 
> the stateid
> when deciding whether to grant access when a mode bit (or acl) change
> since the original open has denied access granted by the open?  (E.g.,
> an open O_CREAT|O_RDRW with mode bits that deny write is usually made
> possible by a server hack that allow the file owner to bypass access
> checks; in v4 a server can turn off that hack for accesses that don't
> use an appropriate stateid and principal.)

No, as far as I know there's no guarantee for access based on open state.
Remember that NFS OPEN is decoupled from open() syscalls on the client and
some of them may be hidden from the server.  I/O ops may be issued with
different credentials than the one used to open the file and therefore need
to go through access control at I/O time.  If the client used credentials X
to open the file and later uses credentials Y to access the file and at that
later time access for Y is disallowed the server has no way to know that the
file was ever successfully opened for Y as the client might had only
verified that Y is allowed to open the file using ACCESS and not open.

This quote from section 14.2.16. (Operation 18: OPEN - Open a Regular File)
give more background on that subject.

   Based on the share_access value (READ, WRITE, or BOTH) the client
   should check that the requester has the proper access rights to
   perform the specified operation.  This would generally be the results
   of applying the ACL access rules to the file for the current
   requester.  However, just as with the ACCESS operation, the client
   should not attempt to second-guess the server's decisions, as access
   rights may change and may be subject to server administrative
   controls outside the ACL framework.  If the requester is not
   authorized to READ or WRITE (depending on the share_access value),
   the server must return NFS4ERR_ACCESS.  Note that since the NFS
   version 4 protocol does not impose any requirement that READs and
   WRITEs issued for an open file have the same credentials as the OPEN
   itself, the server still must do appropriate access checking on the
   READs and WRITEs themselves.

Benny

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]]On Behalf Of
> J. Bruce Fields
> Sent: Friday, September 23, 2005 8:43 PM
> To: Garth Goodson
> Cc: [email-masked]; Noveck, Dave
> Subject: Re: [nfsv4] Eliminating stateid's from pnfs IO
> 
> 
> On Fri, Sep 23, 2005 at 09:52:02AM -0700, Garth Goodson wrote:
> > Is this the section that causes most grief, if not let me 
> know which is?:
> > 
> > 5.2 Global Stateid Requirements
> > 
> > The stateid used for I/O MUST have the same effect and be 
> subject to the 
> > same validation on storage device as it would if the I/O was being 
> > performed on the metadata server itself (in the absence of 
> pNFS). This 
> > has the implication that stateids are globally valid on both the 
> > metadata and storage devices. This requires the metadata server to 
> > propagate changes in lock and open state to the 
> data-servers, so that 
> > the data-servers can validate I/O accesses.
> 
> Sorry, yes, that's what I was thinking of.
> 
> > There are some things in section 5.4, but this is more to 
> do with lock, 
> > open-state, and ACL propagation.  It mentions stateids, but 
> nowhere does 
> > it explicitly mention stateids must be propagated.  I think we both 
> > agree that I/Os must have the same effect (in terms of 
> validation) as 
> > they would on the metadata server, and I think you are 
> saying that this 
> > can be achieved in ways other than propagating stateids.  
> If so, I would 
> > probably be ok with looser language, such as:
> > 
> > I/Os using stateids obtained from the metadata server MUST 
> have the same 
> > effect, in terms of access, lock, and open state validation.  The 
> > simplest mechanism to achieve this is through the propagation of 
> > stateids to the data servers, and the validation thereof; 
> however, other 
> > mechanisms may exist to ensure the desired effect is obtained.
> 
> That might reduce the burden on the server maintainer.
> 
> One remaining thing I'm confused about, that effects the 
> requirements on
> servers, is whether in 4.0 servers are *required* to use 
> stateid's like
> file descriptors: for example, is the server required to use 
> the stateid
> when deciding whether to grant access when a mode bit (or acl) change
> since the original open has denied access granted by the open?  (E.g.,
> an open O_CREAT|O_RDRW with mode bits that deny write is usually made
> possible by a server hack that allow the file owner to bypass access
> checks; in v4 a server can turn off that hack for accesses that don't
> use an appropriate stateid and principal.)
> 
> > Note: I'm not willing to say anything about clients enforcing 
> > intra-client accesses.  That is a discussion you'll have to 
> have with 
> > the client guys...
> 
> OK.  I need to catch up on the other threads and think about this some
> more, thanks.--b.
>