Subject: Re: NFSv4 Locking case study

From: Mike Eisler <[email-masked]>
Date: Sun, 2 Mar 2003 15:39:33 -0800 (PST)

> From: Stevan Steve Allen <[email-masked]>
> Date: Thu, 27 Feb 2003 01:28:03 -0800

[...]
> Given applications which support byte range, or file share, semantics are
> considered well behaved Applications which do not support such semantics
> are considered rouge applications according to file access needs of various
> customer solutions and platforms.  Such rouge applications may need to be
> restricted from access.

I don't think there's anything you can do about applications
that refuse to honor the application specific handshake for
accessing the file.

> Need:

> To ensure proper semantics, range level sharing solutions for performance
> may require file share only or stateid of all zeros implementations to be

Note that all bits zeros stateids come from the client, not the application.

> restricted.  Clients should not interpret this error as re-tryable.  For
> reference, the NFSv4 draft mentions similar byte range semantic enforcement
> (chpt. 8. "for example, some UNIX-based servers").   We suspect a need for
> a specific error to represent this condition.   NFS4ERR_LOCKED is not
> appropriate as it signifies a re-tryable condition.  We are proposing an
> permanent error to be returned to the application layer.

What if the application then decides to retry? How has this
solved anything?

> Question.  OPENs rflags "OPEN4_RESULT_LOCKTYPE_POSIX indicates the server's
> file locking behavior supports the complete set of Posix locking
> techniques."  I do not believe this indicates I/O requests require
> proper<?> byte range locking is required.

Correct. If he server returns NFS4ERR_LOCKED though, this
tells the client that byte range locking is needed.

---

From: Stevan Steve Allen <[email-masked]>
Date: Fri, 7 Mar 2003 11:36:51 -0800

Back on the public forum.  This discussion is in regards to the error being
returned by UNIX-based servers described in 8.1.4. which require locks
being held prior to I/O.

David...

Let's say mandatory byte range locking is being implemented as a
performance solution, and the platform semantics for byte range requires
proper locking to be obtained *prior to the I/O as mentioned in 8.1.4.

For POSIX compliant applications using proper byte range locks and possibly
lock promotions, we understand the file update semantics for modifying
existing ranges of the UNIX-based server as describing an application which
acquires a lock prior to a read and releasing the lock after the write to
ensure atomic updates and buffer validity.  Robust clients which perform
locking on behalf of the application around each separate read or write
request do not provide the same buffer integrity semantics as standard byte
range locking applications and are considered rogue applications in some
environments.

To us, the NFSv4 client is the first application which has introduced to
our file access services the concept of an application with morphing lock
powers beyond our expected semantics and ability to detect.  Without the
new robust client, we believe a standard non-locking application will
either retry the same failing request or report the error to the user.
Most all file access implementations are prepared for same request retry
semantics.

Standard applications do not try out various locking techniques until
successful.  For our environment we are concerned about these new helpful
lock morphing clients, not the standard applications behind them.

For some server platforms, a zero stateid or no lock held condition will
indicate a rogue client or rouge/buggy application which can not be
supported. The client either does not support byte range protocols and will
continue to fail, or the client may morph into some unknown and
undetectable locking semantic as described above.   Cases where a client
sends a zero stateid, or requests an I/O for which no-one holds a range
lock can be easily detected by the server using the NFSv4 mandatory test
lock.  This may represent a one time chance to catch the overly helpful
client before he puts on a locking mask which may violate expected
semantics.

Given knowledgeable applications get it right the first time, on 1st
detection of improper locking semantics by the server, we wish to stop the
client from performing it's robust locking magic and return a "halt" error
back to the non-compliant application in order to ensure the semantics for
atomic I/Os are being enforced (given a stable environment).

Read:

Depending on the size of the read operation, the request may be broken into
multiple RPCs due to buffer constraints of a platform or transmission
protocol.   A resulting non-atomic read operation would be similar to
watching a movie and have it change to another movie just before the big
ending.  If the solution requires atomic reads, a restrictive error should
be returned which is not interpreted by the client as retryable by changing
locking semantics.

General proposal:

To reduce non-atomic atomic I/O problems we are thinking.. if the server
can get the error past the client back to the application layer, it is
believed the application will not change locking semantics and will either
retry the exact same request which is proven to be detectable if the error
still exists, or halt the request and return the error to the user.  If
only we can get the error past the client.

Proposed Errors:

NFS4ERR_LOCKED as some have proposed may provoke further attempts using
alternate locking techniques by the client.  NFS4ERR_LOCKED is being
rejected for this discussion.

We believe NFS4ERR_ACCESS, NFS4ERR_INVAL and NFS4ERR_ADMIN_REVOKED already
perform the reaction we desire. e.g. Pass error back to application which
will retry the same detectable* request or stop.  Overloading these values
do not have the client interpretation problem as with NFS4ERR_LOCKED as the
error is always passed back to the application layer which is more
predictable.

If there was only the need to present an access denied error when improper
writes are detected,  I would recommend using the existing NFS4ERR_ACCESS
to fail the request.  Meaning: "Access is denied due to improper locking
and we can not handle your mighty morphing powers" should be included in
the NFS4ERR_ACCESS error description.  As much as I would enjoy such
verbiage, NFS4ERR_ACCESS is not available for read.  The same error should
be available for read and write.

NFS4ERR_ADMIN_REVOKED and NFS4ERR_INVAL are available for both read and
write.  We believe these errors will always be passed by the client back to
the calling application layer.  And further believe standard applications
will respond in a predictable way.  Either retrying the exact same request
which is proven detectable or halt the request.

Final mention:

It was noted that ideally servers will be able to detect any malformed lock
according to their underlying semantics and this is not a problem.
Realistically, this represent future effort as most platforms are currently
unprepared for new unknown semantics and even the underlying OS does not
have this extreme checking based on the assumption that all lockers use
compliant semantics.

---

From: Mike Eisler <[email-masked]>
Date: Sat, 8 Mar 2003 10:12:08 -0800 (PST)

> only we can get the error past the client.

OK. I now understand what you want. You want an error code that
prevents the NFS filesystem layer from automatically acquiring the lock
on the byte range, and so force the applicattion to explicitly get the
necessary byte range lock.

This is reasonable except for the following:

- Introduction of this error will break existing POSIX applications
	However, to avoid such breakage, the specification
	could give POSIX-based NFSv4 clients an "out": Unless the application
	has explicitly requested otherwise, the new error will not be
	propogated to the application.

- You haven't listed any existing operating environments that will
	take advantage of this. We'd probably want to identify these
	environments so that an appropriately named flag for the OPEN
	return results is listed.

This is too late for NFSv4.0. It would be reasonable to consider
for a future minor version. I suggest making a proposal, in
the form of an individual internet-draft submited to ietf, and ask
the wg to review it.

> error is always passed back to the application layer which is more

Except that these errors already have other meanings. _ACCESS is closest
to what you want, but not distinct enough. _INVAL and _REVOKED are
completely inappropriate IMHO, and are very unlikely to cause
applications to behave they way you want (the app is more likely
to exit with a fatal error).

	-mre