Subject: RE: idea - atomic append for NFS writes

From: Mike Eisler <[email-masked]>
Date: Tue, 27 Mar 2001 10:48:54 -0800 (PST)

>> >The WRITE_APPEND op contains a "record-length" as well as the
>> >"data-length", and the return value includes a "record-offset".
>> >The record-length indicates how much data the client wants to
>> >atomically append.
>> >The data-length indicates how much data is in the request.  This bit
>> >goes at the end (or maybe the beginning, whichever is easier for
>> >servers). The remainder of the record (if any) is filled with nuls.
>> >The "record-offset" indicates the file address of the start of the
>> >appended record.  The client can then fill in the remainder with
>> >normal writes.
>> 
>> This is a cheap form of offset reservations in stable storage
>> that Peter Staubach mentioned. And my objection is the same:
>> what if the client doesn't follow through?
>
>What if a client does an EXCLUSIVE CREATE and doesn't follow through
>with a SETATTR?  You get a file with very odd metadata.

Usually a file with an odd modification time. 9 years of experience
says it's ok. I think this pales in comparision to a block
of zeros in a log file.

>What if a client does an unstable write and never follows through
>with a COMMIT, (and so doesn't discover that the server restarted and
>some of the data was lost).  A different client could have seen the

To get that situation you have to have the both client and server fail
during the window. In any case, since I don't subscribe to the file
sharing myth, the worse that will happen is that a client failed to
issue a COMMIT before it wrote data. That's no different than the
client failing to flush its dirty cache with a stable write. NFSv3's
safe asynchronous writes are safe.

>new data before the server restart, and then suddenly sees old data
>after the restart.  Very odd.

The server could, and technically should commit dirty data before allowing
another client to read it.

>You could avoid the issue with append by sending a sequence of "I am
>going to do a big write_append soon, here is part of the data I will
>be using" requests followed by a "ok, here is the last bit, do the
>append now" request.

I don't see how you avoid blocks of zeroes in the file.

>> >  2/ allowing the server to signal 'recall' in a non CLAIM_PREVIOUS
>> >     delegation, particularly in response to an "append" open.
>> 
>> I've no idea what that means. Recall means callback. If there is
>> another client that wants to read or write the file, you have to issue
>> a callback; return an error to the second client; or allow the access
>> without a callback, the latter two being incorrect.
>
>Normally a delegation is recalled with a call back and the server will
>not issue a delegation without confirming that it can make a callback.
>However there is (currently) one exception to this.
>
>In section 9.2.1, the paragraph which start "When the server reboots
>or restarts..."  dicusses a scenario where the server may give a
>delegation without first establishing that a calback can be made.
>This happens when the client opens with CLAIM_DELGATE_PREV.
>In this case "a special designation is applied so that the client
>treats the delegation as having been granted but recalled by the
>server."
>
>I am suggestion that in response to an OPEN(append), the server might
>apply a similar designation to the delegation that is given.  The
>client would then perform the write request to append the data as
>necessary, and then would return the delegation immediately.

Sounds like you are simply creating a new type of lock:

	open(append) - to get "delegation" that 
		is really the I/O lock on the file
	write
	return "delegation " to server

Given that, why do you need WRITE_APPEND? As long as  open(append)
is compounded with a getattr to return the file size, WRITE_APPEND
is superfluous right?

The client returns the "delegation" as soon as it is done writing its
record. This allows multiple clients doing an append to do their thing
with the correct semantics. WRITEs and SETATTRs done with non-append
open that change EOF are held off until the append "delegation" is
returned.

I'm sure I'm missing something.

	-mre

---

From: Neil Brown <[email-masked]>
Date: Wed, 28 Mar 2001 11:45:52 +1000 (EST)

> I'm sure I'm missing something.

Not much I suspect.  Possibly some hidden assumption...

Yes, it is a lot like a new type of lock.
It's not a byte-range-lock, because those are normally advisory
(though that can be mandatory for some files).
It's not a reservation because it only wants to restrict access for a
short period of time.
And it's a lot like a delegation, as it promises "now else will change
this file until the delegation is returned".

It says ``I don't mind if someone else has a "read" reservation on this
file, or even a "write" reservation (but a DENY_WRITE would be a
problem), I just want you to make sure that they don't actually write
anything for a couple of round-trips.''

If this is implemented, then the WRITE_APPEND is not needed.  The
reason for suggesting WRITE_APPEND is that it is my understanding
(correct me if I am wrong) that some server implementations might not
be able to provide any exclusive access to a file at all.  They may
simply not be able to implement delegations (or DENY_* reservations).
A  user-level server running on Unix would be a good example.

I wanted to provide such a server with the opporunity to provide
best-effort O_APPEND semantics.  You claim that the best-effort isn't
good enough, and I admit that my enthusiasm for it is waning.

NeilBrown