Subject: RE: Esoterica (Was append writes)

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 20 Mar 2001 08:14:45 -0800

In addition to other stuff, Mike Eisler wrote:
> I do agree with you about the .nfs files in NFSv4. They should go away
> in v4.0.

There was a big discussion (in length, not in participants) about 
this at Connectathon.  The upshot was that Kendrick put silly rename
back in his client and that's the way we tested.  I'm not thrilled
with that result.

Since the spec doesn't say anything about this, this should certainly
be nailed down in the next go-round.  Spencer is of the opinion that 
this stuff cannot be removed in v4.  I'll let him give his reasons.
	
> p.s. While we are re-opening old cans of worms, anybody recall why
> exclusive create semantics for directories, symlinks, and special
> devices aren't in the NFSv4?

I looked in the archive.  There may be some later stuff but the latest
mentions of this I was able to find were:

     http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/1999/1032.html

in which I mentioned it more-or-less in passing.  And 

     http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/1999/1033.html

in which you said,

> Another thing I'd have liked to see, which I'm sure I mention once or
twice in 
> last 3 years, was proper support for non-idempotent exclusive CREATE of 
> non-regular files. But this isn't pressing either. Since we've reached 
> consensus on minor versioning, there's a plethora of stuff that can be 
> considered later.

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 22 Mar 2001 15:58:57 -0600

> this stuff cannot be removed in v4.  I'll let him give his reasons.

I will put it on the list of clarifications.

Now for my reasons...

To this point, we have added state to the NFSv4 server but have done
it in such a way that the server is not required to store more state
than can be normally represented in the target file system.  Locking
state, open state, delegation state, will be reclaimed by the client
when a server restarts.  As an optimization the server can store
additional state for faster recovery.

To remove the need for the client to rename a file to .nfsXXXX when a
client removes the file and that file is open, the server will need to
move that file to another location in the filesytem itself or
effectively hide it from the clients.  If the server fails, the server
will now have to find and deal with these removed but accessible files
as part of its recovery procedure.  This type of state recovery is a
departure from the current model and seems difficult to provide
consistently amongst a set of disparate file systems at the server.

-- 

- Spencer -

---

From: Neil Brown <[email-masked]>
Date: Fri, 23 Mar 2001 10:39:31 +1100 (EST)

> consistently amongst a set of disparate file systems at the server.

Maybe I'm being too unix-o-centric, but:

   translate "unlink or last link" to "rename to lost+found/uniqname"

Unix servers already need to cope with open-but-unlinked files
magically appearing in lost+found after a server restart so doing this
doesn't make life any harder.  The only change to recovery would be
that files should not be removed from lost+found until after the grace
period was over, and then only if they haven't been accessed since
reboot.

I'm not certain of the situation with Win32 system, but if they allow
a file to be unlinked-while-open, then they must already have some
mechanism for cleaning up after a restart.  I seem to remember seeing
things like "c:\file.001" on a Win95 system.  I assume that NTFS does
similar things.

But if you think that has to wait until 4.1, I guess that's your call
to make.

NeilBrown

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 22 Mar 2001 18:50:45 -0600

> doesn't make life any harder.  The only change to recovery would be

I know of at least one unix system (AIX) that does not follow this
scenario.  It manages the allocations for the unlinked/open file in
such a way that if the server reboots, the allocations are freed as a
result of the reboot.  AIX keeps two maps for the allocations.  A
permanent map and a working map.  The permanent map represents the
unlinked state of the file and the working map represents the
opened/referenced state.  Server reboots; working map lost, permanent
map used.  In this example, the amount of work to support the unlinked
but available file will be problematic.

> period was over, and then only if they haven't been accessed since

We have not discussed the issues of namespace yet either.  If the
server is using persistent filehandles, then the recovery will be done
by filehandle.  If it is providing this unlinked service along with
volatile filehandles, then it would need to keep a mapping of names to
'saved-but-unlinked' files.

> But if you think that has to wait until 4.1, I guess that's your call

The decision will be made by the working group.  At this point in our
work, I believe that a mechanism can be defined for the future that
allows a server to support this.  I will put this on our list of
things to be resolved before the next draft and we can, at the very
least, allow for additional flags or attributes that can enable this
in a minor version.

-- 

- Spencer -

---

From: Ric Werme <[email-masked]>
Date: Thu, 22 Mar 2001 21:06:48 -0500

> 'saved-but-unlinked' files.

I thought of that, but it seems to me that if the client unlinked the file,
that should be tacit acknowledgement that it won't be referring to the
file again by name.  If client A unlinks the file, what happens to client
B?  It may expect it can still use a pathname.  When client A does the close,
will client B still be able to read the file?   Keeping the .nfsxxx files
at least has the advantage of minimizing surprise.

I thought of another problem with rename to /lost+found within a file system.
Suppose /usr is a file system and I export /usr/foo.  I open /usr/foo/bar
and then unlink it.  If it gets renamed to /usr/lost+found/.nfsxxxx, 
will the client still be able to read it?  What about systems like Linux
that will say you need to reestablish the file handle after a rename?  Those
clients certainly will fail.

	-Ric Werme

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 22 Mar 2001 22:46:18 -0600

> at least has the advantage of minimizing surprise.

Your correct in the context of persistent filehandles (my mistake).
With volatile filehandles, the client needs a name to recover from an
EXPIRED error.

-- 

- Spencer -