Subject: Problems with fh_expire_type attribute

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 26 Feb 2002 20:22:14 -0800

I've been looking at how to manage filehandles in the
context of migration and see some problems with the 
fh_expire_type attribute as it is currently defined.

What I'm looking at is a way to allow filehandles to be
modified as a result of migration (i.e. allowing migration
where the source and target file systems are layed out
differently), without running afoul of the problem
that remapping-by-name of volatile filehandle for open
files (that have been renamed) can result in opening the
wrong file.  Re-opening the wrong file as a result of 
migration is just unacceptable.

So, what I supposed I would do, is that at migration
I would expire all filehandles for files that were not 
open.  Filehandles for files that were open would *not*
expire at that time.  Instead, as each file was closed
by the last client who had it open, the file handle would 
expire at that point and be subject to normal remapping.
Once every file that had been open at migration time
had been closed, then all the file handles (by this time
they are the native ones for new fs) would persist, until
the next migration.  To deal with server reboot after 
migration, it is necessary to keep the table of temporary
(i.e. old-fs-style) handles on disk, to ensure that you 
that a file which is accessed as part of reclaiming an open
(from before the migration) is treated properly.

This seems to me to be the best current strategy for managing
filehandles across migration, where persistent filehandles
cannot be used.  Now the question is, "What is the proper value
of fh_expire_type that correponds to this arrangement?".  The
answer is that, unfortunately, there isn't any.

Some out there are saying "Huh?", but bear with me.

Can you make it FH4_VOL_MIGRATION?  No, because the spec says
this means the filehandle *will* expire during system migration
and in this scheme, it may or may not, depending on whether the
file is open or not.  Further, some handles will expire a 
considerable time after migration, including cases where the
expiration will be seen by clients that never saw the original
migration.  I think they would be serverely confused if they
depended on the value FH$-VOL_MIGRATION.

How about FH4_VOL_MIGRATION|FH4_NOEXPIRE_WITH_OPEN.  Several 
problems with this.  The first is that the spec says that 
means that FH4_VOLATILE_ANY and FH4_VOL_RENAME do impact expiration
whien the file is open.  It does not say the same about
FH4_VOL_MIGRATION.  It does not say anything about this case.
In any case, the problem noted in the previous paragraph,
that some clients may see expiration without seeing migration 
applies here as well.

How about FH4_VOLATILE_ANY|FH4_NOEXPIRE_WITH_OPEN?  It might  
seem that'd be the ticket since migration time is part of any  
time as is final-close-after-mgration time.  However, for 
what appears to me to be no very good reason, the spec says  
that FH4_VOLATILE_ANY means not only that a handle may expire 
at any time but also that it will expire at migration and 
rename.  As far as migration goes, this is probably OK (if 
strange) since any file handle will be expire unless excepted 
by the NOEXPIRE_WITH_OPEN, but the rename is a killer.  The
file handles will not expire as a result of rename, open
or not. 

It seems to me that if the ANY bit is set it should mean
that handles may expire at any time and that's it.  Other 
information about what happens at migration or rename
can be communicated with bits that are defined for those
purposes.

So I would revise this stuff as follows:

FH4_PERSISTENT

     No change from the current spec.

FH4_VOLATILE_ANY

     May be set independently of FH4_VOL_MIGRATION and
     FH4_VOL_RENAME.

FH4_NOEXPIRE_WITH_OPEN

     May only be set if FH4_VOLATILE_ANY is set.  Indicates
     that when a file handle may be expired, it will not be
     expired if the file is open.  Note that this bit does
     not modify the effect of FH4_VOL_MIGRATION and 
     FH4_VOL_RENAME since those bits mandate that the filehandle 
     will be expired at certain times.

FH4_VOL_MIGRATION

     The filehandle will expire at migration.  No requirement
     that FH4_VOLATILE_ANY not be set.

FH4_VOL_RENAME

     The filehandle will expire due to a rename.  Note that 
     this is different than the current spec which says "may".
     Don't see the point of this.  How can a client use the 
     fact that a filehandle may expire as a result of rename?
     He can use the fact that it will expire to avoid having 
     to pass a filehandle that he knows is expired.  But if it
     just a case of it possibly being expired, how is this 
     different form FH4_VOLATILE_ANY?  The client has to save
     name information and has to be prepare to remap expired 
     file handles.

     Similar considerations make it undersirable for 
     FH4_NOEXPIRE_WITH_OPEN to appliy to FH4_VOL_MIGRATION
     and FH4_VOL_RENAME.  Those latter two bits allow the
     client to consider expiration as having occurred when 
     certain events happen without having to ask the server.
     Anything that makes that conditional means the client 
     has to check with the server anyway, so we might as
     well have FH4_VOLATILE_ANY|FH4_NOEXPIRE_WITH_OPEN.

With that rearrangement, my migration approach would work
comfortably with FH4_VOLATILE_ANY|FH4_NOEXPIRE_WITH_OPEN

Is anybody considering handle management approaches (either
for migration or anything else) that won't work with this 
way of doing things?  Is there anything the client can do
better with the information presented the way it is defined
in the spec currently?  I haven't been able to think of
any but there are more things in Heaven and Earth ...