Subject: RE: [nfsv4] re: ReleaseLockOwner

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 8 Sep 2003 12:52:02 -0700

> Well Dave, your response w.r.t. ReleaseLockOwner has reminded my about
> an area of the RFC that still seems unclear to me (yea, I'm a bit slow,
> ok, alright, I'm a Lot slow:-).

Of course it couldn't possibly be that the lack of clarity is a problem
with the spec, rather than with you :-)

> You mention that OpenOwners can be released when no outstanding Opens
> remain, but that LockOwners have no similar situation.

> But, isn't it true that LockOwners must be associated with OpenOwners
> and cannot exist after the associated OpenOwner goes away?
> (And, as such, can be released at the same time as the associated
> OpenOwner?)

I've just looked a bit at this area of the spec and I think there
are some problems with it, and I'll try to figure this stuff
out below.  But for now, as to your specific point, I don't think
that whichever way you come down on this issue, you dispense with
RELEASE_LOCKOWNER.  You can still have a an open file and have an
unbounded set of different lock_owner's that get locks on that open
file (not truly unbounded since there can only be 256 to the approx.
four-billionth power different lock owner strings but that's 
still a large number) so you may need a way to get rid of a 
lock_owner when it will no longer be used even if you can get rid
of the lockowner at close.

> The first time I coded this part of my server, I didn't assume a
> relationship, except for getting the new LockOwner, and that seemed
> fine until I bumped into Close. Close requires that the server either
> release all record locks associated with the Open or report an error.
> (See Sec. 14.2.2, second para in under "Description".) This implies
> a relationship between Opens and Locks and I've never been sure what
> this relationship is and how to maintain it?

The relationship that it implies is that each lock be assignable to the
open that it is associated with.  The open can be determined from the 
open stateid that generated the lock stateid that the lock is associated
with.  I think the uncertain point is how that relationship relates
to the lock_owner's and open_owner's.

> At this time, my server has a hierarchy of:
> OpenOwner
>	related Opens
>		related Lockowners
>			Locks (for that file opened by that OpenOwner)

> Now, I'm pretty well convinced this is not correct, since it requires the
> client to do a separate open_to_lock_owner4 for each of the files it
> wants to do locks on. (Using the same lock_owner string.)

Hmmm.  Offhand I can't see why this (separate open_to_lock_owner4 for each
file) is troublesome.

OK. Looking at the spec, I see the issue you are concerned about.  In the
last paragraph of IMPLEMENTATION under LOCK (14.2.10, p. 158) it says:

   The locker argument specifies the lock_owner that is associated with
   the LOCK request.  The locker4 structure is a switched union that
   indicates whether the lock_owner is known to the server or if the
   lock_owner is new to the server.  In the case that the lock_owner is
   known to the server and has an established lock_seqid, the argument
   is just the lock_owner and lock_seqid.  In the case that the
   lock_owner is not known to the server, the argument contains not only
   the lock_owner and lock_seqid but also the open_stateid and
   open_seqid.  The new lock_owner case covers the very first lock done
   by the lock_owner and offers a method to use the established state of
   the open_stateid to transition to the use of the lock_owner.

So the problem here is that this explains what to do if the lock_owner
is not known to the server, but doesn't correctly deal with the case where 
it is known.  Note that it says, "In the case that the lock_owner is
known to the server and has an established lock_seqid, the argument
is just the lock_owner and lock_seqid".  In fact the argument, is the
*lock_stateid* and lock_seqid.  My guess is that lock_stateid was
substituted for lock_owner relatively late in the editing process with
the idea that the lock_stateid gives you the right lock_owner.  The
problem that this creates however is that you can have a situation in
which the lock_owner is known to the server, but you still don't a
lock_stateid for the approrpiate file/open_owner/lock_owner 
combination.  What do you do in this case?  It seems the only way 
you can get a lock_stateid when you don't have one is to do the 
open_to_lock_owner case of LOCK, which the text quoted above says
you should only use if the lock_owner is not known to the server.

It seems to me that there are two ways to resolve this.  You can just
decide that the text about the use of open_to_lock_owner is wrong
(as it is about what is in the exist_lock_owner4 case) and that you 
simply use the open_to_lock_owner4 case when you have to get a 
lock_stateid when you don't have one for the current open_stateid and 
lock_owner.  This would be my preference.

An alternative is the vox spec vox dei approach which would simply
decide that if the spec doesn't provide for a particular case, that
case must somehow be foreclosed.  In that case, if a given lockowner
needed to get a lock on a different file or a different open file for
the same underlying fh, it would have to be represented by a different 
lockowner, so that the bad case of a known lock_owner with no lock_stateid
would not arise.  

The problem with this is that it is really not consonant with section 
8.1.3 where the lock_owner concept is introduced:

   o  A variable length opaque array used to uniquely define the owner
      of a lock managed by the client.

      This may be a thread id, process id, or other unique value.    

It would seem that a restriction that a given thread can only hold locks for
a single file is fairly counter-intuitive in this context given what would
normally expect a lock_owner to be (I guess politically it would be the extreme
Levelers' locking model: Why should one lock_owner be allowed to have locks for
multiple files when others don't? :-).  Also, if you had to create multiple 
lock_owner's to represent a single process locking two different files, it would
not be possible to find some very real deadlocks since thread A locking files
X and Y and thread B locking files Y and X would appear as four different
lock_owner's, and thus no deadlock is determinable.

> I'm now thinking the relationship is:
>			OpenOwner
>	related Opens			related LockOwners
> 					(ones that did open_to_lock_owner4
>					 using a stateid for any of the
>					 related Opens)
>						Locks (for all files that
>						       use this LockOwner)

It isn't clear from the above whether you are restricting each lock_owner
to locks for a given file, since you say above:

> Now, I'm pretty well convinced this is not correct, since it requires the
> client to do a separate open_to_lock_owner4 for each of the files it
> wants to do locks on. (Using the same lock_owner string.)

I'm wondering whether this hierarchy changes anything in that regard.

>From your hierarchy, it isn't clear to me how you deal with the case
of the same file (same fh, that is), when there are two different opens
(different open_owner's, that is) and the same lock_owner tries to get
locks associated with both opens.  Does your client structure prohibit 
that.  What would your server do if 

> Now, a Close looks for any Locks on any of the "related LockOwners" that
> are for the same file as the Close.
>
> In this case, there is still a relationship between OpenOwner and LockOwner.
> (If that relationship doesn't exist, I can't see any way of tying Locks
> to Opens, so that Close can be correctly implemented.)

Why not?  Our server treats open_owner's and lock_owner's as independent
(both are sub-ordinate to the associated client).  We then keep track of 
locks and opens in terms of State objects which are used to generate 
stateid sequences and can be found from a stateid.  Then in the hierarchy
open State objects are sub-ordinate to open_owner's and lock State objects
are tied both to the associated lock_owner and the containing open
State object (they are tied to the intersection of two related hierarchies).
Truth-in-engineering: although this gives us the wherewithal to either
free locks or check for their existence, it doesn't seem, at least from
a quick read of the code, that we actually make that check.  Having said
that, I'd better make sure this is fixed for the bakeoff in Austin. 

> What does everyone think? 

I'm curious about what exactly everybody has implemented in this area and
how we make sure that things are interoperable even in the face of situations
which are uncommon but possible.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: 08 Sep 2003 17:38:43 -0400

>>>>> " " == Dave Noveck <Noveck> writes:

     > It seems to me that there are two ways to resolve this.  You
     > can just decide that the text about the use of
     > open_to_lock_owner is wrong (as it is about what is in the
     > exist_lock_owner4 case) and that you simply use the
     > open_to_lock_owner4 case when you have to get a lock_stateid
     > when you don't have one for the current open_stateid and

     > An alternative is the vox spec vox dei approach which would
     > simply decide that if the spec doesn't provide for a particular
     > case, that case must somehow be foreclosed.  In that case, if a
     > given lockowner needed to get a lock on a different file or a
     > different open file for the same underlying fh, it would have
     > to be represented by a different lockowner, so that the bad
     > case of a known lock_owner with no lock_stateid would not
     > arise.

Why would you need a lock_seqid in the 'open_to_lock_owner4' if there
was a prohibition against presenting a lock_owner in more than one
open_to_lock_owner4?
Surely, if that were the case, then the open_seqid alone would suffice
in order to ensure once-only semantics.

Cheers,
  Trond