Subject: [nfsv4] NFS4.0 question regarding the use of an existing lock-owner with a stateid for a previously closed file

From: Saul Tamari <[email-masked]>
Date: Sun, 16 Dec 2012 00:39:59 -0500

Hello,

I have a question regarding the use of one lock-owner for locking byte
ranges on two different files.

The sequence I'm not sure about is:
1. OPEN file-A, open_owner -> open_stateid-A1, fh-A
2. LOCK some byte range on file-A given open_stateid-A1,
new_lock_owner=true, lock_owner -> lock_stateid-A1
3. LOCKU with lock_stateid-A1 -> lock_stateid-A2
4. CLOSE fh-A, open_stateid-A1
5. OPEN file-B, open_owner -> open_stateid-B1, fh-B

How is the client supposed to lock a range on file-B given the same
lock_owner as in step 2 ?
A. LOCK file-B given lock_stateid-A2 ? Is lock_stateid-A2 still valid at
this point so it can be used to locate lock_owner ? Wasn't lock_stateid-A2
freed in step 4 ?
B. LOCK file-B using new_lock_owner=true and establish a new lock_owner? so
the client might initiate a new lock_owner for every file that it opens (if
it closed previous files) ?

Thank you,
Saul

---

From: Marcel Telka <[email-masked]>
Date: Sun, 16 Dec 2012 11:56:20 +0100

> lock_owner as in step 2 ?

lock_owner is tied to the open_stateid, so to lock file-B you need to have set
new_lock_owner=true in the LOCK request.

> this point so it can be used to locate lock_owner ? Wasn't lock_stateid-A2

You cannot use lock_stateid-A2 to lock file-B.

-- 
+-------------------------------------------+
| Marcel Telka   e-mail:   [email-masked]  |
|                homepage: http://telka.sk/ |
|                jabber:   [email-masked] |
+-------------------------------------------+

---

From: Rick Macklem <[email-masked]>
Date: Sun, 16 Dec 2012 09:33:30 -0500 (EST)

> new_lock_owner=true in the LOCK request.
I think you might also want to put a ReleaseLockOwner op after 4, because
RFC-3530 wasn't clear on how long a lock_owner persists. I believe that,
when a server sees a lock_owner string in a new_lock_owner==true case that
it already has a record for, that it should initialize the seqid to whatever
is provided as an argument, but I am not sure all servers do this?

Btw, the case without step 4 is somewhat problematic too. It is not obvious
that you can use lock_stateid-A2 and, even if you can, you can only use it
until step 4 occurs.

I found that the easiest way to avoid this problem was to put the
file handle in the lock_owner string. That way the lock_owner string
is never the same for two different files and it avoids the problem.
When I combined that with doing a ReleaseLockOwner when the client is
done with a given lock_owner string, I had code that seemed to work ok
against all NFSv4.0 servers that I have tested against.

I don't think the RFC required servers to handle multiple instances of
the same lock_owner string (created via new_lock_owner==true for different
open stateids), although some servers allows it. As such, I did the above
to avoid that case. I also don't believe the RFC clearly defines when a
server will discard a lock_owner if a ReleaseLockOwner isn't received,
so doing them clarifies that.

Good luck with it, rick

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "Myklebust, Trond" <[email-masked]>
Date: Sun, 16 Dec 2012 18:03:07 +0000

If File B != File A, then you need to do a open_to_stateid regardless... Lock stateids are tied to the (lockowner, open_owner, filehandle) triple, so you cannot take a byte-range lock stateid from File A and apply it to File B.

The only question here is which value to use for the lock_seqid argument in the open_to_stateid. If the lock_owner has been established already through a previous open_to_stateid to File A, and that file is still open, then you should try using the next value for lock_seqid in the sequence for that lock_owner. The reason is that the NFSv4 server is obliged to keep the lock_owner around for as long as an "associated file" is still open (see Section 15.39.5. in RFC3530bis draft 21) or until the client calls RELEASE_LOCKOWNER. The notion of an "associated file" isn't really described anywhere, but the assumption is that it means an open file stateid for which a lock stateid has been established through open_to_stateid.

Note that the rules governing lock owner lifetimes are different in NFSv4.1, but there you don't have a lock_seqid to worry about anyway...

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Rick Macklem
> Sent: Sunday, December 16, 2012 9:34 AM
> To: Marcel Telka
> Cc: nfsv4
> Subject: Re: [nfsv4] NFS4.0 question regarding the use of an existing lock-
> owner with a stateid for a previously closed file
> 
> Marcel Telka wrote:
> > > I have a question regarding the use of one lock-owner for locking
> > > byte ranges on two different files.
> > >
> > > 1. OPEN file-A, open_owner -> open_stateid-A1, fh-A 2. LOCK some
> > > byte range on file-A given open_stateid-A1, new_lock_owner=true,
> > > lock_owner -> lock_stateid-A1 3. LOCKU with lock_stateid-A1 ->
> > > lock_stateid-A2 4. CLOSE fh-A, open_stateid-A1 5. OPEN file-B,
> > > open_owner -> open_stateid-B1, fh-B
> > >
> > lock_owner is tied to the open_stateid, so to lock file-B you need to
> > have set new_lock_owner=true in the LOCK request.
> >
> when a server sees a lock_owner string in a new_lock_owner==true case
> that it already has a record for, that it should initialize the seqid to whatever
> I found that the easiest way to avoid this problem was to put the file handle
> in the lock_owner string. That way the lock_owner string is never the same
> for two different files and it avoids the problem.
> I don't think the RFC required servers to handle multiple instances of the
> same lock_owner string (created via new_lock_owner==true for different
> open stateids), although some servers allows it. As such, I did the above to
> avoid that case. I also don't believe the RFC clearly defines when a server will
> discard a lock_owner if a ReleaseLockOwner isn't received, so doing them
> clarifies that.
> 
> Good luck with it, rick
> 
> > > A. LOCK file-B given lock_stateid-A2 ? Is lock_stateid-A2 still
> > > valid at this point so it can be used to locate lock_owner ? Wasn't
> > > lock_stateid-A2
> > > freed in step 4 ?
> >
> > > B. LOCK file-B using new_lock_owner=true and establish a new
> > > lock_owner? so the client might initiate a new lock_owner for every
> > > file that it opens (if it closed previous files) ?
> >
> >
> > --

---

From: Saul Tamari <[email-masked]>
Date: Sun, 16 Dec 2012 14:44:28 -0500

Hi,

On Sun, Dec 16, 2012 at 1:03 PM, Myklebust, Trond <
[email-masked]> wrote:

> If File B != File A, then you need to do a open_to_stateid regardless...
> Lock stateids are tied to the (lockowner, open_owner, filehandle) triple,
> so you cannot take a byte-range lock stateid from File A and apply it to
> File B.
>

So for each file that needs to be locked by a <client-id, open-owner,
lock-owner> there has to be at least one LOCK request with the
new_lock_owner=true flag? new_lock_owner means a new-lock-owner for this
specific file.

> The only question here is which value to use for the lock_seqid argument
> in the open_to_stateid. If the lock_owner has been established already
> through a previous open_to_stateid to File A, and that file is still open,
> then you should try using the next value for lock_seqid in the sequence for
> that lock_owner.

In this last sentence when you say "you should try using the next value for
lock_seqid" you mean that the client (not the server) should pass the next
lock_seqid along with the new_lock_owner request?

How is the server expected to behave if the lock_seqid for the
new_lock_owner=true request on File B doesn't match the previously
established lock_seqid on File A (in a case that File A is still open)?

> The reason is that the NFSv4 server is obliged to keep the lock_owner
> around for as long as an "associated file" is still open (see Section
> 15.39.5. in RFC3530bis draft 21) or until the client calls
> RELEASE_LOCKOWNER. The notion of an "associated file" isn't really
> described anywhere, but the assumption is that it means an open file
> stateid for which a lock stateid has been established through
> open_to_stateid.
>
>
> Note that the rules governing lock owner lifetimes are different in
> NFSv4.1, but there you don't have a lock_seqid to worry about anyway...
>

yep. I will need to start work on 4.1 soon.

[snip]

Thanks,
Saul

---

From: "Myklebust, Trond" <[email-masked]>
Date: Sun, 16 Dec 2012 21:00:05 +0000

On Sun, 2012-12-16 at 14:44 -0500, Saul Tamari wrote:
> Hi,
> 
> On Sun, Dec 16, 2012 at 1:03 PM, Myklebust, Trond
> <[email-masked]> wrote:
>         If File B != File A, then you need to do a open_to_stateid
>         regardless... Lock stateids are tied to the (lockowner,
>         open_owner, filehandle) triple, so you cannot take a
>         byte-range lock stateid from File A and apply it to File B.
> 
> new_lock_owner=true flag? new_lock_owner means a new-lock-owner for
> this specific file.

For each _filehandle_, actually. stateids are always per-filehandle,
even in the case where the server associates multiple filehandles per
file. See section 9.1.3 and 9.11, for instance in RFC3530bis draft 21.

>         The only question here is which value to use for the
>         lock_seqid argument in the open_to_stateid. If the lock_owner
>         has been

I meant open_to_lock_owner

>          established already through a previous open_to_stateid to
>         File A, and that file is still open, then you should try using
>         the next value for lock_seqid in the sequence for that
>         lock_owner. 
> 
> In this last sentence when you say "you should try using the next
> value for lock_seqid" you mean that the client (not the server) should
> pass the next lock_seqid along with the new_lock_owner request?

Yes. Whatever seqid you used in the last LOCK/LOCKU call to that
lock_owner, bump it by 1.
Note that this means you MUST serialise LOCK/LOCKU calls for the same
lock_owner. In addition, when using open_to_lock_owner, you need to
serialise with OPEN/OPEN_DOWNGRADE/CLOSE/... for the open_owner.

Since the serialisation is a major PITA for most purposes, a number of
clients do not bother reusing lock_owners across files, but rather use a
new lock_owner for each file. The Linux clients all do this, for
instance.

> established lock_seqid on File A (in a case that File A is still

It should return NFS4ERR_BAD_SEQID.

>         The reason is that the NFSv4 server is obliged to keep the
>         lock_owner around for as long as an "associated file" is still
>         open (see Section 15.39.5. in RFC3530bis draft 21) or until
>         the client calls RELEASE_LOCKOWNER. The notion of an
>         "associated file" isn't really described anywhere, but the
>         assumption is that it means an open file stateid for which a
>         lock stateid has been established through open_to_stateid.
>         
>         
>         Note that the rules governing lock owner lifetimes are
>         different in NFSv4.1, but there you don't have a lock_seqid to
>         worry about anyway...
> 

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: Saul Tamari <[email-masked]>
Date: Mon, 17 Dec 2012 07:15:58 -0500

Thanks for the clarifications.

On Sun, Dec 16, 2012 at 9:33 AM, Rick Macklem <[email-masked]> wrote:

> > > valid at
> > > lock_owner? so