Subject: [nfsv4] Returning NFS4ERR_CB_PATH_DOWN

From: "Khan, Saadia" <[email-masked]>
Date: Tue, 3 May 2005 11:28:42 -0700

This is what the spec has to say about NFS4ERR_CB_PATH_DOWN:

   When the client holds delegations, it needs to use RENEW to detect
   when the server has determined that the callback path is down.  When
   the server has made such a determination, only the RENEW operation
   will renew the lease on delegations.  If the server determines the
   callback path is down, it returns NFS4ERR_CB_PATH_DOWN.  Even though
   it returns NFS4ERR_CB_PATH_DOWN, the server MUST renew the lease on
   the record locks and share reservations that the client has
   established on the server.  If for some reason the lock and share
   reservation lease cannot be renewed, then the server MUST return an
   error other than NFS4ERR_CB_PATH_DOWN, even if the callback path is
   also down.

and the more confusing part is:

This difficulty is solved by the following rules:

   o  When the callback path is down, the server MUST NOT revoke the
      delegation if one of the following occurs:

      -  The client has issued a RENEW operation and the server has
         returned an NFS4ERR_CB_PATH_DOWN error.  The server MUST renew
         the lease for any record locks and share reservations the
         client has that the server has known about (as opposed to those
         locks and share reservations the client has established but not
         yet sent to the server, due to the delegation).  The server
         SHOULD give the client a reasonable time to return its
         delegations to the server before revoking the client's
         delegations.

      -  The client has not issued a RENEW operation for some period of
         time after the server attempted to recall the delegation.  This
         period of time MUST NOT be less than the value of the
         lease_time attribute.

   o  When the client holds a delegation, it can not rely on operations,
      except for RENEW, that take a stateid, to renew delegation leases
      across callback path failures.  The client that wants to keep
      delegations in force across callback path failures must use RENEW
      to do so.

so it seems like just because the callback path is down, the server
should not
revoke delegations, so in case the server finds out that the callback
path is down because it is in the middle of recalling the delegation, it
probably needs to wait for 
a renew from that client so that it can send it NFS4ERR_CB_PATH_DOWN and
then 
wait for atleast one lease period before revoking the delegation. 

Does this seem like the right approach? Are there other server
implementations which have implemented this approach and do current
clients correctly handle this error case?

Thanks.
Saadia

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "J. Bruce Fields" <[email-masked]>
Date: Tue, 3 May 2005 15:12:44 -0400

On Tue, May 03, 2005 at 11:28:42AM -0700, Khan, Saadia wrote:
> should not revoke delegations, so in case the server finds out that
> the callback path is down because it is in the middle of recalling the
> delegation, it probably needs to wait for a renew from that client so
> that it can send it NFS4ERR_CB_PATH_DOWN and then wait for atleast one
> lease period before revoking the delegation. 

That's how I read it.  With one exception: the "at least one lease
period" requirement is only for the length of time the server has to
wait for the client to send the renew that the server replies
CB_PATH_DOWN to:

>	The server SHOULD give the client a reasonable time to return
>	its delegations to the server before revoking the client's
>	delegations.

Once it's gotten the chance to return CB_PATH_DOWN, the time to wait is

> Are there other server implementations which have implemented this
> approach

Linux returns CB_PATH_DOWN, but in some situations I think it may not
wait the correct amount of time before returning the delegation--but
that's a bug I need to fix.

> and do current clients correctly handle this error case?

The Linux client has code to handle this case, which looks like it does
exactly the right thing.  I don't know how much it's actually been
tested.

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 3 May 2005 17:58:08 -0500

On Tue, Khan, Saadia wrote:

<...>

> clients correctly handle this error case?

Yes, this is appropriate.

The Solaris client, upon receipt of NFS4ERR_CB_PATH_DOWN, will return
all of its delegations.

The Solaris server will do the CB_PATH_DOWN notification but there may
be one bug lurking in there someplace.

I will take the opportunity to close a thread of discussion we had
awhile back that is similar to this.

It was the race condition that the client and server have when the
client is provided a delegation for OPEN and the OPEN response is
on its way back to the client and the server needs to recall the
delegation and the deleg recall may make it to the client first.
The client will likely return an error to the CB_RECALL and we
discussed the various ways to fix this and I believe came to consensus
on the following.

If the server receives an error, like NFS4ERR_BADHANDLE, in response
to CB_RECALL, it should not immediately revoke the delegation but
retry the CB_RECALL within a lease period.  Once a lease period has
passed from the initial CB_RECALL, the server will then revoke the
delegation.

The Solaris server doesn't do this currently but I will be making
a change soon to correct this.

Spencer

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 3 May 2005 17:59:34 -0500

> just described as "a reasonable period":

I agree.  The server can extend the period if it observes the client
making a reasonable effort to return delegations (just as it would if
a CB_RECALL has been serviced and there are WRITEs being done for the
file).

Spencer