Subject: RE: file_id

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 4 Dec 2002 07:58:07 -0800

I would think the ones from Microsoft don't let you rename a file
while it is open.  There may be holes due to reclaim when the server
goes down.  For example, the server reboots, someone renames file 
during the grace period, you try to reclaim your open and you lose.

I just checked the spec and RENAME and REMOVE do not list GRACE 
among the error returns.  It seems to me that servers whose fs
semantics prohibit rename and remove while a file is open should
have the option of returning GRACE when these operations are done
during the grace period.  Maybe this is something for the issues 
list.

-----Original Message-----
From: Jim Rees [mailto:[email-masked]]
Sent: Tuesday, December 03, 2002 12:49 PM
To: [email-masked]
Subject: Re: file_id 

When I pushed for adding a unique file identifier to the protocol, I was
told there were some servers that would be unable to provide this, because
all they have is the file name.  These servers hash the name to get the file
handle.  If the file is renamed, the handle becomes stale, and the client
must re-open.

I don't know which servers these would be, but I would guess they come from
Microsoft.

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 4 Dec 2002 14:06:52 -0600

> during the grace period.  Maybe this is something for the issues 

Possibly but in the interim the server has the option of returning
NFS4ERR_DELAY.

-- 
Spencer

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 04 Dec 2002 16:40:42 -0500

>Possibly but in the interim the server has the option of returning

Huh? More like, NFS4ERR_DEADLOCK :-) The client could have the
file open itself. DELAY makes no sense in this case because the
condition to unblock it is unbounded in time.

Tom.

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 4 Dec 2002 15:51:29 -0600

> condition to unblock it is unbounded in time.

Dave's specific comment was for the case that the server was within
the GRACE period and wanted to let the requesting client know that it
was not yet complete with grace and that the requested RENAME/REMOVE
was being postponed because there was a chance that a reclaim OPEN
would be received that would actually change the retransmitted
RENAME/REMOVE request to NFS4ERR_FILE_OPEN.

There is no deadlock in this particular case and hence the reason why
I suggested the use of NFS4ERR_DELAY to tell the client to delay and
retransmit its request.

Spencer

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 04 Dec 2002 17:06:50 -0500

Okay, call it livelock. The client and server will enter an
infinite loop of rename-delay-rename-delay. In fact,
returning GRACE is only slightly better, in that it hints
at a termination condition outside of the client's control.

I see NFS4ERR_FILE_OPEN in the rename error cases,
isn't this the right one?

Tom.

---

From: Mike Eisler <[email-masked]>
Date: Wed, 04 Dec 2002 15:11:27 -0800

Let me see if I can decipher what Spencer and Dave are saying.

There exist a set of servers that have a operating
system semantic which don't allow rename/remove of
open files. Let's call this set W.

There is a subset of  W such that the servers'  records of
open files are not preserved in stable storage through
server reboots. Therefore, this subset, call it X, must rely
on the client to reclaim open state during the grace period.

Therefore servers in subset X, in order to preserve the
aforementioned semantic, must not remove or rename files
during the grace period.

Returning NFS4ERR_FILE_OPEN doesn't work because
the file might not be open. Servers in subset X won't know
either way until the grace period is concluded.

I don't see the livelock or deadlock. Progress resumes
when the finite grace period ends.

If a server is in set W, it would be best to among
the W - X subset, though that has higher implementation
complexity.

    -mre

>>>>>while it is open.  There may be holes due to reclaim when the
>>>>>
>>server
>>

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 04 Dec 2002 21:10:26 -0500

I disagree with the statement "the file might not be open".
The file is most definitely open, or the grace period wouldn't
be in effect. I think what you're arguing is that the server is
somehow aware that the client is not connected, which seems
a completely wrong thing on which to base the server's reply,
to say nothing of impossible over non-connection-oriented
transports.

Tom.

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 4 Dec 2002 20:19:23 -0600

Tom,
  Not sure if you have the right picture about how server recover and
grace period works for NFSv4.  If the server restarts it has a grace
period that is of duration equal to the lease period that the server
provided to its clients in its previous instantiation.  The clients
will be renewing their leases either explicitly with RENEW or by
issuing other state based operations.  The client may not immediately
(upon server availability) be notified of the server's failure and
recovery.  Therefore, the client may not have yet recovered/reopened
files that were previously opened against the previous server
instantiation.  It is also possible that the client has a lot of state
to recover and that the latency between first and last recovered open
file may be of some significant time.
  Therefore, it is quite possible that the file open at the client has
yet to be reclaimed at the server because of this delay.  Once the
grace period has ended, the server will no longer accept reclaim-OPENs
and can then respond to the RENAME/REMOVE appropriately (either with
NFS4_OK or NFS4ERR_FILE_OPEN - for the case we are discussing).

Spencer

-- 
Spencer

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 4 Dec 2002 20:40:52 -0600

On Wed, Talpey, Thomas wrote:
> I believe I have the same picture of grace periods as you. But
> I view the grace period as favoring the client, not the server.
> So if the server restarts and finds a lease in effect, the grace
> period is there to hold that guarantee to the client. The client
> may take a while to come back and reclaim, but by no means
> is this a reason for the server to start telling other clients to
> sharpen their knives.
> 
> Put it this way. Choosing NFS4ERR_FILE_OPEN versus
> NFS4ERR_GRACE versus NFS4ERR_DELAY in no way
> changes the effect on the file in question. And the latter two
> introduce a new client behavior if they encourage a retransmit
> cycle, of possibly very long duration. Why do this?

The server has to respond with something.  NFS4ERR_FILE_OPEN is
inappropriate because that says:

   NFS4ERR_FILE_OPEN     The operation can not be successfully processed
                         because a file involved in the operation is
                         currently open.

and if the client doing the RENAME/REMOVE were to receive this it
would return an error to the caller/application.  The other reason the
server can not return this error is that it doesn't know (in the case
that Mike stated) if the file will be OPENed/reclaim during the grace
period.  

So that leaves NFS4ERR_GRACE or NFS4ERR_DELAY.  The client is free to
set its own timeouts for retransmitting its request for those
operations.  I don't see a problem with that.  The client request will
be resolved at the end of the server's grace period.

Spencer

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 04 Dec 2002 21:37:20 -0500

Tom.

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 04 Dec 2002 21:54:38 -0500

Okay, so we do disagree on what the grace period means.

I assert the file is open during the grace period. Server failure
doesn't affect the client's open state. Only server failure *plus*
grace period changes that, and grace periods can be long.

Tom.

---

From: Mike Eisler <[email-masked]>
Date: Thu, 05 Dec 2002 07:08:38 -0800

The grace period means the same thing as it meant in the NLM
days. Would you similarly assert that during the NLM grace period,
all files had locks on the entire byte range, and therefore,
a non-blocking NLM lock request should returned the error
that indicates the file is locked?

The server in set X has no record of which files were open at the
time it crashed or shutdown. The grace period is to give clients
that had open state (and lock and share state) time to
re-establish that state. Returning _FILE_OPEN is wrong, because
clients won't retry the OPEN as a result. Whereas, returning _DELAY or
_GRACE is workable because that tells clients to retry the OPEN at a
later point. As the spec is written today, only _DELAY is correct.

I don't fathom how you can assert that the file is open during the grace
period. All you can assert is that the file is possibly open. During the
grace period it will be decided if the file is in fact open.

Again I don't see the deadlock/livelock, and I don't see the problem.

    -mre