Subject: RE: [nfsv4] Follow up on close() during Grace.

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 30 Nov 2004 16:10:26 -0500

No conclusion.  I presented some stuff on this at the IETF meeting
before the last one.  I think the Spencer made the slides 
available on the web somewhere.  If they aren't I can probably 
dig those up.  I think the result of the discussion was
that we decided that this was certainly small enough that it 
would fit as in as one of the "miscellaneous minor items" for
v4.1 if we decided this was needed but there was no consensus on 
that latter point.

As I recall, Trond said something approximately like.  "You don't
need that (i.e. RECOVERY_DONE) because, if you ... then ..." but 
the discussion was rushed as my time was coming to a close and I 
didn't catch the specifics and I've been meaning to follow up with 
him ever since.

So your message seems like the appropriate cue to do that.

So Trond, does this ring a bell?

As far as Mike's comments about better (i.e. finer-grained) error
codes, I can't recall anybody disagreeing about that, so maybe there
is a consensus that this belongs as one of the minor items to be
included in v4.1, whenever we decide to go ahead with writing a 
v4.1 spec.

-----Original Message-----
From: Robert Gordon [mailto:[email-masked]] 
Sent: Monday, November 29, 2004 3:24 PM
To: NFSv4
Subject: [nfsv4] Follow up on close() during Grace.

I've been catching up on the WG-Mail and noticed that the
following thread seemed to "wither" from a WG-Mail perspective.

Was there a conclusion off-list ?

> On Jul 25, 2004, at 9:49 AM, Noveck, Dave wrote:
>
>> Brian, Spencer.  Does this mean we can't mention DONE_RECOVERY in San

>> Diego?
>> I have this vision of us sitting in the meeting with laptops and 
>> sending
>> email to the working group back and forth but the words DONE_RECOVERY

>> never
>> passing our lips :-)
>
> I will assign 10-15 minutes for you to present the issues and make 
> sugguestions
> of how to address it and where it best fits in our current or future
> minor versioning proposals.
>
> Spencer

and this was Mike's Last comment on the subject:

> 	From: 	  [email-masked]
> 	Subject: 	RE: [nfsv4] more re: close during grace
> 	Date: 	July 26, 2004 1:26:46 PM PDT
> 	To: 	  [email-masked]
>
> > Note also that when we have such a client [that fails to issue
> > DONE_RECOVERY], the fact is marked in
> > stable storage so that if we (the server) reboot, when that
> > client tries to reclaim he will get NO_GRACE even if this time
> > he is within the grace period.
>
> I think that rather than NO_GRACE, there should be
> a new error code, for this situation in the interest of
> clarity.
>
> In fact at least three new error codes would be clearer:
>
> 1. For the case when a client is reclaiming stuff during
>    a server instance, and then sends a steady state
>    operation to increase (or if the server insists,
>    reduce) state.
>
> 	NFS4ERR_FINISH_YOUR_RECOVERY
>
>    The client must respond with DONE_RECOVERY to get out of
>    this situation.
>
> 2. For the case when the client is reclaiming stuff,
>    but the server knows that it rebooted at least once
>    with the client failing to issue DONE_RECOVERY (edge
>    condition 2 in section 8.6.3, or the modified
>    edge condition Dave came up with that I described
>    last week); and the server cannot determine if a
>    conflicting lock was granted.
>
>        NFS4ERR_MISSED_RECOVERY
>
>    At minimum, the cliebt must respond with DONE_RECOVERY
>    to get out of this situation. In addition, he is
>    advised to acquires the lock state he needs without
>    the reclaim flags to the affected operations, then
>    do a GETATTR "change" operatioa, and then compare
>    the attribute with what was cached. This is so the client
>    can determine if another client changed things
>    from underneath him.
>
> 3. As 2, but the server knows a conflicting lock
>    was granted.
>
>       NFS4ERR_CONFLICTED_STATE
> >
> > -----Original Message-----
> > From: Nicolas Williams [mailto:[email-masked]]
> > Sent: Monday, July 26, 2004 2:38 PM
> > To: Spencer Shepler
> > Cc: Noveck, Dave; [email-masked]; [email-masked]
> > Subject: Re: [nfsv4] more re: close during grace
> >
> >
> > One issue with DONE_RECOVERING: what if it never comes, or takes too
> > long to come?  At some point one of the clients must lose...  That 
> may
> > require administrative action on the client to recover, but
> > it's better
> > than DoSing the server.
>
> I'm assuming most clients will return errors to applications,
> and/or send SIGLOST.
>

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Tue, 30 Nov 2004 19:00:41 -0500

> So Trond, does this ring a bell?

This would be related to the

        Trond suggests using setclientid to time shift grace period
        forward

comment in the San Diego IETF minutes, right? That was several beers ago
8-)

I think I vaguely remember...

The issue was that section 8.6.2 the RFC states that the grace period
should be "at least as long as the lease period for the previous server
instantiation", but offers no method for a client to otherwise determine
the duration. So clients will want to assume that they have less than 1
lease period remaining in which to re-establish all state (assuming the
server supports a grace period).

The question then as far as the server is concerned is "when did the
client start its stopwatch"? It does not know when the client received
NFS4ERR_STALE(CLIENTID|STATEID), but it does know that the client will
immediately try to reply with a SETCLIENTID call.

Hence the server may assume that all clients should have finished
recovering all locks when the last client to send a SETCLIENTID has
waited 1 lease period (Note: any remaining clients that have waited more
than 1 lease period to send the setclientid will in any case expect to
have lost all their state).

Cheers,
  Trond

-- 
Trond Myklebust <[email-masked]>