Subject: RE: Issue 05 was (lease expiration, stable storage (was "question s/comments on draft-ietf-nfsv4-rfc3010bis-02-04.txt")

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 20 Aug 2002 19:48:19 -0700

> After digesting the flurry of mail on this topic, here are my
> thoughts:
> 
> - I think that using the NLM/NSM protocols as a standard is setting
>   the bar a bit low.  On the other hand, I guess they provide us with
>   enough operational experience to conclude that incorrect lock
>   reclaims are not a big problem in practice.

I think the protocol has to provide enough information that servers
can do better than NLM/NSM, and I think it does.

The question is whether servers should be required to do better,
given that this has not been a pressing issue (i.e. we disconvered
it from protocol analysis of V4, rather than dissatisfaction with 
NLM).  Given all the desirable things that are in V4, my preference
would be to get people using it (and finding the issues that we haven't
thought of), rather than hold things up because this (valid) issue
is not resolved in every implementation. (or even any initial 
implementation).

> - I don't understand why the problem should be considered more severe
>   when the server implements mandatory locking.  It's not like the
>   application gets to choose which model (mandatory or advisory) the
>   server is using, or even detect reliably which model the server is
>   using.

The application may not be able to decide but those who are responsible
for running the application may very well require that it only be
run for NFS files if mandatory locking is available.

I think the basic logic is that if you are OK with the NLM advisory
locks, then you are certainly going to be OK with the much better V4
handling of advisory locking.

In the case of NLM/NFS, you have no connection between IO and locks.
No stateid's in the writes which would enable the server to detect a
stray (old) write that is now inappropriate.  So if those are the
semantics that you provide, and they are good enough for your
application, then NFS-v4 is not only going to do as well, it is 
going to do a lot better.

      Suppose you do a LOCK. 

      Then you issue a WRITE.  You get no reply so you issue a few
      more and then finally you get a reply to (one of) your writes.

      Now you do an UNLOCK.

      There is now no guarantee about the processing of those other
      write retries.

NLM/NFS-v23 may try to avoid late processing of those WRITE retries
with the reply cache, but the protocol has no guarantees that these 
will be caught, and also the server can reboot so that the reply cache
is of no use and if there are WRITEs lingering in the network, they
will be acted on.  The grace period doesn't do anything to stop them.
And V4 solves all of those issues.

Now if you need mandatory locking for you application, then clearly 
NLM/NFS-classic is not going to do it for you and so the excuse,
"Well, it is better than NLM" doesn't carry that much weight.  If
V4 is going to support the applications that have much stronger
requirements, such as mandatory locking, then V4 implementations
are going to be measured by diffrent standards, and the bar is 
a good twenty feet higher than NLM.

> - Spencer wrote
> 
>   > I don't want to mislead the client implementor into thinking that
>   > receiving a STALE_STATEID within the lease period is death to all
>   > state.  It may be that the client's state for that single file has
>   > been revoked and nothing else.
> 
>   Is this a statement that the client needs to handle STALE_STATEID
>   differently, depending on whether it thinks it still has an
>   unexpired lease?  That would be a pretty major departure from how
>   STALE_STATEID is currently dealt with in the spec.

I agree.  STALE_STATEID should be returned only if the stateid could
not have been issued by the current server boot instance.  If the
server deallocates the state structure as a result of administrative 
action, then I would assume that BAD_STATEID wuld be returned.

> - I didn't see consensus on either of the following two issues:
> 
>   - administrative commands and their impact on client recovery

Well my view is that administrative commands can free locks and
I think the spec already says as much.  I don't see how any of this
would affect client recovery.  If the server says "Your open file
is closed.  Live with it", the client lives with it and there isn't
any recovery.  What am I missing?

>  - what the spec should say about stable storage and lock recovery

I don't think there is a consensus but there is a fairly short list
of alternatives.  

    The server MUST (by storing state in stable storage or otherwise),
    ensure that it never grants a reclaim, when the possiblity exists
    that a conflicting lock was granted to another client between the
    time that the original client lock was granted and the reclaim.

    The server SHOULD ... <the statement above>.  Clients SHOULD use
    the change attribute to check for file locks being granted 
    inappropriately, when the server does not checks as above.

    The server SHOULD ... <the statement above>, but MUST do so if
    mandatory locking is supported.  Clients SHOULD ... <as above>.
  
>   I doubt that the Last Call period is long enough to make the
>   necessary changes to provide single-file recovery (to support
>   administrative commands).  I think this is an issue that should be
>   deferred.

I don't understand what sort of single-file recovery we need to provide.

>   I guess the stable storage issue is small enough (given the history
>   we have with NLM/NSM) that it can be deferred.  On the other hand,
>   given that we know about the issue and that the consequences are
>   pretty severe (file corruption), I don't see how v4 can get past
>   Proposed Standard until this is addressed.  I'm willing to let
>   others with a better understanding of IETF policies and politics
>   make the call on what to do here.

The spec cannot defer the issue, but that doesn't necessarily mean
(I don't think) that it cannot be somewhat liberal about what 
server implementations may do.

---

From: [email-masked]
Date: Wed, 21 Aug 2002 12:03:33 -0700

>>>>> "Dave" == Dave Noveck <Noveck> writes:

    Dave> Well my view is that administrative commands can free locks
    Dave> and I think the spec already says as much.  I don't see how
    Dave> any of this would affect client recovery.  If the server
    Dave> says "Your open file is closed.  Live with it", the client
    Dave> lives with it and there isn't any recovery.  What am I
    Dave> missing?

Spencer's mail talked about using STALE_STATEID for administrative
actions affecting a single file.  It wasn't clear to me that the
client could deal with this properly.  But the current spec uses a
different error, and Mike E. has proposed a new error for this case,
so I no longer have a concern here.  (And I no longer have a concern
about whether this can be resolved during the Last Call period.)

    >> - what the spec should say about stable storage and lock

    Dave> I don't think there is a consensus but there is a fairly
    Dave> short list of alternatives.

    Dave>     The server MUST (by storing state in stable storage or
    Dave>     otherwise), ensure that it never grants a reclaim, when
    Dave>     the possiblity exists that a conflicting lock was
    Dave>     granted to another client between the time that the
    Dave>     original client lock was granted and the reclaim.

    Dave>     The server SHOULD ... <the statement above>.  Clients
    Dave>     SHOULD use the change attribute to check for file locks
    Dave>     being granted inappropriately, when the server does not
    Dave>     checks as above.

    Dave>     The server SHOULD ... <the statement above>, but MUST do
    Dave>     so if mandatory locking is supported.  Clients SHOULD
    Dave>     ... <as above>.

Okay, those sound like reasonable choices.  I would change the wording
of the last two slightly, in that the client doesn't know what
checking the server will do.

    Clients SHOULD use the change attribute to check for file locks
    being granted inappropriately, in case the server does not check
    as above.

I think the first choice is best, followed by three and then two.  But
I could live with any of them.

mike