Subject: RE: questions/comments on draft-ietf-nfsv4-rfc3010bis-02-04.txt

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 21 Aug 2002 19:23:16 -0700

Mike Eisler: 
> "Noveck, Dave" wrote:
> > 
> > 
> > 
> > > Regarding,
> > >
> > >    The server must be able to distinguish the following situations and
> > >    return the error as specified:
> > >
> > >    o    The stateid was generated by an earlier server instance (i.e.
> > >         before a server reboot).  The error NFS4ERR_STALE_STATEID should
> > >         be returned.
> > >
> > >    o    The stateid was generated by the current server instance but the
> > >         stateid no longer designates the current locking state for the
> > >         lockowner-file pair in question (i.e. one or more locking
> > >         operations has occurred).  The error NFS4ERR_OLD_STATEID should
> > >         be returned.
> > >
> > >         This error condition will only occur when the client issues a
> > >         locking request which changes a stateid while an I/O request
> > >         that uses that stateid is outstanding.
> > >
> > > So which op, the I/O or the locking op gets the error?
> > 
> > The IO, if it was issued with the then-current stateid and is processed by
> > the server after the locking op has changed the stateid.
> > 
> > > And why should any error be returned at all?
> > 
> > The error is returned because the stateid passed is incorrect.
> > 
> > An example of where this is valuable is the case in which you have a lock,
> > and are doing a WRITE, which is retransmitted, one or more times.  Once
> > you get back the reply to the WRITE (one of them anyway), you issue a
> > LOCKU, while there might be other instances of the WRITE in the network.
> > If it is a bad thing to process those laggard writes following the LOCKU
> > (and maybe a LOCK from someone else) that you'd like to protect against,
> > and I think it is, then returning OLD_STATEID in this case provides exactly
> > that protection.

> Interesting. This seems to take us down the slippery slope of
> fully sequenced RPC that goes beyond just locking operations.
> On the other hand, even with NFSv[23], clients are doing synchronous
> IO when advisory locking is on, so there is already an expectation
> of reduced IO throughput when locking is enabled.

These IO's need not be synchronous or sequenced among themselves.
It is only with respect to the locking operations (which act as
barriers) that they have to be sequenced.

This slope does not seem very slippery since this is the way the 
V4 locking model has been from the beginning (or at least since
I first saw it over three years ago).  It was always the case that
if you issued an IO with a stateid from before a LOCK and it 
executed after the LOCK, it would be rejected, although the
error definitions have changed.

> Still, in the past, we've relied on the duplicate request
> cache to catch dangling writes, and operational experience
> for the past 15 years or so says it works fine.
> I think returning _OLD_STATEID on
> an I/O ought to be a server implemention choice.

One case in which we have no relevant operational experience
is when mandatory locking is involved.  

I don't see what the value of making this optional is.  Aren't 
you essentially saying that once these requests are filtered
through the duplicate request cache (and taking into account
the way people will do IO when locking is in effect), the
OLD_STATEID error will never be returned?  So the only 
downside is the cost of comapring the sewquence in the
stateid, and that is trivial.  

In the case of LOCK (as opposed to LOCKU) you can argue that 
the barrier doesn't provide any value, and might conceivably
constain client IO patterns.  How about allowing the server
to accept an earlier sequenced state-id if it can determine
(by saving information about earlier state instances), that
no lock conflicting with the IO could possibly have been 
granted between the time the state presented was generated and
the time the IO was executed.  (Example implementation: you
save in the state structure a count of the number of LOCK
operations without a LOCKU, and except anything between the
current state sequence value and that value minus the count
of successive lock operations).