Subject: RE: Stateid and seqid

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 1 Feb 2001 06:32:25 -0800

Sergei Klyushin wrote: 
>     > >
>     > > According to spec., NFS Server must store and replay last response
> to any
>     > > request with seqid only, but not for all requests. Looks like
> processing
>     > > of replay is implementation depended. It must be based on seqid,
but
> spec.
>     > > says nothing more. Probably, I would compare full request also (at
> least
>     > > for the first v.4 implementation) to avoid logical bugs.
>     >
>     > There is a real problem with the way the spec describes this, which
>     > has led to two divergent interpretations.  I think we need to
resolve
>     > this.  For example the spec says (section 8.1.5):
>     >
>     >    If a request with a previous sequence number (r < L) is received,
> it
>     >    is rejected with the return of error NFS4ERR_BAD_SEQID.  Given a
>     >    properly-functioning client, the response to (r) must have been
>     >    received before the last request (L) was sent.  If a duplicate of
>     >    last request (r == L) is received, the stored response is
returned.
>     >
>     > The problem is such phrases as "request with a previous sequence
> number".
>     > Requests don't really have sequence numbers, LOCK/UNLOCK, etc.
> operations
>     > do.  I suspect that this text was written before COMPOUND became the
>     > all-encompassing way to do things and was never rewritten to reflect
>     > the fact that operations and requests are no longer one-to-one.
>     >
>     > So what do you implement?  I've implemented my seqid stuff to be
>     > operation oriented; I save the response for the operation and
>     > replay that.
> Sorry for misunderstanding. I've very ease used "request".
> My implementation is operation oriented also. As I understand idea of
seqid
> is to guarantee sequence of OPEN,LOCK,UNLOCK,CLOSE, in another words
> OPERATIONS
> that take seqid, and reply must be stored for such OPERATIONS, but not for
> the whole request
  
So that leaves Kendrick with the other interpretation, unless I've
misunderstood him too.  Anybody else have a server that does this
one way or the other?  I suppose if there is a third interpretation 
out there, we'd better hear about that too. :-(

---

From: "Kendrick M. Smith" <[email-masked]>
Date: Thu, 1 Feb 2001 14:17:14 -0500 (EST)

> out there, we'd better hear about that too. :-(

We interpreted the spec as saying that the request-level caching should be
done, but operation-level caching makes a lot more sense (and also seems
necessary for correctness in all cases), so if everyone agrees that
operation-level caching is correct, we'll switch to operation-level
caching.  (The only server implementation we haven't heard from yet is
Sun's.. correct?)

On Tue, 30 Jan 2001, Noveck, Dave wrote:

> Sergei Klyushin wrote:
> > So, you don't store stateid (or even full request) for reply message,
> > but you are based
> > only on seqid = stored_seqid? How you could be sure, that it's real
> > replay, but not logical error?
> > I want to be sure at this point, that I understand replay correctly:
> > for me it's real
> > retransmission of exactly the same package. Is it true?
> 
> Right now, I only check the seqid and operation.  I might save
> more information if I find that there are broken clients that
> are sending a seqid that matches a previous request with a
> different request.  Saving the whole operation seems excessive
> to me, particurlay given OPEN which can be fairly large.  If  
> this really turns out to be a problem (and I kind of doubt it  
> will), then maybe saving a 64-checksum of the operation seems   
> reasonable.
> 
> The issues are similar to the normal reply cache.  I don't know  
> what other systems do, but there we match on transaction id and
> then check a few other items.  We certainly don't save and check   
> the entire request.  I doubt many systems do.

We currently store the entire request because we inherited this behavior
from the Linux NFSv2/v3 server, which does store requests in its replay
cache. On the other hand, it seems to be safe to store only the RPC XID
(which is what FreeBSD does, at least), so we'll probably eventually move
toward an xid->response cache (probably not before Connectathon, though..
my mental list of "things to do before Connectathon" is reaching
frightening proportions!)

Kendrick