Subject: RE: [nfsv4] Out of sync stateid scenario that looks difficult to resolve

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 15 Sep 2003 14:04:50 -0700

>    >> Now I am curious, though. Where in the RFC does it say
 >    >> explicitly that OPEN will invalidate an existing stateid, and
 >    >> why should it be necessary?
 >
 >    > The RFC doesn't require that a stateid change values on
 >    > operations like OPEN (upgrade) or OPEN_DOWNGRADE but it makes
 >    > the server implementation much easier with respect to tracking
 >    > a single valid stateid for a file.  The amount of state that
 >    > needs to be kept is a little less, the server can easily
 >    > determine if pending reads/writes fall under the appropriate
 >    > stateid, and when debugging client and server interaction it is
 >    > easier to track the transitions that occur with stateids.

> As I said earlier, the only case where this is a problem is OPEN,
> since there the client does not a priori know what the filehandle is,
> and so cannot predict which stateid is liable to be invalidated.

> Would be consistent with RFC3530 to recommend as a "best practice"
> that servers should send an OPEN_CONFIRM request in the particular
> case where the result of the OPEN would be to invalidate an existing
> stateid?

> Since the original OPEN reply returns the filehandle, this would allow
> the client to put its house in order before accepting the upgraded
> stateid for the given file.

So I suppose that you are assuming that the server will shomehow keep 
the information for the previous stateid as well as the new one so that
it can act on the old one before the CONFIRM, act on the new one after
the CONFIRM, and also restore the old one if the confirm does not happen.
Since my server doesn't currently do this, I'm definitely against it :-)

Note that Spencer's (I think it was Spencer's) original point was that the
purpose of the invalidation was

> it makes the server implementation much easier with respect to tracking
> a single valid stateid for a file.  

Well we have one that is valid but in this proposal we have to keep two.

> The amount of state that needs to be kept is a little less, 

I'm not sure it is less, and if you do the OPEN_CONFIRM thing, then it
seems like it is more.

> the server can easily determine if pending reads/writes fall under the 
> appropriate stateid, 

Let's take a case.  I have a file open for read and it was state id X 
with seqid 1.  Now, asynchronously some read's are issued with that 
stateid X:1.  Now an open for write is done upgrading the open for that
file (and it may be through a different name) and so you get back, by
hypothesis, stateid X:2.

So if we get a read issued before the open, should we reject it?  After
the upgrade, we will still be able to read, as we can see by looking for
the current privileges recorded in the object for stateid X (in this model
we won't look at the seqid, since the state object has been updated to
include read and write privileges).  Now if somehow the client issued a
write, before the upgrade, then we could see that fact with the seqid,
but is that really an advantage?  Why not just look at the write and
and the current options and see if it is what we are currently allowed
to do?  Finding that a client is issuing writes when he doesn't have the
privilege of doing them in a situation in which they are valid when
they are received on the server isn't high on my list of priorities.  

Am I missing something here?  Is there some case where this sort of state 
invalidation really gives you a substantial benefit as far as rights
checking? 
   
> and when debugging client and server interaction it is easier to track 
> the transitions that occur with stateids.

This is true but is it worth it?  Our server does currently do this, but
if does add complexity and so if it problematic for clients, and there isn't
a big benefit, I'd be inclined to get rid of it.  

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Mon, 15 Sep 2003 17:40:32 -0400

>>>>> " " == Dave Noveck <Noveck> writes:

     > Let's take a case.  I have a file open for read and it was
     > state id X with seqid 1.  Now, asynchronously some read's are
     > issued with that stateid X:1.  Now an open for write is done
     > upgrading the open for that file (and it may be through a
     > different name) and so you get back, by hypothesis, stateid
     > X:2.

     > So if we get a read issued before the open, should we reject
     > it?  After the upgrade, we will still be able to read, as we
     > can see by looking for the current privileges recorded in the
     > object for stateid X (in this model we won't look at the seqid,
     > since the state object has been updated to include read and
     > write privileges).  Now if somehow the client issued a write,
     > before the upgrade, then we could see that fact with the seqid,
     > but is that really an advantage?  Why not just look at the
     > write and and the current options and see if it is what we are
     > currently allowed to do?  Finding that a client is issuing
     > writes when he doesn't have the privilege of doing them in a
     > situation in which they are valid when they are received on the
     > server isn't high on my list of priorities.

     > Am I missing something here?  Is there some case where this
     > sort of state invalidation really gives you a substantial
     > benefit as far as rights checking?
   
If all servers are prepared to do what you propose above, then there
is no problem, and we can drop this thread entirely.

My understanding of last week's discussion, though, was that in the
above scenario, the server may also reject those reads with stateid
X:1 if they arrive after the OPEN that upgraded the server state for
that given <open_owner/filehandle> to read/write.

To summarize:
------------
All I'm wanting is to avoid having to serialize all READ/WRITE
requests with every new OPEN request that shares the same open_owner
on the off chance that the server will change the stateid, and refuse
to accept the old one. If the server is prepared to accept my
in-flight reads with X:1==(READ,DENY_NONE) for a short period after it
has created state X:2==(READ/WRITE,DENY_NONE), then I'm a happy
camper...
If OTOH it starts returning NFS4ERR_OLD_STATEID, so I have to resend
all my reads with the new stateid, then my network admin may take
offence...

Cheers,
  Trond

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 15 Sep 2003 20:26:19 -0500

> I'm not sure it is less, and if you do the OPEN_CONFIRM thing, then it

Without any data to back this up, I believe that adding an
OPEN_CONFIRM round trip will be unnecessary in most all cases (in that
the client will not have in-flight READs/WRITEs that will be
effected).  The choice of adding a round-trip, like OPEN_CONFIRM, is
too heavy vs. having the client recover/resend the READ/WRITE when
NFS4ERR_OLDSTATEID is received.  The client already needs a recovery
framework for things like NFS4ERR_STALE_STATEID; adding the bit of
code for _OLDSTATEID recovery in this scenario is relatively small.

It will be good to have implementations in place to obtain real data
behind this discussion.

> a big benefit, I'd be inclined to get rid of it.  

The same type of checking (on the server) and recovery (on the client)
needs to be in place for dealing with the transitions of
lock_stateids.  Our server is currently implemented to share the
stateid checking so it is simpler (not a reason to choose a particular
approach but it is simpler).

Spencer