Subject: Re: [nfsv4] Out of sync stateid scenario that looks difficult to resolve

From: Carl Burnett <[email-masked]>
Date: Fri, 12 Sep 2003 09:41:51 -0500

Trond writes:

"Are you perhaps assuming that open_owner == posix file handle (as per
the recommendation in the RFC). That is a concept which doesn't appear
to scale on the server side. We have already hit problems with it
under testing against existing commercial NFSv4 servers which quickly
run out of resources...

For that reason, we'd prefer to use a limited number of open_owners
(say related to the number of allowed simultaneous RPC requests on the
wire) which we share among all the users."

I completely agree that a model where every file object has a unique 
open_owner is non-scalable. We too believe in an approach where a pool 
owners can be shared among many objects. We do however continue to find 
new challenges with this approach that keep increasing the complexity of 
the client implementation. Even with a pool model, IMHO, its going to take 
some pretty serious NFS server hardware to serve V4 at any level of scale.

Just as food for thought, I think removing open share reservations from 
NFS V4 would go along ways towards allowing a significantly more scalable 
protocol. Are share reservations really worth the scalability issues they 
present? Think about it. Locking and delegations could still be supported, 
but with a different bootstrapping mechanism.

Trond also writes:

"Now I am curious, though. Where in the RFC does it say explicitly that
OPEN will invalidate an existing stateid, and why should it be necessary?"

I think this comes from the property, that within the open stateid, there 
is a seqid that the server should increment each time it processes an 
operation that uses the stateid. Thus the stateid continuely moves forward 
in time. As Spencer hinted, this is an addtion to any info within the 
opaque portion of the id that the server is managing, which could change 
with each use of the stateid as well.

Carl Burnett
AIX Kernel Architecture - Network File System
(512) 838-8498, TL 678-8498
(please reply to [email-masked])

Trond Myklebust <[email-masked]>
Sent by: [email-masked]
09/11/2003 07:49 PM

        To:     [email-masked]
        cc:     [email-masked]
        Subject:        Re: [nfsv4] Out of sync stateid scenario that looks difficult to resolve

>>>>> " " == Spencer Shepler <[email-masked]> writes:

    >> So now I'm confused. Does this also mean that you cannot
    >> upgrade/downgrade the open stateid if there is already a lock
    >> stateid hanging off it?

     > For a posix locking scheme, that doesn't apply.  The first
     > close() for a file will release all file locks (if they are
     > left) and therefore the lock stateid is no longer needed.

Are you perhaps assuming that open_owner == posix file handle (as per
the recommendation in the RFC). That is a concept which doesn't appear
to scale on the server side. We have already hit problems with it
under testing against existing commercial NFSv4 servers which quickly
run out of resources...

For that reason, we'd prefer to use a limited number of open_owners
(say related to the number of allowed simultaneous RPC requests on the
wire) which we share among all the users.

Now I am curious, though. Where in the RFC does it say explicitly that
OPEN will invalidate an existing stateid, and why should it be necessary?

Cheers,
  Trond

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: 12 Sep 2003 10:55:38 -0400

>>>>> " " == Carl Burnett <[email-masked]> writes:

     > I think this comes from the property, that within the open
     > stateid, there is a seqid that the server should increment each
     > time it processes an operation that uses the stateid. Thus the
     > stateid continuely moves forward in time. As Spencer hinted,
     > this is an addtion to any info within the opaque portion of the
     > id that the server is managing, which could change with each
     > use of the stateid as well.

No... The open_owner has a seqid, but the stateid is supposed to
obviate the need for a specific seqid once state has been established.
IOW the sequence ids are only required for state changing operations.

What Spencer rather appears to be stating is rather that stateids may
be invalidated by all state changing operations. As OPEN is such a
state changing operation then he argues it may invalidate a stateid.

This gets ugly because the client does not a priori know which stateid
is going to change before it issues the OPEN. Unlike all other state
changing operations, OPEN takes an open_owner and _path_ argument. The
state that it may modify, however, is tied to an open_owner and
_filehandle_ tuple...

Cheers,
  Trond