Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 17 Nov 2004 09:46:28 -0500

Short answer: I don't know yet.

Slightly longer answer: I think we have to understand better how locking,
including byte-range locking (possibly mandatory) will fit into the model.

Some possiblities are that the meta-data server assigns it as part of the
layout, that it is obtained via an fh-based variant of open, and that it is
a fixed special stateid.

The basic reason for a stateid is so that IO operations can be tagged with
what is essentially an owner so that the server can determine if they 
conflict with a lock held by another or are being done by the same open/lock
owner and are therefore exempt from such a check.

Now I assume that share reservations are checked at the meta-data server and
even if you do an OPEN nobody is going to be doing them on a region with 
other than DENY=NONE.  I don't think we've talked about where byte-range
locking is done, but I wasn't expecting that the data repository would 
have these and even if it did there would be some that crossed the region
boundaries.

So to me the first question is whether we need a stateid.  If we do, I'm
not clear that we want to force the client into having two more round trips
to each of the data servers to get them, and even if that is the case, I
don't like the implementation complexity caused by forcing the implementer
to duplicate directory structure on multiple machines and keep them in sync.
Maybe we will be forced to do that but I would want to explore lot of 
alternatives before I went there.

> If the client
> gets what's effectively a region referral to the data server, complete
> with filehandle, then how does it get the stateid?

It is not a referral, as far as I am concerned.  A referral transfers 
responsibility for something and the client goes off and handles that.
It is more like a delegation in that there is still state and ultimate
authority on the meta-data server.  First, the layout is recallable.
Second, primary state handling has to be on the meta-data server.  The
basic open is on the meta-data server.  Otherwise, share resrevations
can't work right.

-----Original Message-----
From: Talpey, Thomas 
Sent: Wednesday, November 17, 2004 8:15 AM
To: Noveck, Dave
Cc: William A.(Andy) Adamson; [email-masked]
Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt 

At 03:57 PM 11/16/2004, Noveck, Dave wrote:
>>At 02:56 PM 11/16/2004, William A.(Andy) Adamson wrote:
>> an fs_locations-like format with server and pathname is one way to express a 
>> file layout that allows striping of a file over NFSv4 data servers. this 
>means 
>> that the pNFS client is doing an OPEN (pathname) to each data server in 
>order 
>> to establish clientid, stateid, etc. so, why the FH argument in the layout? 
>> the OPEN will return the data servers idea of FH and the stateid.

>
>It's one way, but I don't think it is the best way.

Sorry - I don't understand what the best way is, then. If the client
gets what's effectively a region referral to the data server, complete
with filehandle, then how does it get the stateid?

Andy's suggestion seems clean in that regard, since it requires an
open which of course returns one. What's the better way?

Tom.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Craig Everhart <[email-masked]>
Date: Wed, 17 Nov 2004 10:54:45 -0500

> alternatives before I went there.

My model for this is based on block storage, in which everything other than
read and write is handled by the metadata server.  Thus, you open the file
on the metadata server, you get byte-range locks from the metadata server.
If you need mandatory locks, you get them from the metadata server.
The referenced third-party data source is used *only* for read and write.
Share reservation collisions are handled by interactions with the metadata
server.

By the way, the pNFS model prompted by FC block storage also suggests that
file attributes like EOF also reside with the metadata, so that GETATTR
and SETATTR that deals in file length are strictly metadata-server
operations,
not data-server operations, as well as byte-range locks and share/deny
reservations.  Does this jive with the pNFS models prompted by the other
two architectures for third-party storage (objects and files)?  Seems like
it would be OK--after all, the other third-party storage models are usually
composed of multiple sub-things (multiple objects, multiple files, striped
or
concatenated or whatever), so that the whole-file attributes (like
whole-file
EOF) and share/deny reservations need to be dealt with by the central
metadata
server for that whole file.

Yes, yes, we could also get a lot more clever and design more kinds of
third-party storage, including some based on iSCSI so it fits more
obviously
under the IETF umbrella, with just the properties one desires--perhaps
object-inspired security, byte range locking (advisory or mandatory),
whatever.
It's a potentially fertile piece of engineering.  In the abstract it should
happen.  But I don't think that its existence should gate pNFS, which
(IMHO)
should exist to manage existing kinds of storage architectures.

            Craig