Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 16 Nov 2004 15:57:47 -0500

> an fs_locations-like format with server and pathname is one way to express a 
> file layout that allows striping of a file over NFSv4 data servers. this means 
> that the pNFS client is doing an OPEN (pathname) to each data server in order 
> to establish clientid, stateid, etc. so, why the FH argument in the layout? 
> the OPEN will return the data servers idea of FH and the stateid.

It's one way, but I don't think it is the best way.

> there is the question of a storage management protocol between the NFSv4 
> metadata server and the NFSv4 data servers - as dave pointed out, a SETATTR 
> with ACLs needs to be propagated to the data servers. 

I didn't say that.  I only said that the ACL's have to be enforced
somehow with the how up to the implementation.

> the namespace will need 
> to be kept in sync -  RENAME, REMOVE, CREATE, will also need to be 
> synchronized between metadata and data servers.

Which is exactly why I think doing this in this way is a bad idea.

If the metadata server has info on the data then RENAME has no 
need to propagate changes to the data servers.  Also REMOVE can
be done asynchronously.  CREATE does require communication from
metadata server to data repository but it looks to me like the
cleanup requirements do deal with failures are a lot simpler if
I can just send out requests and get back filehandles.  If the
metadata server fails before everything is created, I just have
a few empty files which can be cleaned up at leisure as opposed
to a situation in which CREATE's have been done on the repository
and they must be expunged immediately for correct operation to
proceed.

So it appears that even within the v4 case there is significant
disagreement about this would all work.  Which is another reason 
that getting that first storage protocol may take some time, with
storage protocols two through 4 billion taking a lot more time.

-----Original Message-----
From: William A.(Andy) Adamson [mailto:[email-masked]]
Sent: Tuesday, November 16, 2004 2:57 PM
To: Nicolas Williams
Cc: Noveck, Dave; Mike Eisler; Brent Welch; [email-masked];
[email-masked]
Subject: Re: [nfsv4] comments on draft-welch-pnfs-ops-00.txt 

> On Tue, Nov 16, 2004 at 01:14:46PM -0500, Noveck, Dave wrote:
> > So the question is, how do you create a meaningful requirement?  
> > OK, it's mandatory to provide a NFSv4-style layout.  Nothing could
> > be simpler.  The layout for file handle FH says that the data is
> > at FH' and IP address X (one of the servers IP addresses) and your
> > done.  Is that OK?  The performance of accessing the data using 
> > that layout will be no better than accessing it over standard 
> > NFSv4, but do we want to get in the business of making performance
> > requirements?
> 
> Huh?  Why couldn't the meta-data server say
> 
> [{fs_location1, FH', byte-ranges}, {fs_location2, FH', byte-ranges}, ..,
>  {fs_locationN, FH', byte-ranges}]

an fs_locations-like format with server and pathname is one way to express a 
file layout that allows striping of a file over NFSv4 data servers. this means 
that the pNFS client is doing an OPEN (pathname) to each data server in order 
to establish clientid, stateid, etc. so, why the FH argument in the layout? 
the OPEN will return the data servers idea of FH and the stateid.

there is the question of a storage management protocol between the NFSv4 
metadata server and the NFSv4 data servers - as dave pointed out, a SETATTR 
with ACLs needs to be propagated to the data servers. the namespace will need 
to be kept in sync -  RENAME, REMOVE, CREATE, will also need to be 
synchronized between metadata and data servers.

> And if OSD-like extensions to NFSv4 are needed that's fine with me.
> 
> >                You can argue that this is an evasion of the
> > requirment but it just isn't possible to draw the line between
> > evasion and avoidance in a way that sticks.  So a client would
> > learn to ignore this sort of layout, and use the ones that that
> > server makes faster.  In which case, why should we ask the server
> > to pretend?  If he is set up so that pnfs-blocks is fast, then 
> > why should we ask him to tell the client that pnfs-files is 
> > available, when it isn't, for all practical purposes.
> 
> See above.  I do think that NFSv4 may need extending to make it a
> high-performance proposition for this.
> 
> > > > The reason that we can't make this out of scope is that the proposal
> > > > which is under discussion is intended to allow block protocols.  If
> > > > block protocols were not possible, because of valid security issues,
> > > > then the base protocol would look different.  So, even though we 
> > >                                                 ^^^^^^^^^^^^^^^^^^
> > > > can discuss such security issues, the bar for those raising these
> > >   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > security objections is very high.  I think would have to show that no
> > >   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > possible block protocol could be acceptable (or at least worthwhile), 
> > > > which is a very hard thing to show (since it isn't true or at least 
> > > > I don't think it is).
> > 
> > > Hooey.  The NFSv4 WG charter says nothing about pNFS, so I think all
> > > aspects of any pNFS proposals are up for discussion until consensus is
> > > reached.
> > 
> > Sure, but your right to discuss things has to deal with my (and others')
> > right to decide what arguments to listen to.  What I guess I was saying
> > in the above sentence is that for me, the appropriate time to discuss
> > the security flaws of a particular proposal to use a blocks protocol
> > is when such a proposal is actually made, i.e. there is a document that 
> > really defines what is proposed.  Until then, we are dealing with 
> > stuff that is speculative, unless you can show me that it is impossible
> > that any proposal in this vein will be acceptable.   
> 
> The authors termed the security of pNFS "comparable" to NFSv4's -- in
> their slides at last week's meeting.  No, I have not read their
> document, but this is enough for me to get concerned.  Here I start to
> repeat myself; I'll not and instead I offer to review the new version of
> the pNFS I-D(s).
> 
> > > Anyways, the burden is on the authors.  Let them show us the various
> > > combinations they have in mind and let them tell us what their security
> > > and operational properties are.
> > 
> > I agree, once they want to make specific proposals on additional storage
> > protocols.  For me, saying that objects are interesting and that the
> > general pnfs framework should accommodate blocks protocols, does not 
> > count as making a specific proposal.
> 
> I will now shut up and wait for the -01.  We're close to going in
> circles.
> 
> The authors now have several copies of my comments on what I'd like to
> see from them, as well as lots of comments from others, including
> Mike's, who started the whole pNFS security discussion.
> 
> Over to the authors.
> 
> Cheers,
> 
> Nico
> -- 
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nfsv4
> 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 17 Nov 2004 08:15:25 -0500

At 03:57 PM 11/16/2004, Noveck, Dave wrote:
>>At 02:56 PM 11/16/2004, William A.(Andy) Adamson wrote:

>It's one way, but I don't think it is the best way.

Sorry - I don't understand what the best way is, then. If the client
gets what's effectively a region referral to the data server, complete
with filehandle, then how does it get the stateid?

Andy's suggestion seems clean in that regard, since it requires an
open which of course returns one. What's the better way?

Tom.