Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 24 Nov 2004 10:06:50 -0500

Sure it's OK to return an IO error if data is inaccessible.  What could
be wrong with that?
 
The spec does not define exactly what circumstances would justify you
returning an IO error.  If it did the spec would be way too big.
 
If a cable is disconnected and you can't reach the disks, then sure, you
can return an IO error.
 
So I suppose it could be argued that if there were a manufacturing
problem and the cable was never installed then the same error makes
sense and that's what an implementation would do.
 
So from there, it's a hop, skip, and a short sub-orbital rocket flight
to the idea that that if the design of the server has no cable, no cable
connector, no electronics to drive the non-existent signal into the
non-existent cable, then you can return an IO error since the data is in
fact inaccessible.
 
The problem is that the result is that you have a v4.x server which
cannot interoperate with v4.0 clients, no matter what legalistic
arguments you can make about the formal sufficiency of returning an IO
error.  The point of the minor versioning rules is to allow a situation
in which support for new features (e.g. pnfs layouts) is optional and
that people with existing clients are not forced to change unless they
want the new feature(s).  I think that a putative v4 server which cannot
succeed in executing at least the Connectathon tests does not provide
the necessary interoperability.
 
We have a lot of issues to decide about what needs to be mandatory in
order to prevent pnfs from Balkanizing but I don't think that it should
be an open question whether a pnfs server has to inter-operate
meaningfully with a v4.0 client.  I'm sure there will be arguments about
"meaningfully" but getting through the Connectathon tests is a very low
minimum bar.

	-----Original Message-----
	From: Julian Satran [mailto:[email-masked]] 
	Sent: Sunday, November 21, 2004 4:37 AM
	To: Noveck, Dave
	Cc: Mike Eisler; [email-masked]; Nicolas Williams; Brent Welch
	Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

	[email-masked] wrote on 17/11/2004 18:04:01:
	
	> > I for one I stated several times that it should be OK for a
pNFS 
	> > server not to support V4 as accessing data on the
"traditional" V4 
	> > path (from the server). 
	>   
	> I for one disagree very strongly.  Although it may sound like
it, 
	> "pNFS" is not a separate protocol.  It is a set of extensions
to 
	> v4 and a pNFS server is an NFSv4 server with some optional
features 
	> enabled (just as soon as we define those).  Not acting like a 
	> normal v4 server in response to a read (for example) probably 
	> violates the minor versioning rules (can't say for sure
without all 
	> the details) but it certainly violates the spirit.  We
shouldn't 
	> go there. 
	
	And I still think it is OK to return an error (or in the worst
case a referal) if data is not accessible. 
	What "spirit" (or which "spirit" :-)) is violated? 
	  
	>   
	>   
	> -----Original Message-----
	> From: Julian Satran [mailto:[email-masked]]
	> Sent: Wednesday, November 17, 2004 2:28 AM
	> To: Noveck, Dave
	> Cc: Mike Eisler; [email-masked]; [email-masked];
Nicolas 
	> Williams; Brent Welch
	> Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt
	
	> 
	> 
	> [email-masked] wrote on 16/11/2004 17:32:27:
	> 
	> > So it appears that Nico and Brent are in violent agreement
that
	> > requiring clients implementing the pNFS extensions to
support NFSv4 
	> > as a storage protocol is an OK thing to do.  I agree,
although
	> > I will note that requiring it will make no difference, since
all
	> > clients are going to do that anyway (if you a v4.x client
and you
	> > have general pnfs client logic, you would have to be crazy
not to
	> > support v4.x as a storage protocol).  But anyway, maybe we
have 
	> > a consensus on this issue and can move on.
	> > 
	> > Note however that nobody has said that requiring pNFS
servers to
	> > support v4 as a storage protocol is an OK to thing to do.  I
	> > certainly don't think it is and I suspect Brent agrees.  If
	> > Nico also agrees then maybe we can move on.  Otherwise, we
can
	> > have a detailed discussion on this issue.
	> > 
	> 
	> I for one I stated several times that it should be OK for a
pNFS 
	> server not to support V4 as accessing data on the
"traditional" V4 
	> path (from the server). However I have no opinion about
mandating a 
	> pNFS server to support V4 maps. 
	> Mandating support for V4 maps might even get David Black and
my 
	> position closer :-) 
	>  
	> > As far as Brent saying:
	> > 
	> > > I would like to table all discussion of block protocol
security
	> > > as being out of scope.  I think it clouds the issue and
gets in
	> > > the way of working out an acceptable pNFS extension.
	> > 
	> > It certainly is tempting, but I don't think we can say, in
Lenin's
	> > words, "Comrades, this is no time to have an opposition".
	> > 
	> > The reason that we can't make this out of scope is that the
proposal
	> > which is under discussion is intended to allow block
protocols.  If
	> > block protocols were not possible, because of valid security
issues,
	> > then the base protocol would look different.  So, even
though we 
	> > can discuss such security issues, the bar for those raising
these
	> > security objections is very high.  I think would have to
show that no
	> > possible block protocol could be acceptable (or at least
worthwhile), 
	> > which is a very hard thing to show (since it isn't true or
at least 
	> > I don't think it is).
	> > 
	> > Nico writes:
	> > > So what's in scope is: what sort of protocol should pNFS
clients use to
	> > > talk to the storage.
	> > 
	> > Protocol or protocols?  If it is one, the only possible
answer is
	> > NFSv4 and I think everybody agrees that that is a member of
the set 
	> > of storage protocols and in fact the first one that we
should be
	> > working on.  Any dissent?
	> > 
	> > So I think the question is best put as:
	> > 
	> >    What sort of protocols may pNFS clients use to talk to
the storage?
	> > 
	> > And allowing some scope for individual choice within that
set.
	> > 
	> > When Brent writes:
	> > > 2. pNFS should allow different storage protocols.  These
are OPTIONAL
	> > > storage protocols.
	> > 
	> > I agree with him, but the optionality of these additional
storage 
	> > protocols is not guaranteed to make this stuff safe from
IESG
	> > opposition.  As Nico indicates, we don't know what position
the
	> > IESG will take on this.  We may not even know after they
have
	> > made a decision and carefully explained it to us :-).  This 
	> > kind of uncertainty is just something we are going to have
to
	> > live with, but it should not prevent progress on the generic
	> > parts of pnfs and on storage protocols that won't provoke
	> > these objections.
	> > 
	> 
	> Some storage protocols (mainly block) are insecure in the
sense that
	> they can't control except at volume level access to storage. 
	> I was not aware that this is an area of concern for IESG (as
it is 
	> beyond securing data in flight). 
	> 
	> > When Nico writes:
	> > 
	> > > Perhaps this will fly past the IESG, perhaps not.  I think
we should
	> > > inquire with them at some point (and their answer may be
that they can't
	> > > yet answer so you just have to try to deliver a document
to them and
	> > > see).  Of course, first we need WG consensus on this.
	> > 
	> > I think we need to figure out how we can make progress
without necessarily
	> > have a consensus on the general goodness of block protocols
for pnfs
	> > because we may never get it.  Brent has talked about the
optionality 
	> > of additional storage protocols and my understanding is that
he is
	> > talking about the fact that implementers may choose to
implement them
	> > (or not) and that, if implemented, users may choose to use
them (or
	> > not).  The protocol development process needs to have a form
of 
	> > optionality which I'll call "severability" to distinguish
it.  We need:
	> > 
	> > (1) A general pnfs architecture that allows a set of storage
protocols.
	> > 
	> > (2) A description of how v4.x would be used as a storage
protocol,
	> >     including security considerations.
	> > 
	> > (3-n) A description of how the additional storage protocols
would work
	> >       in the pnfs framework, including security
considerations, with one
	> >       document for each protocol.
	> > 
	> > For me, (1) and (2) are the highest priority items and (1)
is required
	> > for everybody.  I'm assuming that there will be a v4.x spec
that covers
	> > (1) and (2) and that there may have to be negotiation with
the IESG
	> > as to (3), (4), etc. as those documents are produced but we
have to be 
	> > clear that that (1) and (2) can go ahead while hashing out
the issues 
	> > associated with (3-n) to the IESG's satisfaction.  The
explanation in
	> > (1) will have to explain that the intention is to allow
block protocols
	> > if and only if the security issues can be resolved.  I take
the laissez-
	> > faire position that their optionality constitutes an
adequate resolution,
	> > but I am worried that the IESG will adopt a more dirigiste
approach.  I
	> > think pnfs as a whole has to proceed regardless of that (I
fear) endless
	> > debate.    
	> > 
	> > 
	> > -----Original Message-----
	> > From: Nicolas Williams [mailto:[email-masked]]
	> > Sent: Monday, November 15, 2004 11:58 PM
	> > To: Brent Welch
	> > Cc: Mike Eisler; [email-masked]
	> > Subject: Re: [nfsv4] comments on draft-welch-pnfs-ops-00.txt
	> > 
	> > 
	> > On Mon, Nov 15, 2004 at 07:12:12PM -0800, Brent Welch wrote:
	> > > Let me try some simple logic.
	> > > 
	> > > My kitten has green eyes.  Therefore all kittens have
green eyes.
	> > > 
	> > > Block storage protocols are insecure, 
	> > > therefore all storage protocols are insecure.
	> > 
	> > NFSv4 is not a block storage protocol.  Neither will pNFS be
one either.
	> > 
	> > You're comparing kittens to fully grown dogs.
	> > 
	> > Or is pNFS an improvement on block storage?  I don't see it
that way.
	> > Perhaps you do and can persuade us of this view?  I do see
pNFS as an
	> > improvement on _NFS_.
	> > 
	> > Seriously, figure out what security (and operational)
properties you
	> > want a pNFS to provide, figure out what you can get with
each storage
	> > protocol option and let's see where this goes.
	> > 
	> > Also, do we really have enough info right now to figure out
that, yeah,
	> > pNFS w/ iSCSI can be made to work w/o mods to deployed iSCSI
boxes?  I
	> > suspect the answer is "no."  Further, I suspect the answer
is that mods
	> > will be needed.  Finally, I'd much rather see a pNFS
solution that has
	> > security properties that truly are comparable to NFSv4's.
	> > 
	> > > I would like to table all discussion of block protocol
security
	> > > as being out of scope.  I think it clouds the issue and
gets in
	> > > the way of working out an acceptable pNFS extension.
	> > 
	> > We're not trying to improve block storage protocols, but
we're trying to
	> > come up with a parallelized NFS protocol -- a much, _much_
different
	> > beast that block storage protocols.
	> > 
	> > So what's in scope is: what sort of protocol should pNFS
clients use to
	> > talk to the storage.
	> > 
	> > And BTW, I've yet to argue that iSCSI needs improvements.
	> > 
	> > > I am trying to communicate three points:
	> > > 
	> > > 1. pNFS can use NFSv4 as the storage protocol, and should
be just
	> > > as secure as NFSv4.  I think REQUIREing that NFSv4 be
implemented
	> > > by all clients as one of their possible storage protocols
is OK,
	> > > and makes perfect sense.
	> > 
	> > This is my position.  (Though some client<->storage
extensions may be
	> > needed for some things, for all I know.)
	> > 
	> > > 2. pNFS should allow different storage protocols.  These
are OPTIONAL
	> > > storage protocols.  This is important to me because my
preferred
	> > > storage protocol is object storage (iSCSI/OSD).  And,
there are others
	> > > for whom other storage protocols are important (legacy NFS
servers,
	> > > PVFS implementations, block storage on a SAN).  Harking
back to the
	> > > alice in wonderland logic above, it should be OK to allow
other
	> > > storage protocols w/out compromising pNFS when used with
NFSv4 
	> > > or some other secure storage protocol.  Please don't
misinterpret this.
	> > > OF COURSE if a customer chooses to use an insecure storage
protocol
	> > > they are taking a risk.  But, that doesn't mean all
customers are
	> > > taking the same risk because they don't have to use or
provide those
	> > > insecure storage protocols in their environment.
	> > 
	> > Perhaps this will fly past the IESG, perhaps not.  I think
we should
	> > inquire with them at some point (and their answer may be
that they can't
	> > yet answer so you just have to try to deliver a document to
them and
	> > see).  Of course, first we need WG consensus on this.
	> > 
	> > > 3. Object storage has a secure protocol, but it does place
a
	> > > demand on the pNFS client to NFSv4 server communication.
	> > > This is worth debating. It requires that capabilities and
signing
	> > > keys be returned to the client over a secure channel so
they cannot
	> > > be snooped.
	> > 
	> > NFSv4 as the storage protocol is like an OSD that knows how
to evaluate
	> > ACLs.  Alternatives include a pNFS-server<->storage protocol
that
	> > enables a client to do X, and your capabilities token
approach is a
	> > third approach.
	> > 
	> > Cheers,
	> > 
	> > Nico
	> > -- 
	> > 
	> > _______________________________________________
	> > nfsv4 mailing list
	> > [email-masked]
	> > https://www1.ietf.org/mailman/listinfo/nfsv4
	> > 
	> > _______________________________________________
	> > nfsv4 mailing list
	> > [email-masked]
	> > https://www1.ietf.org/mailman/listinfo/nfsv4
	> _______________________________________________
	> nfsv4 mailing list
	> [email-masked]
	> https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Dean Hildebrand <[email-masked]>
Date: Fri, 26 Nov 2004 16:18:24 -0500 (EST)

> "meaningfully" but getting through the Connectathon tests is a very low

Would the NFSv4.X server have to survive Connectathon alone?  It seems
that if pNFS extensions allowed a pNFS server, in conjuction with a set of
data servers, to pass Connectathon then that is sufficient.  This could be
thought of an extension of where a NFSv4.X server uses FS_LOCATIONS to
redirect all clients to other servers, thereby not passing Connectathon by
itself but can "get by with a little help from its friends".

Dean
University of Michigan

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 26 Nov 2004 18:50:48 -0500

> itself but can "get by with a little help from its friends".

Whatever you do has to conform to the rules for minor versioning in
RFC3530. They explicitly state that:

   11. A client and server that support minor version X must support
        minor versions 0 (zero) through X-1 as well.

So there is no question that a pNFS server needs to be able to be able
to talk to NFSv4.0 clients.

    12. No new features may be introduced as mandatory in a minor
        version.

So from the rules, it would appear that a client or server is allowed to
call itself compliant with the new minor revision, without actually
having to implement the pNFS functionality.

Cheers,
  Trond

-- 
Trond Myklebust <[email-masked]>