Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 19 Nov 2004 12:27:42 -0500

> Spencer Shepler suggested that this W-G should also
> work on the server-to-server storage management
> protocol which I think is a fine idea, yet I
> think that it shouldn't be mandatory to implement,
> as other implementation that use different
> architectures such as SAN or OSD storage
> might find this requirement cumbersome.

I think lots of people (among all the variants) might find 
it cumbersome, but that's just a guess.  We're judging 
something that doesn't exist.  I guess that doesn't
inhibit me the way it probably should :-)

I don't think Spencer said that the working group should
work on this, but he can clarify.

What I understood Spencer to say was that he was troubled
by the statement that such a protocol was out of scope.
Now my interpretation of that is that if someone were to
devote the effort coming up with a proposal in this area,
we should listen, and do that seriously.  The thing is
I don't expect such a proposal to be forthcoming. I 
think the working group already has plenty on its plate
with pNFS (no matter how many variants we decide to 
handle initially, as long as it is greater than zero), 
other v4 minor versioning work, RDMA, etc.  Is there 
anyone out there who really wants to devote serious 
effort to a storage management protocol?

Maybe I've misunderstood Spencer.  He did make reference
to migration stuff and some issues we have had with 
implementing it, and suggested that somehow, doing the
server-to-server protocol when we were doing v4
itself would have avoided those.  First of all, I don't
think so.  I think the problems derive from the fact that
we put this stuff in at the last minute when the doors of
v4.0 were closing.  The fact is that nobody had time to
do the server-to-server protocol.  There was no chance it
would have been done in the v4 time frame unless that
timeframe were very considerably expanded, as the fact 
that it still hasn't gotten done might suggest.  That 
was another fine idea that could not command sufficient 
interest or resources to get done in a timely fashion.  I 
suspect that that the storage management protocol would be 
similar.  A lot of work, not a whole lot of real interest, 
something that may happen in the future when those things 
change.

Spencer wrote:
> Secondly, this discussion appears to have a lot of the "how is it
> coordinated" text and it concerns me that if the backend behavior and
> capabilities are not reasonably understood that the client->server
> result will be unsatisfactory.

I do agree that when we describe how a storage protocol will work we need
to have a mental model of how, in general, it is expected that the storage
management protocol will work.  And in order to make sure that everybody 
has the same mental model it helps to write it down.  I think the documents
that describe storage protocols should keep that in mind.

-----Original Message-----
From: Halevy, Benny [mailto:[email-masked]]
Sent: Friday, November 19, 2004 10:02 AM
To: 'Craig Everhart'; Yoder, Alan
Cc: [email-masked]
Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

I agree with Craig and I'd like to add a couple of points:

All major SAN FS players were involved with pNFS
from the very early stages.  Doing pNFS as an extension
to NFSv4 was the proposed way to go
from day one (http://www.citi.umich.edu/NEPS/)
and that was discussed and accepted both by the
participants of the ad-hoc pNFS discussion group
and by members and chairmen of the nfsv4 W-G.

I assume that the SAN filesystems folks have
joined this working group if they weren't subscribed
already.

> Personally, I'd argue against trying to merge
> them, on the grounds that that is the job of
> a management layer.  SNIA is going to wrap a
> management layer around parallel access 
> protocols no matter whether there is one big
> one or N little ones.  The time it takes to
> do this will be about the same in either case.
> And since N little IETF protocols can be developed
> in parallel in less time than one big one, that
> argues that you'll get a better time to market
> for the whole package by keeping pNFS small.

I don't see pNFS as "one big one".  The proposed
extensions represent a functional layer
common to several existing SAN and object based
filesystems. At the pNFS level, the gory details
of the underlying filesystem are passed to the
client in embedded, type-coded opaque data structures
(e.g. the layout).  Our intention is to standardize
sub-types within the IETF as separate RFCs that can
(and should) be developed in parallel to the pNFS
extensions.

Spencer Shepler suggested that this W-G should also
work on the server-to-server storage management
protocol which I think is a fine idea, yet I
think that it shouldn't be mandatory to implement,
as other implementation that use different
architectures such as SAN or OSD storage
might find this requirement cumbersome.

 -----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Craig Everhart
Sent: Friday, November 19, 2004 5:32 AM
To: [email-masked]
Subject: RE: [nfsv4] comments on draft-welch-pnfs-ops-00.txt

Alan Yoder <[email-masked]> wrote on 11/16/2004 01:03:27 PM:

> In other venues, protocols that do for block 
> protocols what pNFS does for NFS are being
> worked on.  The folks I've talked to call
> these "SAN Filesystems".  

Right.

> If anyone wants these efforts to be merged, 
> I'd think they ought to be talking to 
> the SAN Filesystem people too, not just the 
> folks here.

Who else needs to be told?

My day job is working on a SAN file system, so at least
some of the SAN Filesystem people are here.

> Personally, I'd argue against trying to merge
> them, on the grounds that that is the job of
> a management layer.  SNIA is going to wrap a
> management layer around parallel access 
> protocols no matter whether there is one big
> one or N little ones.  The time it takes to
> do this will be about the same in either case.
> And since N little IETF protocols can be developed
> in parallel in less time than one big one, that
> argues that you'll get a better time to market
> for the whole package by keeping pNFS small.
> 
> Alan

Are you saying that nfsv4 isn't the right group to
be discussing this, that it's not properly an IETF
thing?

I don't believe that the pNFS effort has imagined that there
would be one big protocol, but rather a smaller one with
a choice of third-party data access protocols.  Perhaps, as
discussion has happened here, use of one of those protocols would
be required for conformance.  I don't really say it that
way; here's why.  The pNFS effort was doing a couple of
things: one of them was to define new third-party data
transports as NFS extensions, but equally (or, from my
own point of view, more so) it was trying to integrate
third-party transport mechanisms into NFS, allowing
NFS to be the client-server protocol that could be used
as part of a SAN Filesystem (as you name it).  I'm
myself very interested in an NFS client being able to
do block SCSI over FC to a storage device.  But the particular
protocol that could be used to talk to a storage device is
more likely to be chosen in advance by whoever deploys the
storage device.  It could be that a single machine could
talk block SCSI over FC to one device, OBS SCSI to another
device, and one or two variants of sub-file that we see
being discussed on the list to a third or fourth device.
But it may not be likely that *all* those protocols will
be deployed at a given client, and it's not likely that their
definitions and interoperability stories will mature at the
same rate.  So what we had discussed as a possibility was
thinking of the pNFS extension to NFS as a single entity with
multiple sub-types.  The sub-types could progress towards
interoperability status independently, while at least one
of them would be required in order to allow the production
of "working code" for pNFS.  That is, pNFS proper, dealing
with layouts and such, doesn't care which sub-type is in use,
as long as there is one that is implemented by the particular
client and the particular third-party storage (storage system
or OBS disk or "data server" via NFS).

So some folks are saying that the best way to achieve this
is to pick one sub-type and consider that alone for pNFS first,
then strive to add other sub-types only after pNFS is much
further along.  But doing this risks consensus over the choice
of one of the variants, and it's also a recipe for
substantial delay for the less-preferred sub-types.

Would it be unprecedented to have a handful of possible
sub-types and require one or more of them, but no matter which,
to test pNFS implementations?  I remember the early IP
specification RFC, followed by a few on "IP on Experimental
Ethernet" and "IP on Chaosnet" kinds of stacking discussions,
but they may not be that helpful as models--we're not really
doing stacking, but we have a variety of embeddings for the
protocol.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Spencer Shepler <[email-masked]>
Date: Fri, 19 Nov 2004 15:35:41 -0600

> that describe storage protocols should keep that in mind.

I would like to think that the essence of my comments about the server
management protocol was to "keep the open mind".  I know the effort to
define the individual draft submitted by Brent W. has been going on
for awhile and there has been good discussion so far.  However, the
conversation has moved to the NFSv4 WG and there will be additional
people and further ideas.  Initially, let's keep the discussion open
or at least manage it in a way as to foster the best chance for usable
proposals.

Specifically, Dave Noveck's suggestion of a model of operation for the
storage management protocol is good.  I see that as a minimum and if
more is done, all the better.

Spencer