Subject: [nfsv4] Comments on requirements document

From: Robert Thurlow <[email-masked]>
Date: Thu, 16 Oct 2008 09:43:58 -0600

I went through the requirements document today, and here are
my comments.  I think there are a number of opportunities to
simplify this draft.

Draft:
http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-federated-fs-reqts-00.txt

S2 "Draft Goals" seems inappropriate; that should be in the mail
to the WG asking for review.

S3, p5:
"These fileserver collections are, in general, administered by a
single administrative entity."

This simply does not ring true to me; no company or customer
has a single set of administrators for all of their storage.
There is always some delegation involved, and I think this
text lets us off the hook for considering delegated admin.
It at least has no value.

"Current server collections that provide a shared namespace usually do
so by means of a service that maps filesystem names to filesystem
locations.  We refer to this as a namespace database service (NSDB)."

This sounds muddy, mixing current practice and new terminology.
We should be clearer by completing the description of current
practice before we introduce new terminology.

"An approach which requires changing existing NSDB nodes to
collaborate or replacing them with a single NSDB node, while
possible, is not desirable."

I don't think I understand what this means.

S3, p6:
There is no Figure 1.

S4, p7:
"Our objective is to specify a set of interfaces (and corresponding
protocols)"

Remember that the IETF defines protocols, not interfaces.

"Each singleton server can be presumed to provide its own NSDB node,
for example with a trivial mapping to local FSLs."

Is there value in this statement?  I find it confusing, since
I think of an NSDB as necessarily providing LDAP service.

S5.1, p8:
"A fileset is the abstraction of a set of files and their containing
directory tree.  The fileset abstraction is the fundamental unit of
data management in the federation."

This general stuff should not be in this section, IMHO.

S5.1, p8; S5.2, p9; S5.3, p11:
"There are many possible variations to this procedure"

Are there really that many variations in how we want the actors
to use the protocols we specify?

S5.2, p9:
"In this example, we assume that the only thing directly expressed
by the junction is the junction key"

This is the first usage of "junction key", which is not yet defined.
I think we should state that the server MUST be able to map a path
to an FSN, and that how it does that is a private implementation
decision.  This would simplify the example.  There are ways to store
an FSN directly in the local filesystem, which the example obfuscates.

S5.2, p10:
There is no Figure 2.

"This is a degenerate implementation of the NSDB service as a simple
composition of NSDB nodes; we expect that large federations will use
more sophisticated methods to share the FSN and FSL information among
multiple NSDB nodes."

Is this talking about NSDB replication or multiple namespaces?
Either way, is this text necessary?  It adds complexity.

S5.3, p11:
"Given a local path, a remote export and a path relative to that
export, create a junction from the local path to the path within the
remote export."

What do "local" and "remote" mean in this context?  This does not
parse for me.  The steps don't, either, because it's not clear
what entity takes the actions described.

S6, p12:
"Junction key" may not be a useful concept, since the definition
discusses what I think are internal implementation details on the
fileserver.  In particular, assuming it is used to look up an
FSN seems unnecessary.

S7.1, p15:
The federation protocols do not require any changes to existing
client-facing protocols, and MAY be extended to incorporate new
client-facing protocols.

I suggest changing "do not" to "MUST NOT".

S7.1, p17:
A7 seems unnecessary and vague.

S7.2, p18:
"If the fileset is replicated using the federation interfaces"

Does this have meaning?  Does this really mean:
"If the fileset is replicated, a new FSL is associated with the
original fileset's FSN."

"It is possible to create an FSN for a fileset that has not actually
been created.  It is also possible to bind a nonexistant FSL to an
FSN.  It is also possible to create a fileset without assigning it an
FSN."

Are these useful things to say?  The following paragraph mines
this territory in a more useful form.

S7.2, p21:
What does R7 mean?  An example would be valuable here.

What is the value of R8?  I don't understand why we need this, as
opposed to the more general "give me the FSN at this path" we've
specified in the protocol draft.

Rob T
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: James Lentini <[email-masked]>
Date: Thu, 16 Oct 2008 13:34:15 -0400 (EDT)

> to the WG asking for review.

Agreed.

> It at least has no value.

I agree that we can clarify this. I believe the text is describing the 
common practice of creating a single storage system using a cluster of 
individual nodes. These nodes are "a single administrative entity".

> practice before we introduce new terminology.

Agreed.

> I don't think I understand what this means.

Again, I agree that we can clarify this. My interpretation is that 
this text is recording that the goal of the FedFS protocols is not to 
require existing clustered storage systems to replace their internal 
protocols. Instead, the FedFS protocols are designed to present a 
unified namespace across independent file servers, regardless of 
whether those file servers are a single system or a collection of 
systems.

> There is no Figure 1.

My mistake.

> Remember that the IETF defines protocols, not interfaces.

Agreed, we should clarify this. In previous discussions we've agreed 
that we will develop protocols and any programming interfaces for 
these protocols will be informational.

> I think of an NSDB as necessarily providing LDAP service.

I think we should be careful about mandating LDAP as the NSDB 
protocol. In this document, we should only layout the requirements for 
the NSDB. With that in mind, allowing a singleton server to implement 
an NSDB is reasonable, but perhaps this level of detail 
unnecessary/misleading.

> an FSN directly in the local filesystem, which the example obfuscates.

Again, my mistake.
 
> what entity takes the actions described.

In general, I think section 5 needs an update. I agree that we need to 
review the points you list.

> fileserver.  In particular, assuming it is used to look up an

In the most recent versions of the NSDB protocol and schema, junctions 
have been represented in the NSDB. I think we need to discuss the 
requirements for this.

> I suggest changing "do not" to "MUST NOT".

That seems reasonable to me.

> A7 seems unnecessary and vague.

I don't think it is unnecessary. What do you find vague about it?

> "If the fileset is replicated, a new FSL is associated with the

I think your proposed text clarifies the meaning.

> What does R7 mean?  An example would be valuable here.

Here's an example:

Given two federations A and B, each federation will have a separate 
namespace: namespace A and namespace B. A file server may be part of 
both of these federations. This file server may contain FSLs for 
namespace A and FSLs for namespace B.

> specified in the protocol draft.

I'm not sure about this either.

---

From: Robert Thurlow <[email-masked]>
Date: Wed, 22 Oct 2008 14:46:41 -0600

Thanks for the reply, James - really only a trio of things
to clarify below.

> an NSDB is reasonable, but perhaps this level of detail 

Pardon the usage; I'd restate to say that I expect an NSDB
to be an external mapping service, in which case I don't
see why there is value in commenting on how a singleton
fileserver operates.  Aren't singleton servers outside of
our scope?

I said:
>> fileserver.

I withdraw this comment, since I made use of the term
"junction key" in the overview.  The concept is useful,
as long as we don't describe too much about how it is
implemented.

> I don't think it is unnecessary. What do you find vague about it?

At a high level, it says that things implementing this protocol
must implement this protocol (correctly).  That's a tautology :-)
The "Byzantine server" comment doesn't add enough detail to help.
Do you think this paragraph has clear value?  If so, what is it
saying?

Rob T

---

From: "Everhart, Craig" <[email-masked]>
Date: Wed, 22 Oct 2008 23:01:56 -0400

The only reason to think about singleton servers is to consider the
trivial one-server "collection", where we define "federation" as a
relationship among collections of file servers.  I don't think it's
quite out of our scope.  After all, you may well want to include
singleton servers in a broader federation, and it doesn't seem
reasonable to define yet another way to include them if, for instance,
you were installing a referral in an existing collection (part of a
federation) that pointed to a file system on that singleton server.

And that singleton server might well be one that shares a copy of the
root file system, right?  Or am I being impractical here?

		Craig

> -----Original Message-----
> From: Robert Thurlow [mailto:[email-masked]] 
> Sent: Wednesday, October 22, 2008 4:47 PM
> To: Lentini, James
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Federated-fs] [nfsv4] Comments on requirements document
> 
> Thanks for the reply, James - really only a trio of things to 
> clarify below.
> 
> James Lentini wrote:
> 
> >> Is there value in this statement?  I find it confusing, 
> since I think 
> >> of an NSDB as necessarily providing LDAP service.
> > 
> Pardon the usage; I'd restate to say that I expect an NSDB to 
> be an external mapping service, in which case I don't see why 
> there is value in commenting on how a singleton fileserver 
> operates.  Aren't singleton servers outside of our scope?
> 
> 
> I said:
> >> S6, p12:
> "junction key" in the overview.  The concept is useful, as 
> long as we don't describe too much about how it is implemented.
> 
> 
> >> S7.1, p17:
> At a high level, it says that things implementing this 
> protocol must implement this protocol (correctly).  That's a 
> tautology :-) The "Byzantine server" comment doesn't add 
> enough detail to help.
> Do you think this paragraph has clear value?  If so, what is 
> it saying?
> 
> 
> Rob T
>

---

From: Robert Thurlow <[email-masked]>
Date: Thu, 23 Oct 2008 10:39:32 -0600

> root file system, right?  Or am I being impractical here?

Let me list some things to see what you agree with.  The
following things seem true to me:

- A fileserver is part of the federation if it shares an FSL
   listed in the NSDB

- A fileserver is part of the federation if it contains any
   junctions that point to FSNs in the NSDB

Neither of these cases seem to be about what I'd call
singleton servers.  I'd say a singleton server will not
either return a referral or be the target of a referral,
which is why I think it's out of scope.

Rob T

---

From: "Everhart, Craig" <[email-masked]>
Date: Mon, 27 Oct 2008 11:26:42 -0400

Rob, thanks for the message and the clarifying questions. 

> Sent: Thursday, October 23, 2008 12:40 PM
> 

These are helpful cases to consider.  But I'd say that these cases are
both true for singleton servers.

Some structure would be helpful, and you can tell me whether this makes
sense to you or doesn't.  I hope we've been on the same page.

I'm imagining fileservers in collections, where each collection is
basically the span of where a single NSDB (including its replicas, if
any) is authoritative.  Federated-FS is then about having multiple
collections cooperate in presenting a single name space, e.g., for a
single organization.  That Federation, then, has multiple NSDBs.  In
order to construct the namespace that's presented to NFS4 clients, any
given NFS4 fileserver in the federation may encounter junction
information--an FSN, identifying both an NSDB to use and an entry within
that NSDB.  The NFS4 fileserver consults the indicated NSDB for the FSLs
for the target filesystem ("fileset") and returns the information from
those FSLs to the NFS4 client as a referral.  The NFS4 client then
presents the sewn-together name space that the federation has presented
to it.

In this formulation, an NSDB is authoritative for a set of filesets on a
set of fileservers; it is not only authoritative for these
filesets/filesystems, but it is intimately tied to those filesets in
other ways.  For example, suppose an NSDB X were authoritative for
fileservers X1 and X2.  If the administrator for (X, X1, X2) moved a
fileset/filesystem from fileserver X1 to X2 (e.g., because X1 was
getting full), only NSDB X would have to be updated with the new
location.  Vendors are building such systems today.

With Federations, an organization can build a name space that spans
multiple collections, with multiple NSDBs.  A single organization (e.g.,
cnn.com) could buy a functioning collection (NSDB X, fileservers X1..X9)
and have it interoperate with a separate functioning collection (NSDB Y,
fileservers Y1..Y15), and the single organization could use the two
collections as piece parts in building its organizational name space.
With Fed-FS, any of the fileservers knows how to refer to a different
NSDB that is authoritative for an FSN.

So I'm just hoping that a degenerate "collection" of a single server can
play, too.  I'd like for it to be possible for server Z1 to be added to
a federation.  I can't think of a way to do that meaningfully other than
for filesystems housed solely in Z1 to be described in an NSDB (so that
other fileservers in the federations could refer to filesystems on Z1).
It would also be nice if I could install a junction in Z1 (as an FSN)
and have the NFS4 server on Z1 know how to interpret FSNs.

Z1's NSDB could be a tiny "NSDB" service that somehow comes with the
singleton server, or Z1 could in principle be added to one of the other
NSDBs that exist in the federation.

Does this make sense to you?

		Thanks,
		Craig

---

From: Robert Thurlow <[email-masked]>
Date: Tue, 28 Oct 2008 11:32:56 -0600

> NSDBs that exist in the federation.

It is important to be able to support a server that connects in
like this.  But what I don't understand is why they're different -
why we give them a name ('singleton server') to talk about them.
As far as I can see, this is a simple server, reachable from the
global namespace, and not different in interesting ways from the
other servers playing.  What difference makes the term useful?

Rob T

---

From: "Everhart, Craig" <[email-masked]>
Date: Tue, 28 Oct 2008 13:59:02 -0400

From: Robert Thurlow [mailto:[email-masked]] 
Sent: Tuesday, October 28, 2008 1:33 PM
> 
> Everhart, Craig wrote:
> 
> > So I'm just hoping that a degenerate "collection" of a single server

> > can play, too.  I'd like for it to be possible for server Z1 to be 
> > added to a federation.  I can't think of a way to do that
meaningfully 
> > other than for filesystems housed solely in Z1 to be described in an

> > NSDB (so that other fileservers in the federations could refer to
filesystems on Z1).

> > singleton server, or Z1 could in principle be added to one of the 
> > other NSDBs that exist in the federation.
> 
> It is important to be able to support a server that connects 
> in like this.  But what I don't understand is why they're 
> different - why we give them a name ('singleton server') to 
> talk about them.
> As far as I can see, this is a simple server, reachable from 
> the global namespace, and not different in interesting ways 
> from the other servers playing.  What difference makes the 
> term useful?

Well, you're right, there is no reason for any special terminology.
Maybe it's obvious that a "collection" with just one server still
requires an NSDB in order to qualify as a "collection" under this
definition.  (And that that NSDB needs to be authoritative for what's on
that one server.)  The requirement for an NSDB for even such collections
is, I believe, the only point.  (That was the only point when I drafted
this text long ago.  I don't think that it has acquired any others with
time.)

		Craig

---

From: Robert Thurlow <[email-masked]>
Date: Tue, 28 Oct 2008 12:05:26 -0600

> this text long ago.  I don't think that it has acquired any others with

Got it - I agree that there has to be an NSDB, even for the
degenerate case.

Rob T

---

From: James Lentini <[email-masked]>
Date: Tue, 28 Oct 2008 16:25:21 -0400 (EDT)

On Wed, 22 Oct 2008, Robert Thurlow wrote:

> At a high level, it says that things implementing this protocol must 
> implement this protocol (correctly).  That's a tautology :-) The 
> "Byzantine server" comment doesn't add enough detail to help. Do you 
> think this paragraph has clear value?  If so, what is it saying?

I see value in clearly stating that the protocols are not designed to 
protect against one of the entities, say an NSDB, functioning 
incorrectly.
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Peter Staubach <[email-masked]>
Date: Tue, 28 Oct 2008 16:34:57 -0400

> protect against one of the entities, say an NSDB, functioning 

Just out of curiosity, can you give an example of a high level
protocol such as this, which is designed to protect against an
entity which is not functioning correctly, either from an
incorrect implementation or environmental factors?

    Thanx...

       ps

---

From: James Lentini <[email-masked]>
Date: Tue, 28 Oct 2008 17:02:50 -0400 (EDT)

> this text long ago.  I don't think that it has acquired any others with

I agree with the conclusions of this thread that singleton servers are 
important and that the terminology is confusing. I'll point out that 
in the original introduction (page 5 
draft-ietf-nfsv4-federated-fs-reqts-00.txt), the text reads:

 There are also singleton servers that export some or all of their
 filesystem resources, but which do not contain junctions to other
 filesystems.

In other words a singleton server is a fileserver can be the target of 
a referral, but will not return a referral. I believe that this was a 
typo.

In the Glossary on page 14, the following definition is given:

   Singleton Server:  A server collection containing only one server; a
      stand-alone fileserver.

which implies that a singleton server can both be the target of a
referral and generate a referral.

I'll put this on the list of items to cleanup before re-publishing the 
requirements document.
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Robert Thurlow <[email-masked]>
Date: Tue, 28 Oct 2008 15:20:21 -0600

> a referral, but will not return a referral. I believe that this was a 

Does highlighting this type of server have value?  Why?  What
do we need to say about them?

> referral and generate a referral.

I'd prefer to talk somewhere about the fact that a collection
can shrink to a single fileserver plus an NSDB, and try to
avoid defining a new term unless we need to refer to it in
multiple places.

Rob T

---

From: James Lentini <[email-masked]>
Date: Wed, 29 Oct 2008 11:20:37 -0400 (EDT)

On Tue, 28 Oct 2008, Robert Thurlow wrote:

> > important and that the terminology is confusing. I'll point out that in the
> > original introduction (page 5 draft-ietf-nfsv4-federated-fs-reqts-00.txt),
> > the text reads:
> > 
> > In other words a singleton server is a fileserver can be the target of a
> > referral, but will not return a referral. I believe that this was a typo.
> 

Sure, defining the term singleton server isn't important. The goal is 
to make clear that different storage systems (clusters, single 
servers, ...) can participate in the federation.

---

From: "Renu Tewari" <[email-masked]>
Date: Wed, 29 Oct 2008 10:04:38 -0700

One thing I wanted to highlight and I think James mentioned it that
one way of thinking of singleton servers was as legacy servers that do
not understand or parse any of the federated-fs protocols but can
still be part of the federation as the target of a referral. The idea
was to expand the scope of the federation to include such servers. For
example a v3 server can be part of the federation as the target of the
referral and knows nothing of fed-fs.

regards
Renu