Subject: RE: [nfsv4] Global namespace for v4.1?

From: Julian Satran <[email-masked]>
Date: Mon, 12 Jan 2004 16:25:31 +0200

filling the rainbow - Julo

"Noveck, Dave" <[email-masked]> 
30/12/2003 17:23

To
Julian Satran/Haifa/IBM@IBMIL
cc
<[email-masked]>, <[email-masked]>, <[email-masked]>
Subject
RE: [nfsv4] Global namespace for v4.1?

Same color (in italics ) for me. 

"Noveck, Dave" <[email-masked]> wrote on 22/12/2003 17:07:34:

> 
> 
> 
> > It is not the granularity that I am after. I could live with 
directories
> > or even subtrees.
> 
> OK, glad to hear it.
> 
> I was going by your statements:
> 
> > IMHO a good global namespace should be imposed without artificially 
> restricting 
> > the way components are placed. 
> 
> > It should also not have (or not have it standardized) a mapping to 
> local namespaces. 
> > I.E., in theory a file in the global namespace may be located 
anywhere.
> 
> > Probably you may want to restrict this to directories but you should 
not go 
> > beyond this. 
> 
> >From which I concluded that the current v4 model, which made fs's 
(significant
> trees) the unit of migration and referral, was not acceptable to you.
> 
> > The thing that I am concerned about is that building a global map 
based on an 
> > "arbitrary" composition of local maps will not get you the 
transparency that 
> > every enterprise is looking for - location transparency, migration 
> transparency, 
> > replication transparency. Transparency is what will keep applications 
and 
> > administration separate. 
> 
> Maybe I'm not understanding the problem you are worried about.  As far 
as what
> I have proposed, each fs can be at an arbitrary position within the 
namespace,
> and that typically will not depend on the server or servers on which the 
fs is 
> currently hosted.  Does that address your concern?
> 
Arbitrary in reference to what? If it is in a fixed position in the global 
namespace that addresses my concern. 
Arbitrary with respect to its position in the local namespace of the 
server hosting it.  My understand of your concern was what you were 
worried that there was some sort binding between the local name and the 
position in the global namespace and I don't think anyone is proposing 
that there is. 
 
> > As for teleport - I never though about AFS in those terms (should I 
have?).
> 
> I don't know AFS well enough to understand how it deals with the 
granularity
> issue.  I know there is a global namespace but is it the case that 
directories
> within it can be arbitrarily located and still manipulated as if they 
were
> within a single fs?  If so, then it does seem to me kind of 
science-fictional 
> and something that is rather distant, in the context of a protocol like 
NFS 
> where multiple separately-derived implementations need to interoperate.
> 
Not necessarily - in AFS the "local names" in the local filespace have no 
relevance to the global names 
However the same effect could be achieved with "bigger granules". 
Interoperability - I am not aware of any interopearbility issues that 
become simpler by that or another choice of building the global tree. 
Let me  try again.  If your granules are visible to clients as file 
systems, then interoperability becomes a matter of the server providing 
some version of acceptable v4 semantics and the client being able to do 
that.  If you have granules that are smaller than that, in that a client 
sees two files or directories on different servers as part of the same fs, 
then the ante on interoperability goes up quite a bit.  For better or 
worse, v4 allows different servers to implement their own fs semantics 
(within limits) and takles the position that it is not the jobof the 
protocol to impose a semantic uniformity, but merely to provide remote 
access to the fs semantics exported by the server.   Further, an action 
like renaming a directory on one server between two parent directories on 
two different servers becomes very interesting if we have to test with the 
possible range of implementations of client, target directory server, 
source parent directory server, destination parent directory server. 
I already agreed that appropriate granules may simplify some 
implementations (although AFS is a good proof to the contrary!).
I am afraid however that I do not agree to the assumption that different 
fss will surface wildly different semantics. Having a global namespace 
without basic semantic assumtions is hardly acceptable. You do not expect 
clients to be aware of the semantic difference?

As any decent implementation will use some form of name caching any local 
naming action is bound to have global effects. As for semantic uniformity 
I am not sure that I understand or agree. The naming structures and rules 
have to be uniform over the global space for it to be really useful (you 
are not suggesting changing character casing rules or name lengths when 
crossing implementation boundaries). 

I am not suggesting that such variability would be a good thing or 
something that I would implement in a global namespace.  The problem is 
precisely how to prevent it.  V4 servers may implement case-sensitive or 
case-insensitive naming (and it allows servers to refuse to rename open 
files, or to allow that, etc.)  That kind of variability wouldn't be my 
choice but it was the choice of the working group.  If you have a global 
namespace which is fine-grained (i.e. the distribution units are smaller 
than a file-system), then this is an issue that you have to address, 
somehow and it is one of many in which the definition of v4, by allowing 
servers different semantic choices, complicates the implementation of a 
fine-grained global namespace.  It is one of many issues that one needs to 
address. if you want to go down this route.
 
 I think that a clear set of export rules a global namespace constructors 
will simplify interoperability not complicate it and an attempt to build 
something that is no more that the chain of mounts in previous versions is 
not very useful. 
If you can define such rules, and the group agress to them within a 
reasonable time, then that would be great, but I fear that it may be quite 
difficult.  As far as the usefulness of something more limited and 
coarse-grained, I guess we just disagree.  I'm willing to settle for 
something not as good, that I know can be built, without confronting some 
basic issues in v4.  I think the result is still quite useful. 

I guess that the only thing I am looking for  a "single global map" - i.e. 
besides the export "fragments" present in present NFSV4 a map of all those 
fragments being the same for all participating servers. It can be exported 
by a limited subset of servers (at least 2 for redundancy) with some 
simple decision rules if the maps do not agree.
------------------- 

If you limit yourself to referrals servers or the referal-servers will 
have to guarantee the "unique" structure of the global name-space. 
Don't know what you mean here. 
My understanding is that you will use referrals wherever you reach a 
"mount point". Those could be different on each sever (or you have to 
replicate on all or use a separate global name/referral server.

There are two classes of referrals here to deal with.  The first is those 
that are referrals from mount points within a real filesystem (i.e. a 
filesystem that has real files within it and not only directories that 
lead to referrals).  For this class, you only have a consistency issue 
when the filesystem in question is replicated and I would presume that the 
identity of the referral will be assimilated to the general issue of the 
identity of the data within the individual replicas of the replicated fs. 
There has been in the past some discussion of a v4 server-to-server 
migration and replication protocol to allow such replicas to be 
co-ordinated, even among different types of servers, but there has been no 
progress on that recently.  Without that, replication is going to be 
implemented individually and it is the responsibility of the servers 
implementing the repllicas to keep them the same including the same 
referrals.  As a practical matter, the server implementations must be the 
same.
 
The second class of referrals are those in what we might call global 
pseudo-fs's, which implement fs's devoted to providing a global or 
enterprise-wide or site-wide namespace, by means of a set f referrals 
within the fs.  Today, we are in a similar situation to the first case in 
that only implementaton-specific replication is available.  My assumption 
is that work will proceed on a naming service definition (LDAP or 
whatever) and then referral servers or referral fs implementatins will 
fetch data from a common name service with multiple server sproviding the 
same data sicne they are obtaining them from the same source (issues of 
dealing with stale data from the name service as configutations are 
changing is left as an exercise for the I-D writer).

With the scheme I suggested above any client will mount the same global 
structure wherever it sees fit but the global structure is guaranteed to 
be unique.
This will include only "global" fragments. Also the map may use only art 
of the exported fragments - namely those forming the global space.

> The v4 approach relies, for its simplicity, on the fact that the 
separately
> located subtrees are separate fs's and thus you cannot link files 
between them,
> move directories between them, etc.  That's why I characterized that 
> as "walking",
> i.e. something we know how to do, can be done within the bounds of 
RFC3530,
> albeit with a number of rough edges, and only requires that we travel 
the
> road laid out for us, putting one foot in front of the other.
> 
I am not sure that I would characterize moving directories as a bad thing

I wouldn't characterize it as a bad thing, either.  I would however, 
characterize it as a difficult thing when the participating directories 
are in different servers.  If people want this functionality badly enough, 
then the people in this group will work to provide it but it would not be 
easy.  So maybe this indicates that perhaps "teleportation" is the wrong 
metaphor.  Maybe a better analogy would be a controlled fusion power 
reactor.  The basic physics is understood and it is just a matter of 
figuring out the engineering of confining a plasma at the appropriate 
temperatures and pressures.  It's just work and money and time. 

You seem to suggest that it will never work (or it will take as long as 
fusion power). But AFS was (is) here for a long time and people love it 
 
It works in the context of AFS, and that's fine.  Making it work in the 
context of NFS presents a whole different set of challenges.  To stretch 
my analogy a bit more (probably to or beyond the breaking point), the sun 
has been here for a long time and people love it and it produces power 
from thermonuclear fusion with no problem. The problem is transferring 
that form of power production to an environment in which a strong 
gravitational field is not available to neatly solve the confinement 
problem.  There are lots of things that AFS can do easily that NFS has 
difficulty doing.  We want to retain NFS's basic structure ot modify it 
incrementally.  This imposes constraints on our ability to get some of the 
desirable features of AFS into NFS, as much as we'd like them.
 
Perhaps I'm being unduly pessimistic.  My perception of the difficulty is 
basically a reflection of the fact that I don't know how to do this.  If 
you (or someone else) knows how to do this within the constraints of NFS 
(making only moderate changes to NFS, or making radical changes and 
convincing the working group as a whole that these are justified) , then I 
would love to see an I-D explaining how to do this.   I think doing this 
on fine-grained basis is very difficult and that we are going to have 
enough work handling a global namespace on an fs granularity basis. 
However, I would be quite happy to be proved wrong.
 
 The scheme I suggested can be summed up in a page or two (no need for the 
7 pages of overhead). Unfortunately I don't feel qualified to do it (I 
will have to get acquainted with many of the NFSV4 detail). But if nobody 
else comes forward...

---

From: Jon Haswell <[email-masked]>
Date: Tue, 13 Jan 2004 10:45:49 -0800

On Tuesday, 01/13/2004 at 12:41 EST, Jim Rees <[email-masked]> wrote:

<snip>

>
> I don't know if we should use dns, ldap, or something else, but we do
need
> to all agree on the same global root directory if we are going to
implement
> a global name space.  Having each domain keep its own set of referrels to
> all the other domains, if that's the only mechanism, is bound to fail.
>

Can you just clarify what you mean by 'the same global root directory'. I'm
assuming you are referring to the same mechanism, i.e. one type of database
with a defined schema here. I don't believe the actual global root
directory name itself should be specified so users can retrofit a naming
scheme they are familiar with into the system.

I agree that we need a hierarchical naming system from a common root for
each network - we should allow private networks I assume in addition to a
single unified public network with a single root.

Jon Haswell
IBM Research

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Jim Rees <[email-masked]>
Date: Tue, 13 Jan 2004 14:03:51 -0500

Can you just clarify what you mean by 'the same global root directory'. I'm
  assuming you are referring to the same mechanism, i.e. one type of database
  with a defined schema here.

No, I really do mean "the same."  For example, all users of dns are using
"the same" root directory, because they all go to the same set of root
servers (A.GTLD-SERVERS.NET. and friends).  The directory is replicated, but
all replicas contain the same data (if everything is working).

Without this, you don't have what I would call a global name space.

  I don't believe the actual global root
  directory name itself should be specified so users can retrofit a naming
  scheme they are familiar with into the system.

I disagree.  If the same name refers to different files depending on the
client's location, you don't have a global name space.  Maybe v3 users don't
want a global name space.  I don't know enough about v3 users to say.

You can of course have private names that are not part of the global name
space.  But I do think it's valuable to also have a true global name space.

---

From: Mike Eisler <[email-masked]>
Date: Tue, 13 Jan 2004 10:56:40 -0800

Brent wrote:

 > Let's not get bogged down in deciding the naming structure
 > that *we* want.
 > Let's give customers something flexible enough that let's them build
 > what *they* want.

I agree. I've worked for several different places that had the automounter deployed, and
only one of them deployed the automounter the way it came out of the
box (e.g. instead of /home/mre, it was /usr/u/mre or /u/mre).
Besides which, there is the X/Open XFN standard already
out there. I think the lesson learned from that was the customers were as interested
a standards body's notions of naming structure as they were in vendors notions.

In a different message, Brent wrote:

> I hate to belabour a point, but if the client interacts directly
> with the name service (rather than through an NFSv4 server intermediary)
> then there is no new "root server authorization" problem to solve.

My problem is that I've little confidence that there will be
implementations of combined automounter/nfs filesystem pairs that will
support rebinding of mount points. My lack of confidence is derived from
the lack of recenmt progress in current automounter technology.
A good example of this is readonly client side failover feature
that appeared in the mid 1990s.
Nearly ten years later, we still don't have one that re-consults the
automounter map and injects a updated list of NFS servers into the
in-kernel mount table entry, despite the fact that this is easy to
do. Having the client and not the NFSv4 server be the focal point
for the namespace has proven to be an impediment to progress in this area.

> Those clients that are already using a secure name service (e.g. LDAP or
> Active Directory) have previously authenticated the provider of the service
> and rely on it to provide lots of other enterprise data.

Clients have to authorize/authenticate several different servers for
several different services today. I don't see the issue with adding one more.

> Data for a global filesystem name service is just an add-on.

In a different message, Jim wrote:

> Can we please refrain from using v3 terms like "automounter" and "map?"  I
> don't know what these mean.

No sorry. :-) There's plenty of documentation and white papers of these
concepts. This being the NFS working group it is reasonable to expect
people to know what these concepts mean.

The folks who are advocating client-centric global namespace will need to
provide a story for how easy this will be to implement. Using automounter
technology as a basis is a reasonable way to reduce impediments. If people
are advocating a vaporous basis, then my skepticism increases.

> Afs traditionally has the root of the global name space in two places, one a
> static file on the client, the other a set of mount points on the server.
> These have to be in sync for things to work.  This is obviously bad and we
> should not do this.  Afs is currently being re-worked so that dns (via the
> afsdb record) is the root of the name space, and is visible in /afs via
> magic in the vfs.
> 
> I don't know if we should use dns, ldap, or something else, but we do need
> to all agree on the same global root directory if we are going to implement

I like the idea of recommending or mandating a dns record for telling a client which
NFS server(s) know about the root directory. This makes multi-domain easier, and it
makes single domain less crufty.

---

From: Jim Rees <[email-masked]>
Date: Tue, 13 Jan 2004 14:11:08 -0500

I'm not suggesting we ignore v3.  Just that when we use v3-only terminology,
please define it first.

It's become apparent that we don't all agree on the meaning of "global name
space" (see my previous message).  The two meanings I've seen are:

1. truly global, that is, the same name refers to the same file everywhere.

2. site global, that is, the same name refers to the same file everywhere
within a given administrative domain.

It's unclear to me now whether we view "truly global" as a goal.  I think it
should be.

---

From: Jon Haswell <[email-masked]>
Date: Tue, 13 Jan 2004 11:20:14 -0800

> Without this, you don't have what I would call a global name space.

I agree with you here - what I was really thinking about was the ability
for the global namespace to be able to be mounted on each client machine in
a way that mirrors existing pseudo-global namespaces - hence if users are
expecting their data to be at /net/xxx then we should make sure that can
still happen. I absolutely agree that we should copy dns in terms of the
actual root directory and it's consistency.

> want a global name space.  I don't know enough about v3 users to say.

We have to allow admins at each enterprise/customer be abel to map this
global namespace into their existign scheme otherwise they will resists
using it, i.e. I think we need views fo the global namespace. There is one
consistent global view around the world and it is possible for admins to
establish their additional local view that mirrors some existing scheme
they have as another way of viewing a portion of the global namespace.

Jon Haswell
IBM Research

---

From: Jim Rees <[email-masked]>
Date: Tue, 13 Jan 2004 14:33:29 -0500

We have to allow admins at each enterprise/customer be abel to map this
  global namespace into their existign scheme otherwise they will resists
  using it, i.e. I think we need views fo the global namespace.

Just to see if I understand...  I might want to have /home/rees refer to one
place if I'm at UM, and another if I'm at IBM.  That seems reasonable, if
there is also a way to unambiguously refer to the same directory from
anywhere, for example /nfs/umich.edu/home/rees.  Is that what you're
suggesting?

This implies multiple names for the same mount point, which implies possible
cycles in the name space, but I can live with that.

---

From: Jon Haswell <[email-masked]>
Date: Tue, 13 Jan 2004 11:46:40 -0800

> cycles in the name space, but I can live with that.

Yes, thats basically what I am suggesting

Jon Haswell
IBM Research