Subject: [nfsv4] Issue46 - extend fs_location_info to support arbitrary string-based attributes

From: Mike Eisler <[email-masked]>
Date: Thu, 13 Apr 2006 22:36:06 -0700 (PDT)

This is a requirement that came directly from a customer. So on the
theory that customers matter, I'm setting the priority to required.

The problem is that a customer might have defined a set of replicas that
such that each replica is the same logically, but subsets are attributes that
are useful only to certain classes of clients. An example to make this crystal
clear is that a filesystem replica might contain a copy of /usr/bin with
compiled exetuables. There are Linux clients that access these replicas but
some are on an i86-32 bit platform and others are on a sparc-64 bit platform.
Both classes of clients want to see /usr/bin/ls, /usr/bin/cat, but of course
the binary bits are different for each class. So if the client can search for
replicas by say a string attribute called CPU, it can match the replicas
for x86 chip set with the clients running that chip set, and the replicas for
the sparc chip set with the clients running the sparc set.

An analogy is the NFS automounter's builtin variables, such as $CPU.
NFS automounters also allow arbitrary variables to be defined. In either
case, an automounter map entry might have:

   notes  -rw srcsev:/source/notes.$CPU

The NFSv4 global namespace entry for /nfsv4/eisler.org/docs/notes might note
that it needs to look for a version of this file system that has the CPU attribute
set to CPU attribute defined on the client.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 14 Apr 2006 09:25:21 -0400

> set to CPU attribute defined on the client.

Please don't try to introduce the AFS '@sys' crap into NFSv4.

This is something that can and should be done using the automounter:
since it is _client_ driven.
I remember all too well the sort of confusion that we had at CERN
because no two groups of software packagers could agree on the correct
labelling for Linux AFS clients (do you add in the kernel, the
distribution, etc?). @sys has nothing to do with global namespaces.

Cheers,
  Trond

---

From: Carl Burnett <[email-masked]>
Date: Fri, 14 Apr 2006 11:00:32 -0500

Mike,

This does not seem appropriate as an fs_location_info extension. If the 
content is different, its not another instance of the data. I agree with 
Trond's follow up that this is more of a client driven capabilility, just 
as it currently is with automounter and the @sys/@host models from AFS and 
DFS.

I could maybe see it as some type of special extension in a lookup type 
operation to a server, or maybe some type of  named attribute, or maybe 
its just another FSID wide attribute. Something like an architecture type 
string. Of course, if its a server based propery, there will need to be 
some type of naming convention, or more likely, a registration process.

Carl

Carl Burnett
AIX Kernel Architecture - Network File System
(512) 838-8498, TL 678-8498
(please reply to [email-masked])

Mike Eisler <[email-masked]> 
04/14/2006 12:36 AM
Please respond to
[email-masked]

To
[email-masked]
cc

Subject
[nfsv4] Issue46 - extend fs_location_info to support arbitrary 
string-based attributes

The problem is that a customer might have defined a set of replicas that
such that each replica is the same logically, but subsets are attributes 
that
are useful only to certain classes of clients. An example to make this 
crystal
clear is that a filesystem replica might contain a copy of /usr/bin with
compiled exetuables. There are Linux clients that access these replicas 
but
some are on an i86-32 bit platform and others are on a sparc-64 bit 
platform.
Both classes of clients want to see /usr/bin/ls, /usr/bin/cat, but of 
course
the binary bits are different for each class. So if the client can search 
for
replicas by say a string attribute called CPU, it can match the replicas
for x86 chip set with the clients running that chip set, and the replicas 
for
the sparc chip set with the clients running the sparc set.

The NFSv4 global namespace entry for /nfsv4/eisler.org/docs/notes might 
note
that it needs to look for a version of this file system that has the CPU 
attribute
set to CPU attribute defined on the client.

---

From: Mike Eisler <[email-masked]>
Date: Fri, 14 Apr 2006 11:44:28 -0700 (PDT)

> content is different, its not another instance of the data. I agree with 

Carl,

But the content is logically the same.

> Trond's follow up that this is more of a client driven capability, just 

Assuming we have consensus that this is worth doing something about,
how does the client drive it in absence of what I've proposed?

In other words, help a non-AFSer out here. How does AFS do this, and where
are the @<attributes> stored?

> operation to a server, or maybe some type of  named attribute, or maybe 

I understand the LOOKUP approach, but what it means is that the server has to
have all the permutations of all the content in order to satisfy the LOOKUP.
That isn't good economy.

> its just another FSID wide attribute. Something like an architecture type 

That's better economy.

> string. Of course, if its a server based property, there will need to be 

(My thought was that we have a registry: the NFSv4 global namespace; what
Dave Noveck has specified in fs_locagtions_info looks like a registry of
sorts to me).

Using the per FSID approach, another one way to handle this would be for the client
to do a GETATTR or OPENATTR/READ on each file system returned in fs_locations_info,
and mount the one that matches best. The trouble is, it would have to do
this for every referral, and not all referrals refer to content that might be
specific to one or more of a client's attributes. It seems desirable 
if fs_locations_info results can be filtered on the referring server.

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 14 Apr 2006 14:11:59 -0500

> theory that customers matter, I'm setting the priority to required.

I sympathize with this customer, very much because I had a hand in
building and maintaining an automounter-driven namespace that used
automounter variables heavily, and not only the ones that the
automounter defines by itself, but also customer-defined ones as well.

> set to CPU attribute defined on the client.

We've had a long discussion of this in this list in the past.

IMO, the set of variable/values that the various clients care about form
a set of, for lack of a better term, namespace roots.  The client should
use those variable/values to find the namespace root it wishes to mount
and mount it.

E.g., imagine a server that exports the following:

/export/SunOS-5.10-sparc-sun4u/...
/export/SunOS-5.10-sparc-sun4v/...
/export/SunOS-5.10-i386-i86pc/...
/export/SunOS-5.9-sparc-sun4u/...

i.e.,

/export/$OS-$OSREL-$CPU-$ARCH/

or maybe

/export/$OS-$OSREL-$NATISA/

Clients will generally want to mount only one of these paths: the one
that is relevant to them.

But there may be many filesystems in the namespace below those points,
some not on the same server or replicas.

I know of one customer that uses automounter maps and variables in this
way to great effect...

...But how does that map into NFSv4?  Well, using referrals and
client-side crossing of server-side mounts we can put together
server-side driven filesystem hierarchies (as opposed to client-side
implementation of automounter maps), and all that remains is for the
client to pick the right hierarchies to mount in the right places.

So, given a client which used to have an automounted view like:

/imp/generic/{bin, lib, ...}
/imp/SomeApplication/{bin, lib, data, ...}
/imp/SomeOtherApplication/{bin, lib, data, ...}
/legacypath/...

with 'data' directories being separate filesystems on file servers other
than the ones serving bins, for example (i.e., complex multi-
filesystem, multi-level hierarchies), and using all of those automounter
variables plus customer-defined variables as well, ...

...this client now needs to know how to mount the top of the /imp and
/legacypath namespaces from fs_locations that use referrals and/or
server-side mounts to build the entire namespaces, one for each set of
interpolated variables that the customer has.

I.e., instead of having automounter entries in the auto_imp and
auto_legacypath maps for each of those sub-directories, using
hierarchical automount entries and/or autofs mounts, all interpolating
various variables, the client now needs two symlinks of this form, but
with the variables' values substituted:

/imp -> /nfs/<client's domainname>/imp-$OS-$OSREL-$NATISA-$CUSTENV
/legacypath -> /nfs/<client's domainname>/legacypath-$OS-$OSREL-$NATISA-$CUSTENV

like

/imp -> /nfs/sun.com/imp-SunOS-5.10-sparcv9-engineering
/legacypath -> /nfs/sun.com/legacypath-SunOS-5.10-sparcv9-engineering

The root servers for the given domain would have to know how to refer
the client, and so on.

Nico
--