Subject: Re: [nfsv4] Named attributes and getxattr

From: Brent Callaghan <[email-masked]>
Date: Fri, 11 Nov 2005 17:54:53 -0800

On Thu, Nov 10, 2005 at 01:24:22PM -0600, David Robinson wrote:
>> Yup, xattrs are short strings, atomically set and read.  Subfiles are
>> infinitely long, read and written incrementally.  Representing
>> xattrs as subfiles would be an inefficient hack.  I think they need
>> distinct APIs, namespaces, and protocol support.
>>
>
> How important is the atomicity? While NFSv4 has almost no atomicity
> guarentees, would placing the contents in a READ/WRITE surrounded
> by a LOCK/LOCKU (see cur_stateid proposal) be good enough?

The Linux get/setxattr requires the attribute value to be read or
written in a single hit.  MacOSX Tiger has a similar API, though it
adds an offset argument that can be used to get/set attributes in
pieces if necessary.

>
> A well constructed COMPOUND should allow a get or set with
> one round trip, so I don't see much of an efficiency argument.

Since xattrs aren't commonly used, I'll have to take a guess at
what constitutes common usage and assume that apps or
file browsers will likely fetch and cache all the xattrs for a
file, rather than deal with them singly. Caching of xattrs
is also more likely if the files are delegated.

It's easy to write a compound to open a file and fetch all
of its attributes with GETATTR and even to fetch its data,
but named streams are more elusive.  Without prior knowledge
of the attribute names, you need to do an OPENATTR/READDIR
to collect their filehandles - then issue another compound to
collect the values.

The extra over-the-wire calls for each file would not be good
for performance.

	Brent

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: David Robinson <[email-masked]>
Date: Sat, 12 Nov 2005 12:30:19 -0600

> adds an offset argument that can be used to get/set attributes in

If it is allowable to get/set at an offset then it seems clear that
'atomicity' is very similar to the Unix requirement that a single
read() or write() reflects the data at a given point in time and
not interleaved with another write(). While the NFS READ and WRITE
have no such requirement, in reality most servers do make individual
ops atomic.

My fear would be that in order to maintain 'atomicity' we would have
to introduce locking on xattrs which would be a large burden,
especially if the contents of the attribute exceed the transfer size.

But based on your description of the APIs, a server simply needs to
not interleave one get/set with another. Since xattrs are 'small'
then transfer size issues will not be an issue.

> to collect their filehandles - then issue another compound to

Looking at the APIs that are being used, an application must
first call listxattr (or flistxattr or llistxattr) in order
to retrieve the available attributes before it can getxattr,
or it has to a priori know the name of the attribute. This is
exactly the model of readdir/stat. So in your hypothetical
browser it will have to do a listxattr followed by an iteration
over each doing a getxattr. A naive implementation would do
one OTW call for each (or a NFSv3 style protocol).

A smarter client would cache the returned list, and upon
receiving the first getxattr COMPOUND up retrieving
multiple xattrs in a single request.

One problem with trying to build a complex combined
listxattr/getxattr into an op like READDIR or GETATTR
(other than it goes against the architecture of COMPOUND) is
that typically listxattr wants a complete list in a single
syscall, and will error if the supplied buffer is not
big enough. If the list is long, or the attributes are modest
in size, multiple round trips may be needed to return the
all of the data due to transfer size limits.  This would
require us to add some sort of cookie to allow the GETATTR to
be restarted in the middle of the xattr list of the
attribute. We have now just required the cookie functionality of
READDIR to be built into GETATTR. Or worse, we add a secondary
READDIR into the embedded getattr of the existing READDIR.

The only reasonable fallback is to also provide a standalone
mechanism that allows a way to acquire the whole list and
each attribute to cover the case where one attribute approaches
the transfer size in length.

Once we have provided an OP to list all the xattrs and an OP
to retrieve each one at a time, I am less motivated to then
overload the existing GETATTR and READDIR to combine them. That
is what COMPOUND is for.

> The extra over-the-wire calls for each file would not be good

Smart usage of COMPOUND can minimize the number of OTW calls,
it could most definately not be O(N) and more likely O(1). Additionally
the application is going to be making multiple syscalls
so there is an opportunity for doing 1 OTW call per N syscalls
including get-ahead.  There is the opposite issue, if
we only allow a single combined list/get then an app that
has built in its own "user.myxattr" will pay a large server
burden for retrieving data it doesn't need.

My conclusion on this is that we need an OP to list the xattrs
and OPs to get and set each xattr. And we don't need any added locking
other than to say that the get or set OP SHOULD (MUST?) not
interleave with other xattr OPs on the same xattr.

I have not said anywhere here what the OPs should be, these
requirements appear independent of creating a new OP or
reusing an old OP.

	-David

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Sun, 13 Nov 2005 20:24:07 -0800

OS/2 style "weenie" extended attributes are a mismatch for
NFSv4 Named Attributes. "Weenie" EAs include Linux getxattr()
and BSD extattr_get() on "user.*" attributes.

The NFSv4 Named Attributes are subfiles.

Getxattr(), et al, manipulate bona-fide attributes (ignore user.*
for a moment). It is too easy to think of getxattr() as manipulating
opaque thingies. They are not. Linux uses getxattr() API and VFS OP
to manipulate Posix ACLs. NFSv4 treats ACLs as bona-fide attributes,
meaning that they have an attribute number and rigorously defined
marshalling. This will be the case for all attributes. Linux
NFSv4 clients will use getxattr() to manipulate ACLs.

I've only heard of one use of getxattr() user.* in Samba.
It's used to attach DOS names. Jeremy Allison can probably
clarify this requirement. I'm told there is no reason not to
expect Samba to use Solaris Extended Attributes for the same
purpose. Further, Jeremy would like to use the Solaris EAs to
support NT Streams. There are likely other uses and statement
of requirements would be helpful.

NFSv4 Named Thingies, what Solaris calls Extended Attributes
(a complete misnomer, they are subfiles), and NT Streams are
opaque, arbitrary-sized objects. The normal NFSv4 file locking
works fine on subfiles. There seems to be growing use of them
on NT.

I believe NetApp is also using NT Streams to support NFSv4
Named Attributes.

It would be odd to add another opaque-textually-named-thingy
set of operations to NFSv4. One is enough. Because it makes
no sense to transfer bona-fide attributes (like ACLs) through
an opaque-textually-named-thingy interface, I don't see that
requirements for another set of operations has been made.

The only motivation I can see for a second mechanism is that
Linux/BSD/Weenie attributes are a mismatch for NFSv4/Solaris/NetApp
Named Thingies (subfiles, really). There is a desire to support,
at least as a bandaid, the weenie user.* attributes so that the
door is left open to support NFSv4 Named Attributes with an appropriate
mechanism (much like Solaris) in the future. It would be, IMO,
and unfortunate and unnecessary pollution of the file model.
I've yet to hear or read a case that's more than sentimentality
for doing all that.

I was recently in a discussion on the NetBSD reflector about
whether extattr_get() (the BSD counterpart to getxattr()) should
be linked to NFSv4 Named Attributes. A strenuously argued against
it. The comment was the NFSv4 has a design flaw because it can't
do "both". I think that's quite wrong, but I thought I should
pass it along.

Regards,
	-gww

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 15 Nov 2005 15:34:02 -0600

> The extra over-the-wire calls for each file would not be good

That doesn't mean that NFSv4 EAs can't be used to implement xattrs now.
It only means that eventually we need an op targetted at reading [small]
attributes in one go.

Is the choice whther establish conventions atop of what NFSv4.0 offers,
or wait for NFSv4.2?

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 15 Nov 2005 15:42:30 -0600

Gordon makes a big point of the xattr-specific semantics typically
associated with system xattrs as opposed to the octet streams semantics
of NFSv4/NT/Solaris sub-files.

I'm not so sure.  Clearly the system cannot treat user xattrs as
anything other than octet streams, and as for system xattrs, as long as
there's a way to enforce convention that should be enough.

I'm inclined to think that we need a way to reserve NFSv4 named
attribute namespaces for system use, allow servers to enforce content
structure for known system attributes and reject writes to unknown
system attributes.

Nico
--

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Tue, 15 Nov 2005 15:12:24 -0800

> structure for known system attributes and reject writes to unknown

GETATTR/SETATTR. Enforcement is hand-in-glove with marshalling.
"User" xattrs haven't made a case yet. A rush to link user xattrs
to NFSv4 Named Attributes would create a bit of a mess. Better to
wait for bona-fide (ala Solaris/NT/NetApp) subfiles instead, and
I have reason to believe it won't be that long (less than two years).

getxattr()/setxattr() APIs should be present on all platforms
for manipulating bona-fide attributes (ACLs, Integrity labels,
and the litany of coming attributes). Such attributes should
be defined in NFSv4 which, for these reasons, was made extensible.

getxattr()/setxattr() should at least be done as library
interfaces rather than system calls. The idea is to make it
easier for applications and tar(1) variants to readily port
to all platforms.

My research and references here:
    http://nasconf.com/pres04/waidhofer.pdf

I learned since I did this paper that Solaris openat()
is not for "Open ATtributes". Rather, it was a preexisting
API for "AT this point in the name space". A flag was added
for Solaris subfiles (EAs). There is a problem with it that
has to do with attropen() needing to open()/openat()/close()
to return a fd for the attribute. The close() could cause
problems with POSIX locks.

Regards,
	-gww

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 15 Nov 2005 17:42:38 -0600

> I have reason to believe it won't be that long (less than two years).

Isn't that what NFSv4 already has?  Why wait?  Or have I misunderstood?

> problems with POSIX locks.

The client could be smart enough to not do an actual NFSv4 close if that
was important.

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Tue, 15 Nov 2005 16:05:54 -0800

> The client could be smart enough to not do an actual NFSv4 close if that

It's not an NFSv4 issue. Solaris has this problem. If locks
exist on file "x", attropen("x", "ea") will cause the locks
to be lost. Don't need NFSv4 in the mix to have this problem.
Jeremy Allison pointed this out, and Samba has workarounds for
this kind of problem. It's a real issue that will be addressed
by other systems an (moan) lead to divergence. I'm hoping the
Solaris folks come up with APIs that overcome this problem and
other platforms follow that lead.

Regards,
	-gww

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Tue, 15 Nov 2005 16:05:53 -0800

> Isn't that what NFSv4 already has?  Why wait?  Or have I misunderstood?

I mean on the Linux and BSD platforms, don't link xattrs to
NFSv4 Named Attributes. Wait for Linux and BSD platforms to have
subfiles and link those to NFSv4 Named Attributes.

getxattr()/setxattr() manipulate bona-fide attributes, and
will likely have translation: apps can query the POSIX-style
ACL even if the underlying file system supports NFSv4 ACLs,
and vice versa. Very much "views", and entirely consistent
with how ioctl() works.

It seems that NFSv4 GETATTR/SETATTR address atomicity,
encoding, and enforcement issues quite nicely.

The only loser in this scenario is "user" xattrs, which
have not made a case yet. OS/2 has a similar history.
Extended attributes were supported in HPFS to allow additional
attributes (metadata). But it was too tempting to expose
this to the user API. However, those "weenie" user EAs were found
to be inadequate very quickly. NT has legacy support for them
but everything moved to streams (subfiles).

Regards,
	-gww

---

From: David Robinson <[email-masked]>
Date: Tue, 15 Nov 2005 18:27:43 -0600

In reading Robert's e-mail, it seems very clear that
trying to wedge NFSv4 named attributes into "weenie" EAs
(love that term Gordon!) is a really bad idea. Going from
semantically rich to semantically poor is disaster waiting
to happen.

I think my perspective on things may be different than most.
I am working on a system that only has rich subfiles and
has no weenie EA interfaces at all.  You can't manipulate
ACLs using anything other than an ACL specific library,
so I have no desire to try and do that through NFSv4 named attrs.

 > I have particular concerns about what happens if applications begin
 > to rely on the availability of a file stream mechanism, such as is
 > present in NTFS and UDF, and try to use that facility, finding that
 > the backing store is a freebsd/linux/...-style EA mechanism.

So make no mistake about it, I don't want the arbitrary NFSv4
named attr space to get mapped to weenie EAs on a server. The
EA space should at most be a well defined subspace in NFSv4
so it is unambiguous which space it belongs to.

The "system" part of EAs can open an interesting set of cases.
Some system EAs may have no useful meaning across a network
and a server (which clearly understands them) may choose not
to expose them. Others may have informative meaning but
cannot (or should not) be set by a remote client, and others
may be full readable and writeable. I would think that
those choices are left to the server to make. Additionally,
the server has the full caller's secure credentials so it
can make choice on if the caller is authorized to make changes.

> NFSv4 clients will use getxattr() to manipulate ACLs.

Agreed, system EAs are not opaque. A Linux system must translate
getxattr on ACLs into NFSv4 ACL ops, it has no real choice. Likewise
a server may prohibit certain system attributes from being changed
remotely, that just makes sense.

> requirements for another set of operations has been made.

This paragraph got me lost. It seems that you are arguing that
the 'user.*' EAs which are opaque should use the same mechanism
(ignoring namespace issues) that we have in NFSv4 named attrs.

> I've yet to hear or read a case that's more than sentimentality

But this seems to say the opposite, you do want a second opaque
thingy set of operations.

> do "both". I think that's quite wrong, but I thought I should

Wrong that it is a design flaw, or wrong that it can't do "both"?

Focusing on the mechansm, and not the namespace issues, it is
still not clear to me the properties that are inadequate with
using READ/WRITE with the understanding that the server
would perform them as one operation, that would exist in a new
GET/SET op which would have the same one operation requirement.
Nor do I understand what parts of the permission model will
not work, the server has the full callers credentials using
either READ/WRITE or GET/SET to work from.

	-David

---

From: David Robinson <[email-masked]>
Date: Tue, 15 Nov 2005 18:37:09 -0600

> other platforms follow that lead.

Let's not confuse Solaris implementation with the protocol,
I (and now any OpenSolaris member) can change Solaris. This
sounds like a bug due to an internal open of "x" and the crappy
posix semantic that locks are lost on first close. Probably
just an unintended consequence bug that can be fixed.

	-David

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Tue, 15 Nov 2005 17:49:49 -0800

> just an unintended consequence bug that can be fixed.

Trying to converge APIs is beyond the scope of this email-list.
It's just a handy place for me to nudge.

Because of the experimental nature of Solaris EAs, I can see
why openat() was leveraged. I'm sure the POSIX-lock v. close()
problem was just an oversight.

There is a whole lot worked out, and quite well, on Solaris
for this feature. I'd hope, for the sake of convergence, that
others follow Solaris leads -- distinct ftypes for attribute
directories and attribute content, (corrected) APIs, etc, etc.
It'll make things easier for all concerned down the road.

Thanx for letting me nudge.

Regards,
	-gww

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Tue, 15 Nov 2005 18:03:35 -0800

I appologize for wording things poorly.
I'd just gotten home tired from a very long trip and
was a bit hasty to post. Let me clarify.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> David Robinson
> Sent: Tuesday, November 15, 2005 4:28 PM
> To: [email-masked]
> Subject: Re: [nfsv4] Named attributes and getxattr
> 
> 

I agree with all of that. I think the only change I would
make to that is to say "GETATTR-able and SETATTR-able"
rather than "readable and writeable".

> (ignoring namespace issues) that we have in NFSv4 named attrs.

I'm saying that "user" weenie EAs should be abandoned as
quickly as possible on all platforms. Applications using
them should move to subfiles. NFSv4 is in good shape for
the rest. No need (at least no case made) for doing otherwise.

> thingy set of operations.

The existing NFSv4 ops are just fine. I don't want a
second mechanism. I don't think it's helpful. Quite the
contrary (as Dave says above), it would make a mess.

There may be more attributes that need to be
defined -- attribute numbers and structures
in the protocol proper.

> Wrong that it is a design flaw, or wrong that it can't do "both"?

There is no design flaw in NFSv4. I think it is quite wrong to try
and have both weenie user EAs and subfiles on any platform.
There are energectic arguments to the contrary, but, IMO,
they amount to sentimentality.

I very much like the ioctl() feel of getxattr()/setxattr().
Those are wonderful for manipulating attributes.

That these interfaces (and the BSD counterparts) are used
to manipulate weenie user EAs is just an accident, and a
repetition of the history of OS/2. Weenie EAs were abandoned
there. They should be abandoned by Linux and BSD. Solaris never
exposed FSDs (used to store ACLs) as a user API. That was a
good move :)

"But weenie is so cute.... so harmless... so aesthetic....
how could it be a bad idea?" OK, that's an unfair paraphrasing
by me of the counter argument. I'll leave it to believers
in "both" to make their case.

> either READ/WRITE or GET/SET to work from.

I would think the existing GETATTR/SETATTR operations
in NFSv4 would suffice. Perhaps new attributes have
to be defined, but the operations are good to go.
Am I missing something?

Regards,
	-gww

---

From: David Robinson <[email-masked]>
Date: Tue, 15 Nov 2005 22:55:58 -0600

> was a bit hasty to post. Let me clarify.

If I understand you correctly, my summary would be:

     o getxattr/setxattr of "user.*" should be removed, as
       part of that attitude do nothing to enable their
       mapping to NFSv4 named attrs.

     o getxattr/setxattr of "system.*" attributes have
       such tight semantics, those of interest should be
       promoted to full NFSv4 attribute status. For example,
       ACLs have an obvious mapping today. Trying to map
       system attributes by name alone will likely hit nasty
       impedence mismatches, so something else is needed to
       dictate semantics.

     o real subfiles is the answer for user attributes, Solaris
       has a few warts but is a good start for an API.

Given that I haven't believed that doing things like ACL
manipulation through named attrs was ever a good thing, and
I work on an OS with subfiles and no weenie attributes.
I can agree to this. I had not anticipated anyone having the
desire to dump user attributes, but since the discussion
seems to revolve around "system" attributes I guess that is
what people care most about.

So to answer my original question: the getxattr/setxattr
style extended attributes SHOULD NOT be mapped to NFSv4
named attributes. Specifically "system" extended attributes
SHOULD be promoted to regular well define attributes through
minor versions as appropriate, and "user" extended attributes
SHOULD NOT be mapped into the protocol in any form.

Not sure if that will reach consensus, but it is a reasonable
position.

	-David

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 16 Nov 2005 00:17:40 -0600

> SHOULD NOT be mapped into the protocol in any form.

Having to bump the NFSv4 minor version number for every new system
extended attribute seems like a very tough barrier to overcome.

So, assuming that mapping system weenie EAs to NFSv4 named attrs is the
way to go then maybe we should use a different mechanism for negotiating
server support for them.

Nico
--

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 16 Nov 2005 01:00:59 -0600

> extended attribute seems like a very tough barrier to overcome.

So far, there have been no proposals to do so in the running time
of NFSv4.1.  Do we expect and onslaught of requests or are the
two communities unaware of each other to receive such requests.

I am unconvinced that we need to create a negotiation feature
at this point.

Spencer

---

From: Robert Watson <[email-masked]>
Date: Thu, 17 Nov 2005 15:58:32 +0000 (GMT)

On Tue, 15 Nov 2005, David Robinson wrote:

> getxattr on ACLs into NFSv4 ACL ops, it has no real choice. Likewise a 
> server may prohibit certain system attributes from being changed 

Just to capture the diverse set of implementation choices here a little 
better, I thought I'd mention that in FreeBSD, ACLs are manipulated using 
ACL library, system, and VFS calls, which impose syntax (although not 
semantics) on access control lists and access control list operations. 
We offer ACL VOP's that are not unlike the NFSv4 RPCs for ACL 
manipulation.  These library calls, system calls, and VOPs are entirely 
separate from the EA calls at the same layers.  The decision to map ACLs 
into system extended attributes is made in the UFS code on FreeBSD, 
permitting the ACL VOP's to be mapped at a kernel level to other backing 
mechanisms, such as ACL RPCs in NFS.  As such, in FreeBSD, system.* 
extended attributes are in fact opaque, used simply as a meta-data backing 
store for various services that map the raw data into objects with defined 
syntax and semantics.

In the end, it all comes to much the same thing, the layering just happens 
a little differently.

>> It would be odd to add another opaque-textually-named-thingy set of 
>> operations to NFSv4. One is enough. Because it makes no sense to 
>> transfer bona-fide attributes (like ACLs) through an 
>> opaque-textually-named-thingy interface, I don't see that requirements 
>> for another set of operations has been made.
>
> This paragraph got me lost. It seems that you are arguing that the 
> 'user.*' EAs which are opaque should use the same mechanism (ignoring 
> namespace issues) that we have in NFSv4 named attrs.

I think what Gordon is suggesting is something a lot like the FreeBSD VFS 
design decision for ACLs: in the end, some data and objects in a protocol 
or API is opaque, and some has imposed syntax and semantics.  Because NFS 
is a file system protocol, it makes sense that objects immediately 
relevant to the semantics of the file system be exposed as non-opaque in 
the protocol.  I.e., if ACLs are a defined part of the protocol, there is 
quite a bit of usefulness in exposing them explicitly and providing 
separate operations, rather than mapping all such things into one pair of 
RPCs (get, set).

However, I think Gordon is also arguing that there is no reason to have a 
generic meta-data service, as all relevant meta-data will be exposed in 
the file system [protocol].The reason we offer the EA APIs at several 
levels in the FreeBSD kernel is that sometimes there is a desire to have 
higher layers tag objects with meta-data that is maintained but not 
interpreted by lower layers.  So while the file system maps ACL operations 
into EA operations, and exposes ACL functions, higher level services, such 
as mandatory access control, perform that mapping themselves and then 
deliver EA operations to the file system.

We have support for at least four mandatory access control models in 
FreeBSD that use extended attributes -- Biba integrity, MLS 
confidentiality, and SEBSD (type enforcement), as well as a checksum 
verification policy for binary execution.  These plug in using our MAC 
Framework and are (almost entirely) implemented above the file system 
layer.  This is exactly the sort of service that would benefit from an 
NFSv4 attribute service providing an equivilent to the Linux/Mac OS 
X/IRIX/FreeBSD/HPFS extended attribute meta-data storage service.  In 
fact, in order to carry MAC labels in NFS, we have an immediate desire to 
have something like that.  And on looking at named attributes, we rapidly 
became concerned that although this was a useful way to expose a sub-file 
semantic, we actually wanted a meta-data semantic not a sub-file semantic 
for this purpose.

> Focusing on the mechansm, and not the namespace issues, it is still not 
> clear to me the properties that are inadequate with using READ/WRITE 
> with the understanding that the server would perform them as one 
> operation, that would exist in a new GET/SET op which would have the 
> same one operation requirement. Nor do I understand what parts of the 
> permission model will not work, the server has the full callers 
> credentials using either READ/WRITE or GET/SET to work from.

Looking at the BSD EA mechanism, there are a few specific pieces of 
behavioral semantic that we would want to reproduce, which can either be 
directly represented using the current NFSv4 RPCs, or can sort of be 
reproduced:

- Value can be undefined, or defined and 0 or more bytes.  I think this
   can be met by virtue of the fact that subfiles can be present but empty,
   in much the same way a normal file can be present but empty.

- When a write occurs, it results in replacement of the current value.
   This is similar in notion to an environmental variable -- writes are
   atomic replacement, since for meta-data, you usually want the old or new
   value, but not 30% of the old value and 70% of the new value.  This
   means that the attribute is truncated on write if the new value is
   shorter than the old value.  This can be simulated in NFSv4 using
   multiple operations in the same RPC, but it wasn't clear to me that this
   is guaranteed by anything in the protocol, or even common
   implementations.

- Name space and protection semantics.  In FreeBSD and IRIX, protection is
   based entirely on the name space of the attribute.  Name spaces are
   independent (the same name can exist or not exist in different name
   spaces with no interference).  The user name space, while largely unused
   as you observe, is protected using the mode bits or ACLs.  The system
   name space is reserved for system use by appropriately privileged
   subjects.  Historically, portability of "appropriate privilege" across
   distributed file systems has been poor, and I don't have much confidence
   it will improve with NFSv4, since those problems are in large part
   inherent to poor behavior of distributed account systems.  In Linux,
   protection of system name space attributes is a property of the service
   implementing the attribute, so is flexible.  While the name space for
   named attributes is flexible, the protection details are fairly messy,
   so I'm not sure how well simulation would work here.  Especially given
   that a user can potentially relabel an object even though they can't
   change the contents of the object.  Extensible security and distributed
   file systems have never mixed cleanly, since they tend to require strong
   model assumptions, extensible credentials, and various other messy
   things.

- FreeBSD provides a transaction mechanism for in-kernel consumers of
   extended attributes.  We did not expose this mechanism to user space as
   the semantics are quite strong, and we felt not all file systems
   implementing attribute services could provide this semantic, nor that we
   wanted to deal with a way of exposing the transaction service to the
   user process via an API due to the risks involved in defining such APIs.
   However, it's quite useful in the presence of multiple mandatory access
   control policies which require mutual atomicity in updating security
   labels.  This can be simulated using multiple requests in an RPC, but
   failure modes are poor, and races are fairly possible.  Atomic
   multi-label update may be something we have to give up for portable
   environments.

So count me as interested in a discussion of potentially adding more 
traditional UNIX "EA" services to NFSv4, since we have an immediate 
practical need for them, but also as having some reservations about doing 
that.

Robert N M Watson

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Thu, 17 Nov 2005 09:57:56 -0800

> semantic, we actually wanted a meta-data semantic not a sub-file semantic

Are the MAC labels interpretted and enforced above or below
the kernel interface? I'm not sure how "mandatory" is handled
by the library level. Are the MAC labels sensitive to endianess?

Regards,
	-gww

---

From: Robert Watson <[email-masked]>
Date: Thu, 17 Nov 2005 18:10:36 +0000 (GMT)

On Thu, 17 Nov 2005, Gordon Waidhofer wrote:

>> fact, in order to carry MAC labels in NFS, we have an immediate desire 
>> to have something like that.  And on looking at named attributes, we 
>> rapidly became concerned that although this was a useful way to expose 
>> a sub-file semantic, we actually wanted a meta-data semantic not a 
>> sub-file semantic for this purpose.
>
> Are the MAC labels interpretted and enforced above or below the kernel 
> interface? I'm not sure how "mandatory" is handled by the library level. 
> Are the MAC labels sensitive to endianess?

For files/directories, the protections are enforced by the kernel, 
although applications help to set and maintain the labels using a set of 
label manipulation system calls, which are translated into extended 
attribute operations by a combination of the MAC Framework and policy 
module implementation the policy.  Labels, as manipulated by applications, 
are actually text strings, so endian-independent.  As currently stored in 
local file systems, it depends on the policy, but for most policies they 
are stored in machine-local format.  In experimental work to push them 
over the network, they are in transported in the same string format used 
in user space.  In general, in FreeBSD, file systems don't participate 
directly in the enforcement of policies, as that occurs in the cross-file 
system VFS code in the kernel.

In our work to bring mandatory access control to Darwin, we found that we 
did need to perform some enforcement in the local file system code due to 
the presence of complex compound operations that were insufficiently 
visible in the file system independent code -- i.e., operations to search 
the disk catalog, set attributes on multiple objects, etc.  However, the 
checks in those cases are not done directly using the extended attributes, 
rather, a cache of the label hung off the vnode in internalized form (a 
format in which access decisions can be computed efficiently).

With NFSv3, and carrying over to NFSv4, we basically identified three sets 
of technical requirements to carry label data on the wire for files: (1) 
finding a way to transfer the label data, (2) having a way to address 
cache consistency for the label data in order to propagate changes to 
cached labels, and (3) a mechanism for atomic update, as read-modify-write 
operations on labels have atomicity requirements for the security check on 
a relabel event.  In this work, we did not look at requirements for 
transfering subject labels (credential extensions) on the wire.

Robert N M Watson