Subject: RE: Named attributes

From: "Khan, Saadia" <[email-masked]>
Date: Tue, 19 Feb 2002 11:40:15 -0800

If the file already exists and its a guarded create case, then the
server can just return EEXIST without touching the named attributes,
so the server will not remove the attributes. 

If O_TRUNC is set, then the file will be deleted along with the 
named attributes.

As for what the client should do in this case, i would think the
client should not delete any named attributes. 
What are other implementations doing?

thanks.
Saadia

-----Original Message-----
From: Dan Trufasiu [mailto:[email-masked]]
Sent: Monday, February 18, 2002 12:50 PM
To: [email-masked]
Subject: Named attributes

I don't know if this a question for the protocol or only for the
implementation.
If the client requests to CREATE a file and the file already exists, does
the client has to delete/replace/update the named attributes, or the old
named attributes will be removed by the CREATE process?

Thanks

Dan

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 19 Feb 2002 14:43:03 -0600

> If O_TRUNC is set, then the file will be deleted along with the 

Does everyone agree with this interpretation?  If so, I will need to
add this to the RFC update.

> client should not delete any named attributes. 

The client should do what is appropriate for its operating
environment.  If the client's environment expects that named
attributes will be removed in all create cases, then it should make
sure that this occurs.  If not, then it can rely on the server
behavior.

-- 
Spencer

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 19 Feb 2002 17:55:30 -0600

> add this to the RFC update.

As other have pointed out to me, this does seem opposite to the
regular attributes of a file when it is truncated.  The mode isn't
changed with O_TRUNC (unless setuid/setgid is set) or group isn't
reset as a result of O_TRUNC.  Should named attributes be removed as a
result of O_TRUNC.  Is this behavior exist in operating environments
today and if so, which ones?

Spencer

---

From: Neil Brown <[email-masked]>
Date: Thu, 21 Feb 2002 10:15:55 +1100 (EST)

> today and if so, which ones?

A real problem that I have with reasoning about named attributes is
that I don't have a clear model for what they are or how they are
used.
If were knew the *purpose* of named attributes, we could make a lot
more sense when talking about them.

So: what is the point of named attributes?

Ones that I know of are:

 ACLS:
     some filesystems store ACLs as extended attributes which are
     probably much the same as named attributes.
     Presumably if an NFS server were to expose ACLs as named
     attributes, it would want to do so READ-ONLY as there are
     separate mechanisms for modifying ACLs.
     Presumably ACLs would not be modified by TRUNCATE, except
     possibly setuid bits.

  Security Contexts:
     At a recent conference Ted Ts'o mentioned that some people want
     to store "security contexts" in extented attributes in Ext3.
     Presumably they are similar to ACLs and would have similar rules.

  Thumbnails:
     Desktop developers seem to like being able to store all sorts of
     ad hoc information with a file, such as a thumbnail or an icon,
     or MIME type.  For a thumbnail, this is a summary of the object,
     and can be regenerated if needed (i.e. the named attribute serves
     as a cache).  I don't know if they would treat MIME type as a
     regeneratable summary or not, but in either case, these sorts of
     things are very likely to change if the file changes, and
     certainly if the file is TRUNCated.
     You could argue that such things should be discarded whenever the
     file changes at all.

  What else is there?  I would really like to know.

It seems to me that attributes in general can be divided in to 3
classes:

Content:

    These are summaries of the content:
       size, thumbnail, mime_type

    if you have the content, you can deduce the attribute, but storing
    it might be a good optimisation.

Container:

    These say something about the file that stores the content.
      mode, owner, group, ACL, mtime, atime, ctime
      You probably want to backup these attributes, but they are
      probably of no interest when you send someone the file.

Storage:
    These say something about how the file is stored:
      blocks used, blocksize, nlink, ino

Clearly (I hope)
   Content attributes are directly affected by any change to the
     content.
   Container attributes either don't change (ACL), or change in a well
     defined and obvious way (mtime) when the content changes
   Storage attributes are read-only anyway.

So where do named attributes fit?
If we cannot give a clear model of where they fit, then I think the
server must have a default behaviour (based on underlying filesytem
semantics) and the client should be given the choice of finer control.

NeilBrown -- hoping someone will explain named attributes to me.

---

From: Guy Harris <[email-masked]>
Date: Wed, 20 Feb 2002 16:20:23 -0800

>      You could argue that such things should be discarded whenever the

You could also argue that discarding them is the job of the application
changing the file, not the job of the file system....

---

From: "Dan Trufasiu" <[email-masked]>
Date: Wed, 20 Feb 2002 20:02:47 -0500

I consider that the job of discarding the extended attributes is a client
job. If is the job of the application or the file system (on the client)
this depends of the awareness of the file system concerning the existence of
the extended attributes. In the PC case the file system receives the request
at the CreateFile to create some "extended attributes" and for this reason I
consider that for the PC the file system is responsible for processing them.

Dan

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Guy Harris
> Sent: Wednesday, February 20, 2002 7:20 PM
> To: Neil Brown
> Cc: [email-masked]; [email-masked]
> Subject: Re: Named attributes
>
>

---

From: Guy Harris <[email-masked]>
Date: Wed, 20 Feb 2002 17:00:00 -0800

> consider that for the PC the file system is responsible for processing them.

So "the file system" here means "the NFS client code" rather than "the
NFS server code and file system on the server"?

(Maybe I have old documentation, or maybe you're thinking of an OS/2
"DosCreateFile()" or whatever rather than a Win32 "CreateFile()" call,
but I don't see any extended attributes argument to the Win32
"CreateFile()" call.  There's an argument for the security descriptor,
but is that used if the file already exists, or is it only used if the
file is being created?)

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 20 Feb 2002 19:05:11 -0600

> more sense when talking about them.

Okay.  I thought I had placed this in the RFC in a concise way but it
appears that I have failed to do this.  From the introduction (section
1.1.3.2):

"Named attributes are meant to be used by client applications as a
method to associate application specific data with a regular file or
directory."

Section 5 makes mention of this again:

"Named attributes are intended for data needed by applications rather
than by an NFS client implementation.  NFS implementors are strongly
encouraged to define their new attributes as recommended attributes by
bringing them to the IETF standards-track process."

Even though these descriptions are not included in section 5.3 (where
they should be), it is clear from this that the named attribute space
is supposed to be for application use.  The NFS client implementation
shouldn't go around placing its own info into the named attribute name
space because it wants to store implementation dependent data.

One of the reasons for this was that we want to avoid NFSv4
implementations that do not interoperate.  If we don't encourage
implementors to define semantics of additional attributes within the
IETF process and have a minor version of the protocol that includes
those newly defined numbered attributes, we will end up in a situation
similar to where we are today with the various platform specific ACL
definitions.

>      Presumably ACLs would not be modified by TRUNCATE, except

Based on the current protocol definition, the server should not expose
this via named attributes.

>      Presumably they are similar to ACLs and would have similar rules.

Right. Don't expose.  Define them as numbered attributes in the NFSv4
protocol and allow all to play equally.

>      You could argue that such things should be discarded whenever the

As Guy pointed out, that opposite argument can be presented and it
would be one that I agree with. :-)

>   What else is there?  I would really like to know.

It is up to the application.

> semantics) and the client should be given the choice of finer control.

Right. The server should not take any action on create/truncate for
named attributes and it is up to the client/application to effect
change.

-- 
Spencer

---

From: "Dan Trufasiu" <[email-masked]>
Date: Wed, 20 Feb 2002 20:24:51 -0500

Yes, I'm talking about the NFS client code.

About the CreateFile I'm talking about "hTemplateFile" parameter. I didn't
have the intention to present the full procedure only to say that the
application can call the file system to set the extended attributes.

Dan

> From: Guy Harris [mailto:[email-masked]]
> Sent: Wednesday, February 20, 2002 8:00 PM
> To: Dan Trufasiu
> Cc: 'Neil Brown'; [email-masked]; [email-masked]

---

From: Guy Harris <[email-masked]>
Date: Wed, 20 Feb 2002 17:20:01 -0800

> Yes, I'm talking about the NFS client code.

So the NFSv4 protocol doesn't need to specify that when truncating an
existing file in a "create" operation, we do anything with extended
attributes.

> application can call the file system to set the extended attributes.

The documentation I have says the file "supplies extended attributes for
the file being created", which sounds as if the extended attributes are
set only if the file is being newly created, not if the file already
exists and is being opened and truncated^H^H^H^H^H^H^H^H^Hoverwritten.

---

From: "Dan Trufasiu" <[email-masked]>
Date: Wed, 20 Feb 2002 22:05:48 -0500

I consider that the server doesn't have to do anything. The protocol can
specify it if we want to save some interoperability issues in the future.

Concerning the CreateFile API , I was talking about the case CREATE_ALWAYS.

Dan

-----Original Message-----
Sent: Wednesday, February 20, 2002 8:20 PM

---

From: Erez Zadok <[email-masked]>
Date: Thu, 21 Feb 2002 13:06:13 -0500

In message <[email-masked]>, Spencer Shepler writes:
[...]
> similar to where we are today with the various platform specific ACL

Spencer, I agree w/ your analysis.  We shouldn't impose a meaning on these
attributes.  But I also think that Neil brings up a good point.  By the very
fact that we've NOT imposed meaning, other people will.  We've given people
a powerful tool to define arbitrary attributes to files.  My own experience
w/ extensible file systems taught me that if you give people something
powerful, they'll find ways to use it that even the creator didn't think
about.

My personal expectation is that the first community to jump all over named
attributes would be the security folks.  They've always (correctly) griped
about the lack of security facilities in most file systems, and that ACLs
are limited and non-interoperable.  I'd expect security people to find named
attributes a heaven of new possibilities.

But in such a case, they would really like to have the option to truncate a
file and all of its (security-related, or every) named attributes in one
"atomic" operation.  If the current protocol requires a client to exchange
many operations to truncate a file and delete its attributes, that'll open
the door to race conditions and other attacks among multiple clients.

Does the COMPOUND facility allow the composition of an atomic truncate file
+ delete all named attributes?  I don't think it does: as soon as one op in
the compound fails, I believe the whole compound stops, right?

Would it then make sense to add a flag/bit to the truncate command that says
"also delete all named attributes?"  It seems to me a reasonable choice that
won't involve having to impose semantics on named attributes.

Erez.

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 21 Feb 2002 12:36:17 -0600

> powerful, they'll find ways to use it that even the creator didn't think

Right.  It is a difficult task to find the correct balance and my hope
is that we have made reasonable choices and the NFSv4 community can
work effectively within the IETF to effect future change in the form
of NFSv4 minor versions to address the need for change.

> attributes a heaven of new possibilities.

That seems reasonable and again I hope that we add these potential new
features via minor versions.

> won't involve having to impose semantics on named attributes.

This would be reasonable but I am hesitant to drive additional changes
to the current definition without a stronger statement of future
usage.  My preference would be to define this behavior in a future
minor version at this point.

-- 
Spencer

---

From: Erez Zadok <[email-masked]>
Date: Thu, 21 Feb 2002 13:49:32 -0500

[...]
> minor version at this point.

That's reasonable.  After all, the security community hasn't yet told us
they want this ability.

Spenser, do you keep a list of potential issues to re-visit/consider come
time for us to discuss v4.1.  I think it'll be useful to keep such a list
online.  It could help to minimize re-discovering issues/problems.  For
example, we can say about this truncate that we acknowledge it to be a
possible deficiency in the .0 protocol, and it'll be considered for future
revisions.

Such a list could also help implementers of current protocols: they won't
have to try and figure out how to deal w/ truncate and attributes if it's
already mentioned somewhere as a known "issue".

> Spencer

On a somewhat related issue to security, have we had the protocol looked at
by verification experts, or passed through one of those formal verification
tools?  While I'm not a protocol verification expert, I understand that
there are some nice tools out there that could catch problems we might have
missed.  (Come to think of it, this should be something the IETF should do
for all protocol work in progress.)

Thanks,
Erez.

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 21 Feb 2002 14:37:32 -0600

> possible deficiency in the .0 protocol, and it'll be considered for future

I haven't to this point but it is a good suggestion and one that I
will implement and place on www.nfsv4.org.

> already mentioned somewhere as a known "issue".

For this issue specifically, I believe the RFC should call out the
expected behavior but the issue of potential future enhancement will
be listed outside of the RFC.

> for all protocol work in progress.)

No effort like this has taken place.  In general, this would fall to
the RPC layer and the use of RPCSEC_GSS since NFS relies on this
mechanism for enabling security.

-- 
Spencer

---

From: Theodore Tso <[email-masked]>
Date: Fri, 22 Feb 2002 15:59:04 -0500

> attributes a heaven of new possibilities.

Whoa.  Wait a minute.  We need to be a bit careful here.

When I talked about extended attributes in Linux's ext2/3 filesystem
at the conference which Neil mentioned, I was very careful to
distinguish between "system" and "user" attributes.  Security folks
who are looking for various new features in filesystems, whether they
be discretionary access controls (i.e., ACL's, which need to be stored
attached to the file somehow), or mandatory access controls (i.e.,
compartment information, which again need to be stored attached to the
file somehow).  But, these are quite different from user-visible
attributes!   

Security-related attributes may not be visible to userspace, and
certainly who has access-control rights over them is quite different
from user-visible attributes.  (For example, the whole point of
mandatory access controls is that even the owner of the file isn't
allowed to modify the operations of MAC restrictions.)  And even in
the case of discretionary access controls, the question of who is
allowed to see the ACL may be different from who is allowed to view
other meta-data attributes of the file.

Hence, it's probably a good idea to assume that whatever use named
attributes might have in NFSv4, they probably *won't* be useful for
security folks, precisely because general purpose attributes are not
sufficient for what they need.

For local, disk-based filesystems, there needs to be a distinction
made between system attributes which are visible only to the kernel
--- for read and write accesses to these system attributes will be
mediated by security-specific system calls that will be enforcing
their own access control decisions --- and generic, user-visible named
attributes.  (Indeed, in XFS, they actually have entirely separate
namespaces for user and system named attributes.)  For a remote access
filesystem such as NFSv4, I don't think there's any need for system
attributes, however.

As for user-visible named attributes, what I normally tell people is
that they should avoid them at all costs, because they invariably lead
to immense headaches.  The best example of that is MacOS binaries,
which can't be FTP'ed without becoming useless, since the FTP protocol
doesn't understand about resource forks.  Sure, you can use binhex,
but that becomes a major pain in the ass.  There are many other tools
that don't understand about named attributes --- including tar, cp,
zip, etc.  

I can personally only see two good uses for named attributes.  One is
to cache information that can be obtained in other ways, as a speed
hack for GUI desktop managers, for example.  And the other is for
backup or hierarchical storage management systems, where the whole
point is that you don't want to attributes to move around when you
copy a file from one directory to another, or when you edit the file
using emacs, or whatever.

> won't involve having to impose semantics on named attributes.

It's important to have rules about how to treat named attributes.
However, the moment you start doing so, the named attributes will be
good for some things, and less good for others, so in some sense
you've already gone down the steep, slippery slope of assigning
semantics to said feature.  Part of the problem here is that people
who want named attributes often don't bother defining precisely what
it is they want, combined with an assumption that it's possible to
make a generic design that will suit all uses identically.

						- Ted