Subject: Re: [nfsv4] Response to Labeled NFS Document comments.

From: Nico Williams <[email-masked]>
Date: Wed, 6 Apr 2011 01:56:02 -0500

[resend... forgot to reply all.]

On Mon, Mar 28, 2011 at 10:59 AM, Dave Quigley <[email-masked]> wrote:
> On 3/28/2011 11:15 AM, Nico Williams wrote:
>> Regarding attribute change notification, it seems that volatile file
>> handles (i.e., marking the handle stale) seems almost like the best
>> approach here.  However, the way clients recover from stale
>> filehandles is not ideal (re-open by path).  In this case it ought to
>> be possible to add an operation by which to obtain a new filehandle
>> given a stale one.
>>
>> So that's my proposal: add a new operation for re-validating
>> filehandles.  Inputs: old (stale) filehandle.  Outputs: new
>> filehandle.
>>
>> This would also have the side-effect of enabling the use of
>> capabilities (in the traditional, non-Linux sense).
>
> I like this proposal quite a bit. I find it a lot better than re-open by
> path since there is no telling if the file at that path has changed since
> the handle was originally granted to us. Since this really is just a
> metadata refresh and not a complete reopen then having the file change out
> from under us is a bad thing. Would this be a proposal in and of itself?
> Would we incorporate it into one of the other proposals?

Right, it's a very natural extension.  Imagine an implementation whose
FHes consist of something like {{fsid, inode #, RPCSEC_GSS triple,
client host principal, label-of-file, capability, ...},
HMAC(server_secret_key, {fsid, inode #, ...)} such that a fileserver
can make sure that the given FH can only be used by the given client
host, with the same or stronger RPCSEC_GSS triples... and the server
can also detect label changes that the client's not aware of, and it
can make the client renew capabilities too (without the client having
to know anything about how, beyond using the FH refresh op.  Indeed,
the label-of-file item in the above FH construction is a sort of
capability.  The server has the fsid and inode number, so it can find
the file without having to have any path information, provided the
file still exists of course.

IMO this operation should be part of the core NFSv4.2.  However, I
don't mind it being in a separate RFC (if MAC is not made part of
4.2).  This op is applicable outside of MAC and needs to be kept where
implementors will notice even if they don't care about MAC.

Nico
--

---

From: Tom Haynes <[email-masked]>
Date: Wed, 06 Apr 2011 14:01:08 -0500

On 4/6/11 1:56 AM, Nico Williams wrote:
> On Mon, Mar 28, 2011 at 10:59 AM, Dave Quigley<[email-masked]>  wrote:
>>> approach here.

Has anyone implemented volatile file handles in their servers?

I don't think so. I believe the Solaris client can handle them but that
the Linux client cannot.

>>> However, the way clients recover from stale
> HMAC(server_secret_key, {fsid, inode #, ...)}

Dictating the FH format is something we have avoided up until now. I
would prefer we not start down that route.

I also do not see the need to embed redundant information into it.
I.e., the RPCSEC_GSS triple should come out of the RPC layer,
the client host principal should be found another way, and I believe
the server secret key should not be in there either.

And embedding state in the FH is a slippery-slope precedent.

The original problem statement is that we want to be able to inform
the client when a label change occurs on a file it has open. If the
file is closed, we have no need to inform the client.

NFSv4 already has a mechanism for the server to inform the client
that state has changed - i.e., it can use the call back procedures. So
there is no need to stale out the FH and no need for the client to try
to reopen it via the path.

The server detects a a label change. It doesn't need to know what the
client was utilizing, it just needs to know that the client is MAC aware
and wants to know a change has occurred. And it knows this because
the client accepted the registration of the CB procedure. (Which by
the way, solves another issue, how does the server know that the client
is MAC aware?)

It informs the client of the change and the client responds. Until the
client responds, it may find that some of its I/O is not processed the
way it expects.

As the server need not store the information for the client (see my
statement above), the server just informs the client of the change
and not what changed. The client would be responsible for making
a GETATTR call on the FH to find out the change. I.e., it would present
a new compound to the server with its RPCSEC_GSS triple and
other information. I'm assuming here that there might be a label
that the client is not authorized to know on a file, it is sufficient to
know that it is not authorized.

The client is then responsible for making any changes it needs to
based on the new attribute or lack of access.

>   such that a fileserver
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 6 Apr 2011 19:48:00 +0000

Unfortuntae that this SHOULD isn't a MUST:

4.3.  Client Recovery from Filehandle Expiration

   If possible, the client SHOULD recover from the receipt of an
   NFS4ERR_FHEXPIRED error.

I read Nico's suggestion as an internal implementation choice.

btw: I doesn't seem scalable to rely on callbacks to do label change
notification but we haven't even seen discussion of when and how
often that type of change is expected to occur and what the
notification timelines should be if it does.    So, it is hard to evaluate
which solution (if any) would do in solving the problem.

Spencer

From: [email-masked] [mailto:[email-masked]] On Behalf Of Tom Haynes
Sent: Wednesday, April 06, 2011 12:01 PM
To: Nico Williams
Cc: [email-masked]
Subject: Re: [nfsv4] Response to Labeled NFS Document comments.

On Mon, Mar 28, 2011 at 10:59 AM, Dave Quigley <[email-masked]><mailto:[email-masked]> wrote:

approach here.

given a stale one.

filehandle.

 such that a fileserver

Nico

--

nfsv4 mailing list

[email-masked]<mailto:[email-masked]>

---

From: Nico Williams <[email-masked]>
Date: Wed, 6 Apr 2011 15:12:09 -0500

> I don't think so. I believe the Solaris client can handle them but that

They're there for filesystems like FAT.  Perhaps one of the Windows
implementations has it.

Nico> Right, it's a very natural extension.  Imagine an implementation whose
Nico> FHes consist of something like {{fsid, inode #, RPCSEC_GSS triple,
Nico> client host principal, label-of-file, capability, ...},
Nico> HMAC(server_secret_key, {fsid, inode #, ...)}
> would prefer we not start down that route.

I was dictating that?  What part of "imagine an _implementation_ whose
FHes consist of ..." dictates anything?  :)

> the server secret key should not be in there either.

This would be much easier to explain in person... or on the phone.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 6 Apr 2011 15:19:26 -0500

On Wed, Apr 6, 2011 at 2:48 PM, Spencer Shepler <[email-masked]> wrote:
> I read Nico’s suggestion as an internal implementation choice.

Indeed :)

> btw: I doesn’t seem scalable to rely on callbacks to do label change
> notification but we haven’t even seen discussion of when and how

Filesystem object labels should change very infrequently.  Process
(subject) labels could change very frequently (SElinux) or not very
frequently at all (all other MAC systems).

However, object label changes will generally require immediate
revocation (whenever the new label does not dominate the old label,
which is to say: always for Smack, almost always in all other cases,
since declassification is a rare event compared to classification).
The only way I can think of at the moment to reliably achieve
synchronous object label updates in pNFS is via MDS->DS communication.

Using renewable capabilities has the benefit that there is no need to
callback the clients when an object's label changes: they'll find out
when they next try to access that object, and then they'll try to
refresh the capability for it, and if that fails, they'll give up.
This scales somewhat better than calling back all clients with
outstanding opens, which is what I think you were referring to.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 6 Apr 2011 15:28:39 -0500

> This would be much easier to explain in person... or on the phone.

Well, I'll try in e-mail.  If you look at the private FH layout I
posited again you'll see that it looked like:

{{fsid, inode #, RPCSEC_GSS triple, client host principal,
label-of-file, ...}, HMAC(server_secret_key, {fsid, inode #, ...)}

This is not unlike many HTTP cookie layouts.  It's what in some
circles is called a "capability", which used to mean something like
this prior to Linux naming its fine-grained privileges "capabilities".

I was discussing that in the context of RPCSEC_GSS, and, really, v_3_,
which has a compound context feature that multi-user clients _really_
should want to use.  This means that there will always be a client
host principal involved.  But OK, assume there isn't one, so remove
that from the FH layout, and it becomes a weaker cookie than it was
before -- no big deal, really, _as long as_ confidentiality protection
is used.

Note that there's NO state in that imagined FH layout.

Nico
--

---

From: Thomas Haynes <[email-masked]>
Date: Wed, 6 Apr 2011 16:27:20 -0500

> Note that there's NO state in that imagined FH layout.

If the RPCSEC_GSS triple and label-of-file can change, then
they are state.

Your proposal is to use a FH renew to update at least the label-of-file.

---

From: Nico Williams <[email-masked]>
Date: Wed, 6 Apr 2011 17:18:02 -0500

> If the RPCSEC_GSS triple and label-of-file can change, then

You can call it state, but this is actually desirable, very desirable.
 In the case of the RPCSEC_GSS triple it protects the capability from
misuse.  In the case of the file's label: we absolutely want the
client to go refresh its capabilityif the file's label changes.

> Your proposal is to use a FH renew to update at least the label-of-file.

Yes.  I consider that highly desirable.

(There's also the HMAC key to go update too.)