Subject: [nfsv4] Proposed language for NFSv4 minor version 2 draft

From: Chuck Lever <[email-masked]>
Date: Wed, 23 May 2012 16:47:06 -0400

Hi-

Proposed for section 7.7, "Security Considerations:"

"Security attributes are sensitive information.  As such, they should be protected against unwanted alteration while in transit between a client and server.  The authors strongly recommend that labeled NFS deployments utilize a transport mechanism that supports at least integrity protection to prevent man-in-the-middle attacks.  Likewise, it is appropriate to protect client requests that probe a server's level of LNFS support.

"A CB_ATTR_CHANGED operation does not carry security attribute content.  However, a missing CB_ATTR_CHANGED operation may cause a client to continue permitting access that has been revoked on the server."

Are there other windows during which a malicious proxy could replace or inject information that would compromise MAC security?  Any other network-based attack vectors worth mentioning here?  Any philosophizing about protecting the trust relationship between client and server?

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: "Myklebust, Trond" <[email-masked]>
Date: Wed, 23 May 2012 21:09:18 +0000

> Are there other windows during which a malicious proxy could replace or inject information that would compromise MAC security?  Any other network-based attack vectors worth mentioning here?  Any ph…

Just out of curiosity, when should one revalidate labels?

Are we going to assume a close-to-open model, where the client is forced
to do a label revalidation dance on open(), but can then assume that the
label stays unchanged until close()?

What about other cases? Do we just poll at regular intervals and/or when
the server sends CB_ATTR_CHANGED, or are there any special operations
that need stronger security label validation semantics?

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: David Quigley <[email-masked]>
Date: Wed, 23 May 2012 17:40:59 -0400

On 05/23/2012 17:09, Myklebust, Trond wrote:
>> "Security attributes are sensitive information.  As such, they 
>> should be protected against unwanted alteration while in transit 
>> between a client and server.  The authors strongly recommend that 
>> labeled NFS deployments utilize a transport mechanism that supports at 
>> least integrity protection to prevent man-in-the-middle attacks.  
>> Likewise, it is appropriate to protect client requests that probe a 
>> server's level of LNFS support.
>>
>> "A CB_ATTR_CHANGED operation does not carry security attribute 
>> content.  However, a missing CB_ATTR_CHANGED operation may cause a 
>> client to continue permitting access that has been revoked on the 
>> server."
>>
>> Are there other windows during which a malicious proxy could replace 
>> or inject information that would compromise MAC security?  Any other 
>> network-based attack vectors worth mentioning here?  Any 
>> philosophizing about protecting the trust relationship between client 
>> and server?
>

I don't think we can assume that labels won't change after open. In the 
SELinux world we don't make that assumption which is why we check 
permissions on read and write more than just on the open call. We have a 
mechanism that checks for label change or policy cache invalidation and 
unless either of those happened it will go with the cached answer for 
read/write permissions (effectively only checking it the first 
read/write). The way we've been handling it in the code currently is 
that when the local cache gets invalidated in NFS we request the 
security attribute as part of the new file attributes we are getting for 
the cache. On my Fedora system the default for this seems to be every 5 
seconds. We could have a situation where this is too long which is what 
CB_ATTR_CHANGED should handle.

Dave

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 23 May 2012 14:46:26 -0700

On Wed, May 23, 2012 at 05:40:59PM -0400, David Quigley wrote:
> >>between a client and server.  The authors strongly recommend
> >>that labeled NFS deployments utilize a transport mechanism that
> >>supports at least integrity protection to prevent
> >>man-in-the-middle attacks.  Likewise, it is appropriate to
> >>protect client requests that probe a server's level of LNFS
> >>support.
> >>
> >>content.  However, a missing CB_ATTR_CHANGED operation may cause
> >>a client to continue permitting access that has been revoked on
> >>the server."
> >>
> >>Are there other windows during which a malicious proxy could
> >>replace or inject information that would compromise MAC
> >>security?  Any other network-based attack vectors worth
> >>mentioning here?  Any philosophizing about protecting the trust
> >>relationship between client and server?
> >
> I don't think we can assume that labels won't change after open. In
> the SELinux world we don't make that assumption which is why we
> check permissions on read and write more than just on the open call.
> We have a mechanism that checks for label change or policy cache
> invalidation and unless either of those happened it will go with the
> cached answer for read/write permissions (effectively only checking
> it the first read/write). The way we've been handling it in the code
> currently is that when the local cache gets invalidated in NFS we
> request the security attribute as part of the new file attributes we
> are getting for the cache. On my Fedora system the default for this
> seems to be every 5 seconds. We could have a situation where this is
> too long which is what CB_ATTR_CHANGED should handle.
> 

What about the case where the client has a delegation and is not
sending IO to the server? I.e., all of the IO is local until the close?

-- 
[email-masked], ex-cfb

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 23 May 2012 14:48:09 -0700

> "Security attributes are sensitive information.  As such, they should be protected against unwanted alteration while in transit between a client and server.  The authors strongly recommend that lab…

Nit: It is strongly ...

> Are there other windows during which a malicious proxy could replace or inject information that would compromise MAC security?  Any other network-based attack vectors worth mentioning here?  Any ph…

And by the way, thanks for the writeup.

---

From: David Quigley <[email-masked]>
Date: Wed, 23 May 2012 17:53:56 -0400

> sending IO to the server? I.e., all of the IO is local until the 

I'm trying to remember what we said when we originally wrote the 
requirements document. So in the case of when the client has a 
delegation on the file I think its safe to assume that the client at 
that point is the sole arbiter of what the label should be. Until the 
clients gets a request to return the delegation it will be responsible 
for maintaining what the label should be. This shouldn't be a problem as 
any action which would cause the label to change out from under it would 
require the delegation to be revoked. Changing the label should require 
an open on the file with write permissions requested which should revoke 
any read/write delegation on the file. Atleast this is my understanding 
of how things are supposed to work. Feel free to correct me if I am 
wrong with this understanding. If no revocation of the delegation is 
requested then flushing on close should send the label and all relevant 
changes back to the server.

Dave

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 23 May 2012 14:58:41 -0700

On Wed, May 23, 2012 at 05:53:56PM -0400, David Quigley wrote:
> that point is the sole arbiter of what the label should be. Until
> the clients gets a request to return the delegation it will be
> responsible for maintaining what the label should be. This shouldn't
> be a problem as any action which would cause the label to change out
> from under it would require the delegation to be revoked. Changing
> the label should require an open on the file with write permissions
> requested which should revoke any read/write delegation on the file.
> Atleast this is my understanding of how things are supposed to work.
> Feel free to correct me if I am wrong with this understanding. If no
> revocation of the delegation is requested then flushing on close
> should send the label and all relevant changes back to the server.
> 
> Dave
> 

My point is simply that we do have some scenarios which do not
map back to the traditional implementations.

And it turns out this is exactly the scenario we are discussing
in the other thread. Both an open and a directory delegation are
currently the only reasons we need to use the CB.

But back to Chuck and Trond's questions, I think we are covered?

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 24 May 2012 12:17:24 +0000

> CB_ATTR_CHANGED should handle.

Polling every open file and in-use directory every 5 seconds for
something that doesn't really change that often is going to be _very_
hard on the server. In the real world, security policies should not be
changing every 5 seconds.

Can we therefore look into improving CB_ATTR_CHANGED, perhaps by
allowing the client to subscribe to it for specific files/directories.
and allowing it to specify exactly which attributes it cares to be
notified about?

NetApp

---

From: David Quigley <[email-masked]>
Date: Thu, 24 May 2012 09:27:08 -0400

On 05/24/2012 08:17, Myklebust, Trond wrote:
>> the

We don't really actively poll for it. Looking at the default NFS 
settings the local attribute cache gets refreshed every 5 seconds. I 
don't know if this is something that people tune or not but with my 
default kernel and NFS utils configs it seems to happen every 5 seconds. 
I may be confused and this might just be on the lookup path and not on 
the open path but that is what I've been seeing. Actually that probably 
is just the lookup path now that I think about it. What is the current 
behavior for the attribute cache for open files?

Dave

---

From: "Haynes, Tom" <[email-masked]>
Date: Thu, 24 May 2012 08:02:47 -0700

> changing every 5 seconds.

Minor implementation detail. :->

> and allowing it to specify exactly which attributes it cares to be

Ohh, a very good suggestion!

What I like about this is that it solves the issue of the MAC model
wanting to monitor objects other than those with state.

Like other limited server resources, the client should be prepared
for the server to return NFS4ERR_RESOURCE? Except that isn't valid
in NFSv4.1. :->

NFS4ERR_CB_LABEL_DENIED

If we take Peter's suggestion to make this all about labels:

Operation 68: LABEL_WATCH

struct LABEL_WATCH4args {
        /* CURRENT_FH: object to watch */
};

struct LABEL_WATCH4res {
        nfsstat4        lwr_status;
};

If, as Trond suggests above, we let the client inform the server
about what to watch:

Operation 68: ATTR_WATCH

struct ATTR_WATCH4args {
        /* CURRENT_FH: object to watch */
	bitmap4		awa_critical;
	bitmap4		awa_info;
};

struct ATTR_WATCH4res {
        nfsstat4        awr_status;
};

This solves the problem of how we mark something as critical versus info,
the client sets it.

It raises the question as to what if the server doesn't agree with one
of those or is unprepared to monitor some of the attributes.

struct ATTR_WATCH4res {
        nfsstat4        awr_status;
	bitmap4		awr_critical;
	bitmap4		awr_info;
};

If the bit is set in the response, then the server will notify the
client upon a change?

---

From: <[email-masked]>
Date: Thu, 24 May 2012 12:31:26 -0400

> But back to Chuck and Trond's questions, I think we are covered?

Sort of.  "Close enough for government work," in a manner of speaking.

I think you are headed in the right direction but we need to fix up
some details as regards directory delegation.

If you have a directory delegation, then it is recalled if any directory attributes change.

So there are two issues to address:

1) you might prefer a synchronous notification of a label change to a recall.
2) You might not want to have a recall for many sorts of attribute changes.
   Right now you can specify attributes for which you get asynch notification but 
   you can't give a mask about those you are not interested in hearing about at all.

I think these can be addressed but some accusations of "scope creep" may be inevitable.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, May 23, 2012 5:59 PM
To: David Quigley
Cc: [email-masked]
Subject: Re: [nfsv4] Proposed language for NFSv4 minor version 2 draft

-- 
[email-masked], ex-cfb
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 24 May 2012 13:01:23 -0400

> I think these can be addressed but some accusations of "scope creep" may be inevitable.

Well, and also we would need something that handles objects other than
files and directories.

--b.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 24 May 2012 19:05:56 +0000

> of those or is unprepared to monitor some of the attributes.

Since this is basically a request to set up state on the server, you'd
want to add a stateid. Also an ATTR_CANCEL_WATCH operation...

> If the bit is set in the response, then the server will notify the

Yes. The client can then _rely_ on CB_ATTR_CHANGED because it will have
received an actual promise by the server. In addition to security
labels, I can imagine wanting to set notifications for ACL changes, mode
changes, and owner/group_owner changes on non-regular files in order to
avoid polling ACCESS requests (since those attributes are unlikely to
change often).

We would also need to add something to CB_RECALL_ANY to allow the server
to limit the number of outstanding watches if the client tries to abuse
the system.

NetApp

---

From: "J. Bruce Fields" <[email-masked]>
Date: Wed, 30 May 2012 13:16:38 -0400

> to limit the number of outstanding watches if the client tries to abuse

That makes sense to me.

But this is starting to sound like a bigger change for this point in the
process.

And I still don't understand exactly what the labelled NFS folks *need*:

	- Are these callbacks required, or just nice to have?  (Is it OK
	  if a client doesn't notice a label change until its attribute
	  cache times out, whenever that may be?)
	- Why do "open files" get special mention in the requirements
	  document?  Is timely notification less important for regular
	  files that aren't open, or for objects other than regular
	  files?

If we leave 4.2 as it is, would it be good enough to have some use as it
is, or are the incomplete notifications a show stopper?

--b.

---

From: David Quigley <[email-masked]>
Date: Wed, 30 May 2012 13:26:46 -0400

> is, or are the incomplete notifications a show stopper?

In our current implementation we have settled on it being ok if we 
don't get the label notification until the cached attribute times out. 
We don't see it being that big of an issue. If we want to provide a 
callback mechanism that is great because it provides less of a window 
where the label is consistent. As it stands right now its not a hard 
requirement for it being usable just icing on the cake.

For opened/vs non opened files. I think that the wording is just a bit 
weird there. Files which we have label information for due to a lookup 
but aren't open are important as well. The problem is though that it 
will be hard to notify all clients of any given change to a file label. 
I think we settled just on open files for an efficiency standpoint. 
However I can see a situation where we lookup a and get its label. We 
make the local policy decision based on that label but it has changed on 
the server. If the server isn't enforcing a policy it will still allow 
us to go through even though the new label may have been denied. I 
believe this falls under the acceptable window comment above dealing 
with the cached attribute timeout.

Dave