Subject: [nfsv4] Delegations and CB_GETATTR avoidance

From: Spencer Shepler <[email-masked]>
Date: Fri, 14 Mar 2003 03:05:02 -0600

Another interesting topic related to delegation implementation is how
a server might avoid doing CB_GETATTRs for a delegated file in the
event that the client generating the "conflict" actually holds the
delegation.  One easy example is with the sequence:
	PUTFH, OPEN, GETATTR

If the server builds the state for the delegation on processing of the
OPEN and then finds the conflict on the GETATTR, the server may
generate a CB_GETATTR to the client even though the client has not
been made aware of the delegation or even given the filehandle.
Obviously, the server can do some simple processing to avoid the
CB_GETATTR in this case.  There are other simple examples of things
like: PUTFH,READ,GETATTR or PUTFH,WRITE,GETATTR.  The server can once
again do simple examination to avoid the CB_GETATTR.  In the least the
server should employ these tactics to reduce overhead.

One interesting case where the simple optimization doesn't work is
with READDIR.  If the client generates a READDIR and asks for
attributes on the results, it may be that a file contained with the
directory has a delegation and the server will generate a CB_GETATTR.
READDIR is not a state operation so the opportunity simple
optimization doesn't exist.  The suggestion has been made that the
client could build a READDIR request something like: RENEW, PUTFH,
READDIR This way the server could use RENEW to trigger the CB_GETATTR
avoidance.  The Solaris client and server do not currently use this
method but it is being considered.  Any opinions about something like
this and are there other interesting scenarios where the client and
server can cooperate to reduced the need for CB_GETATTR?

-- 
Spencer

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 14 Mar 2003 09:48:19 -0800

I'm glad you brought this up.  My impression is that there is
a very common condition in which you can avoid CB_GETATTR
safely.  I had thought that the spec made a mention of this but
I've looked and the spec doesn't have any indication of this 
and implies otherwise.

Anyway here is what I was thinking.  Let us divide write delegations
into two classes.  Let us call them flush-always and flush-
sometimes delegations.  The difference is that the flush-always 
delegation, by setting the space conditions associated with the
delegation assures that all written data be flushed at CLOSE
time.  You basically tell the client you cannot make any
comittment that there will be space to write even one byte
of the file so that he has to do the write at close time.
For flush-sometimes deleggations, you allow him to avoid this
in some cases, based on how much is written.  This is obviously
better for the client but it is a lot of work for the server.

So I'm going to argue, that in the case of a flush-always write 
delegation, not doing the CB_GETATTR is safe.  If the size or
the modified time has changed (i.e. data has been written by
the delegated client and not pushed to the server), then we know
that the cleint must still have the file open.  But in that case,
the client, even without a delegation would have been able to 
retain data until the CLOSE.  So that means that if you don't
do the CB_GETATTR on a flush-always write delegation, you get
an answer which is as up-to-date as you would have gotten had 
there been no delegation, i.e. one which has always been acceptable
to NFS client.  Doing the CB_GETATTR in this case would get you 
a more up-to-date answer in the dleegation case, and there is 
nothing wrong with being more up-to-date, but it doesn't seem
like you need to do that.

That being said, the spec doesn't clearly indicate the reason that
CB_GETATTR is required, or rather define the constraints to allow 
you to clearly determine when it is and isn't required.  At the
very least we should clarify this for 4.1.

You raise some interesting issues about CB_GETATTR.  I think they
might point to the need for a small protocol extension for v4.1,
but I'm not positive.  I need to think about this some more.

-----Original Message-----
From: Spencer Shepler [mailto:[email-masked]]
Sent: Friday, March 14, 2003 4:05 AM
To: [email-masked]
Subject: [nfsv4] Delegations and CB_GETATTR avoidance

-- 
Spencer

---

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 17 Mar 2003 12:19:00 -0800

> CB_GETATTR in this case.  

And he should since there's no way the client could have anything 
interesting to tell him.

> There are other simple examples of things

You're making the assumption that in processing the GETATTR, you can 
take note of a stateid from a previous op in the same request and 
use that to infer the client making the request.  I think that's
reasonable but the spec doesn't really say that, does it?  It 
probably should though.  The other alternative is to say that I
blew it in not putting a stateid in GETATTR (there is one in SETATTR)
given your other example, the problem is more general.

The other thing I'd say about these examples is that the client should
not be doing these getattr's, at least for the attributes in question.  
If he has a write delegation, then he is the authoritative source of 
information about this stuff, and the server will either waste his time
using CB_GETATTR to get the information the client already had, or it
will be smart and not do that, in which it was pointless to ask for 
this infomration since it reflects only writes that the client itself
has done.  As a server implementer, I'd prefer it if I was allowed
to return NFS4ERR_WHY_ARE_YOU_ASKING_FOR_THIS_USELESS_INFORMATION_HUH. 

> method but it is being considered.  

You're using RENEW as if it were SET_REQUEST_STATE, which seems kind
of reasonable as a general way to deal with the fact that certain 
ops might need access to a stateid, even though the spec doesn't 
provide them one.  In this particular case, I'm a little worried
about this strategy.  The delegated file may be multiply-linked
so by not doing the CB_GETATTR the server is requiring the cient
to look at file-handles for each of the returned directory entries,
see if they match write delegations and if so, get any relevant 
information about delegated writes not propagated to the server and
use that to deletrmine what the real attributes.  Of course, the 
client is indicating his readiness to do that by doing the RENEW
up front.  But the spec doesn't really say that the server is safe
doing this or that the client is required to prepare for that if 
the RENEW is above the READDIR in the request.  Consider a general
optimization package that you wrote to combine ops into single 
requests to reduce latency.  Such a package might tag a RENEW onto
some non-stateid op that was going by anyway.  But if it did that,
the data returned by READDIR would be affected by prepending a
RENEW even though that couldn't be predicted from the spec.
 
> Any opinions about something like

My feeling is that we have some rough edges with delegation in v4.0,
that we can work around in various ways, but that we probably need 
a few little tweaks in v4.1.  If directory delegation goes forward,
this could be addressed in the same document, but even if it doesn't,
we should get those minor tweaks in anyway.