Subject: RE: why no change_info4 in WRITE4resok?

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 5 Nov 2001 12:25:29 -0800

> Indeed, the nfsv3 write response includes a wcc_data structure, which
> seems to be functionally equivalent to the nfsv4 change_info4.  Is there
> some reason I'm missing why this was removed in v4?

The actual sequence of events was that pre- and post-attributes
were eliminated from V4 in all cases and then added back in those
cases in which people writing clients asserted that the corresponding
attributes had proved useful in V3.  WRITE wasn't one of those.

The reason that they were removed initially was that it was felt
that COMPOUND would allow a general facility allowing such special
cases to be removed.  Once it became clear that COMPOUND would
not provide the needed atomicity, change_info was added to the
protocol. 

I'm not sure in what specific cases this would be useful for 
WRITE.  If two clients have the file open for write and are 
not using byte-range locking, then you've got a mess and any
attempt at cache consistency is futile.  Are you assuming
that byte-range locks will be used?  In that case, I don't 
see that this will be the most useful way to proceed.

Let's say I lock a range and then proceed to do some IO.  Now
suppose I could use change_info to determine whether I am the
only one writing to the file.  First note that write 
delegations will take care of the most common case in which
I am the only writing, that is the case in which I am the 
only one with the file open.  There are still going to be
cases in which I might save a fetch of a block if I knew
that nobody else had written it but I think that change_info
is not gong to be the best way to do that.

Here's what I consider the prototypical case:

   1. I lock a region.

   2. I fetch some blocks.

   3. I modify then and write them back.

   4. I release the lock.

   5. Some time later I lock the same or an overlapping region
   and when I have to read the relevant block I'd like not to have
   to fetch them if nobody else has modified them.

So I could do this by using the attributes fetched before 1) 
compared with change-info from one or more blocks written 
during 3) and then further compared with attributes fetched
at 5).  There's considerable complexity doing that if 3)
writes multiple blocks in parallel.  However, if I just 
fetch the change-time before 4) and compare it with the
change time at 5), I have a simpler way to see if I have to
refetch and one which has a much smaller window to unrelated
writes for other regions of the file that would cause me 
to do extra READ's when I don't need to.

So what is the case in which this would be helpful?   

-----Original Message-----
From: Kendrick M. Smith [mailto:[email-masked]]
Sent: Friday, November 02, 2001 9:46 PM
To: [email-masked]
Subject: why no change_info4 in WRITE4resok?

It would be very helpful if the WRITE response included a change_info4
structure, so that the client could maintain cache consistency across
a WRITE.  As things stand, I don't see how the client can do this without
a potential race condition against other clients.  The best the client
can do is to emit a GETATTR (for the FATTR4_CHANGE attribute) before and
after the WRITE, and decide that if the first FATTR4_CHANGE matches its
cache, then the cache is valid and the cached changeid should be replaced
with the second FATTR4_CHANGE.  This is prone to a race condition since
the sequence GETATTR WRITE GETATTR is not guaranteed atomic.

Indeed, the nfsv3 write response includes a wcc_data structure, which
seems to be functionally equivalent to the nfsv4 change_info4.  Is there
some reason I'm missing why this was removed in v4?
Kendrick

---

From: "Kendrick M. Smith" <[email-masked]>
Date: Mon, 5 Nov 2001 15:54:51 -0500 (EST)

> see that this will be the most useful way to proceed.

I am thinking of the case where two clients have the file
opened for write and are not using byte-range locking.  It
seems to me that cache consistency can be maintained in this
case, _provided_ that change_info is returned in the WRITE
response.  Otherwise it is prone to race conditions, and
probably is futile.

Generally speaking, NFSv4 clients will use the FATTR4_CHANGE
attribute as a way to invalidate previously cached data: when
the FATTR4_CHANGE attribute changes, the cached data must be
invalidated.  After a client does a WRITE, the FATTR4_CHANGE
attribute will always change.  The client needs a way to tell
whether this change is (1) a side effect of the WRITE, or (2)
the result of another client's operation at roughly the same
time as the WRITE.  If a change_info4 is returned in the write
response, the client can determine this: if the "before" field
of the change_info4 disagrees with the old FATTR4_CHANGE
attribute, then (2) is true (and the client must invalidate
its cache).  If the "before" field agrees with the old
FATTR4_CHANGE attribute, then (1) is true and the client can
keep its cache while updating the saved FATTR4_CHANGE attribute
to the value in the "after" field.  Of course, if a subsequent
retrieval of the FATTR4_CHANGE attribute disagrees with the
"after" field, then the cache must be invalidated as usual.

Indeed, the linux nfsv3 client does rely on the wcc_data
returned in the v3 write response to maintain cache consistency
in its write path.

All of this is totally analagous to a situation where two
clients are concurrently creating and removing files in a
directory.  Because of the change_info4 structures returned
in the CREATE and REMOVE response, the clients can maintain
cache consistency.

Am I missing some reason why this works for CREATE/REMOVE etc.,
but wouldn't work for WRITE?

Kendrick