Subject: [nfsv4] 12.5.4.2. LAYOUTCOMMIT and size

From: "Erasani, Pranoop" <[email-masked]>
Date: Mon, 3 Mar 2008 01:46:05 -0800

I'm working on the Data ONTAP pNFS server and have some nits to report
and questions to ask (based on draft 20). Apologies if these are already
discussed.
 
1. s/The data type of lora_last_write_offset/The data type of
loca_last_write_offset/
 
2. What does zero-extend actually mean? Could this be clarified?
 
"If the client wants to explicitly zero-extend or truncate a file, the
SETATTR operation MUST be used; SETATTR use is not required when simply
writing past EOF via WRITE."
 
3. I don't see text to the effect that describes the impact of SETATTR
on a file size from a client that does not hold the layout.
 
- Pranoop

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Benny Halevy <[email-masked]>
Date: Mon, 03 Mar 2008 15:43:30 +0200

> writing past EOF via WRITE."

This means extending the file using SETATTR to increase the file's size.
This implicitly appends zeroes after the current EOF.

> on a file size from a client that does not hold the layout.

Hmm, in section 18.30.  Operation 34: SETATTR - Set Attributes:
18.30.4.  IMPLEMENTATION
...
   The file size attribute is used to request changes to the size of a
   file.  A value of 0 (zero) causes the file to be truncated, a value
   less than the current size of the file causes data from new size to
   the end of the file to be discarded, and a size greater than the
   current size of the file causes logically zeroed data bytes to be
   added to the end of the file.

Benny

>  
> - Pranoop
>  
>  
> 
> 
> 
> ------------------------------------------------------------------------
>

---

From: "Muntz, Daniel" <[email-masked]>
Date: Mon, 3 Mar 2008 13:32:17 -0800

This seems to suggest that one can do a SETATTR without holding a
layout, i.e., execute an operation that may change the layout of a file
(by truncation or extension) without acquiring a layout and doing a
LAYOUTCOMMIT.   Pranoop's question was specifically about the case where
a client does not hold a layout.

A related scenario that bothers me, and Trond convinced me is "correct",
is when a client has a layout for a file, does something like extend the
file, but does has not yet done a LAYOUTCOMMIT.  His assertion is that
the client's notion of the file size is not updated until LAYOUTCOMMIT
(which makes sense if you imagine that a GETATTR will not return the
updated size until after a LAYOUTCOMMIT).

  -Dan 

-----Original Message-----
From: Benny Halevy [mailto:[email-masked]] 
Sent: Monday, March 03, 2008 5:44 AM
To: Erasani, Pranoop
Cc: [email-masked]
Subject: Re: [nfsv4] 12.5.4.2. LAYOUTCOMMIT and size

> and questions to ask (based on draft 20). Apologies if these are 
> already discussed.
>  

> SETATTR operation MUST be used; SETATTR use is not required when 
> simply writing past EOF via WRITE."

Benny

---

From: "Halevy, Benny" <[email-masked]>
Date: Mon, 3 Mar 2008 16:51:13 -0500

I'm not sure I see what is the problem.
The text regarding LAYOUT COMMIT asserts that loca_last_write_offset
must be used solely for reporting the actual byte with the highest offset 
onto which the client had previously written (and forgive my probable abuse
of tenses in advance :)  If the client wants to truncate or extend the file it MUST
use SETTER instead.  The client can use the SETTER to truncate/extend the file
without holding a layout as it always could in all published versions of NESS.
The server, BTW, may recall the layout if the client sends a SETTER while it
(or another client) is holding a layout (typically overlapping with the range between
the current and new ELF).
 
Regarding the scenario described in the second paragraph below: true, the client's
updates to the file may or may not be visible to other clients until LAYOUT COMMIT.
This include both data and metadata (e.g. length or modification time).
Note that this makes LAYOUT COMMIT mandatory to succeed when flushing
dirty data from the client's cache (on e.g. sync or close) and before the dirty data
is marked clean.
 
Benny

________________________________

From: Muntz, Daniel [mailto:[email-masked]]
Sent: Mon 2008-03-03 23:32
To: Halevy, Benny; Erasani, Pranoop
Cc: [email-masked]
Subject: RE: [nfsv4] 12.5.4.2. LAYOUTCOMMIT and size

  -Dan

-----Original Message-----
From: Benny Halevy [mailto:[email-masked]]
Sent: Monday, March 03, 2008 5:44 AM
To: Erasani, Pranoop
Cc: [email-masked]
Subject: Re: [nfsv4] 12.5.4.2. LAYOUTCOMMIT and size

Benny

---

From: "Erasani, Pranoop" <[email-masked]>
Date: Mon, 3 Mar 2008 14:59:38 -0800

> The text regarding LAYOUT COMMIT asserts that 
> loca_last_write_offset must be used solely for reporting the 
> actual byte with the highest offset onto which the client had 
> previously written (and forgive my probable abuse of tenses 
> in advance :)  If the client wants to truncate or extend the 
> file it MUST use SETTER instead.  The client can use the 
> SETTER to truncate/extend the file without holding a layout 
> as it always could in all published versions of NESS.
> The server, BTW, may recall the layout if the client sends a 
> SETTER while it (or another client) is holding a layout 
> (typically overlapping with the range between the current and 
> new ELF).

Here's is where I'm hung up. Why is this may?

Layouts are delegation-like. The spec seems to say that delegations are
recalled when a SETATTR comes in from another client?

Or Am I on the wrong track in understanding the essense of "may".

- Pranoop

>  
> Regarding the scenario described in the second paragraph 
> below: true, the client's updates to the file may or may not 
> be visible to other clients until LAYOUT COMMIT.
> Note that this makes LAYOUT COMMIT mandatory to succeed when 
> flushing dirty data from the client's cache (on e.g. sync or 
> close) and before the dirty data is marked clean.
>  
> Benny
> 
> This seems to suggest that one can do a SETATTR without 
> holding a layout, i.e., execute an operation that may change 
> the layout of a file (by truncation or extension) without 
> acquiring a layout and doing a
> A related scenario that bothers me, and Trond convinced me is 
> "correct", is when a client has a layout for a file, does 
> something like extend the file, but does has not yet done a 
> LAYOUTCOMMIT.  His assertion is that the client's notion of 
> the file size is not updated until LAYOUTCOMMIT (which makes 
> sense if you imagine that a GETATTR will not return the

---

From: "William A. (Andy) Adamson" <[email-masked]>
Date: Mon, 3 Mar 2008 18:19:44 -0500

> Here's is where I'm hung up. Why is this may?

some implementations (block, object) use the layout as access control
removing a layout or layout segment to fence a client (block layout) or to
replace a layout with a changed capability (object layout) and so may recall
layout segments that overlap with a SETATTR ELF change.

the file layout does not use a layout as access control - the Data Servers
perform stateid checking on READ/WRITE.  So I/O including SETATTR does not
result in a layout recall.

-->Andy

---

From: Benny Halevy <[email-masked]>
Date: Tue, 04 Mar 2008 11:30:50 +0200

> result in a layout recall.

Right. It boils down to whether the SETATTR invalidates the outstanding layout
(like in the blocks case) and to mutual exclusion (like in RAID-5 striping
over objects where the SETATTR is a write-like operation that must be serialized
with client writes over the truncated stripes)

Benny

---

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 6 Mar 2008 09:00:45 -0500

Layouts are like delegations in being recallable locks that provide
forms of optimization that may be granted or not at the server's
discretion.
 
That doesn't mean that they have the same requirements for simultaneous
access, as Andy points out.  For the files layout type, it may be
perfect valid for a server to give out write layouts to multiple clients
as long as the striping pattern file of the file does not change.  The
problem of multiple writers is one for the application to resolve, by
using appropriate deny modes or byte-range locking, just as would be the
case with non-parallel NFS. 

From: William A. (Andy) Adamson [mailto:[email-masked]] 
Sent: Monday, March 03, 2008 6:20 PM
To: Erasani, Pranoop
Cc: Halevy, Benny; [email-masked]
Subject: Re: [nfsv4] 12.5.4.2. LAYOUTCOMMIT and size

some implementations (block, object) use the layout as access control
removing a layout or layout segment to fence a client (block layout) or
to replace a layout with a changed capability (object layout) and so may
recall layout segments that overlap with a SETATTR ELF change.

the file layout does not use a layout as access control - the Data
Servers perform stateid checking on READ/WRITE.  So I/O including
SETATTR does not result in a layout recall.

-->Andy

	Layouts are delegation-like. The spec seems to say that
delegations are
	recalled when a SETATTR comes in from another client?
	
	Or Am I on the wrong track in understanding the essense of
"may".
	
	- Pranoop

	> close) and before the dirty data is marked clean.
----------------------------------------------------------------------
	> > --