Subject: [nfsv4] Scribe 6/28

From: "Reyes-Ortiz, Talia" <[email-masked]>
Date: Thu, 28 Jun 2012 23:05:01 +0000

Chuck, Tom, Bryan, Trond, Talia, Dean

Note Well Agreement. See: http://www.ietf.org/NOTEWELL.html

START

Dean: Going to remove O_DIRECT to give a specific example for IO_ADVISE_NOREUSE

3905 - Trond and Chuck: What is the STATE_ ID, is it limited? Open block or delegation

Dean: The lifetime of a file hint is the open and the lifetime of byte range hint is a compound

Chuck: Wants an explicit argument to state what the lifetime is

END
Begin next meeting at: 4069

Cheers!
Talia Reyes-Ortiz
Program Manager
CTO Office
NetApp Inc.
408.822.1566 Direct Phone
[email-masked]
[Description: http://media.netapp.com/images/esigs-innovative-company-480x75.png]<http://www.forbes.com/special-features/innovative-companies.html>

---

From: Dean <[email-masked]>
Date: Fri, 29 Jun 2012 13:44:42 -0700

On 6/28/12 4:05 PM, Reyes-Ortiz, Talia wrote:
> **
> *START *

First, I propose to change the following text:
   1466       POSIX_FADV_NOREUSE -  Specifies that the application 
expects to
   1467          access the specified data once and then not reuse it 
thereafter.
to
POSIX_FADV_NOREUSE -  Specifies that the application expects to
access the specified data once and then not reuse it thereafter.
Currently, the NFS client has no way of passing caching hints to the 
server for
forthcoming I/O requests.  This hint allows the application to specify 
data access information
that will allow the server to optimize its caching policy regarding the 
specified data.
For example with forthcoming writes, the server may decide to bypass its 
cache and
use direct I/O to place the data directly on disk.  This may allow the 
server
to avoid polluting its cache and keep data in the cache that has a 
greater possibly of re-access.
Another example is with forthcoming read requests,  where the server may 
decide to either use direct I/O, avoiding
its cache altogether, or use a separate cache that will avoid evicting data
that has a greater possibly of re-access.

Second, I propose to remove the following:
   1498       Another use case is "Direct I/O", which allows a stated 
holder to
   1499       inform the server that it does not wish to cache data. 
Today, for
   1500       applications that only intend to read data once, the use 
of direct
   1501       I/O disables client caching, but does not affect server 
caching.  By
   1502       caching data that will not be re-read, the server is 
polluting its
   1503       cache and possibly causing useful cached data to be 
evicted.  By
   1504       informing the server of its expected I/O access, this 
situation can
   1505       be avoid.  Direct I/O can be used in Linux and AIX via the 
open()
   1506       O_DIRECT parameter, in Solaris via the directio() 
function, and in
   1507       Windows via the CreateFile() FILE_FLAG_NO_BUFFERING flag.

Third, I propose to remove the following text:
   4060       Another useful point would be when an application 
indicates it is
   4061       using direct I/O. Direct I/O may be specified at file 
open, in which
   4062       case a IO_ADVISE may be included in the same compound as 
the OPEN
   4063       operation with the IO_ADVISE4_NOREUSE flag set.  Direct 
I/O may also
   4064       be specified separately, in which case a IO_ADVISE 
operation can be
   4065       sent to the server separately.  As above, IO_ADVISE4_WRITE and
   4066       IO_ADVISE4_READ may be set based upon the type of file access
   4067       specified when the file was opened.

Fourth, I propose to change the following text:
   4055       The most obvious is in direct response to an application's 
execution
   4056       of posix_fadvise.  In this case, IO_ADVISE4_WRITE and 
IO_ADVISE4_READ
   4057       may be set based upon the type of file access specified 
when the file
   4058       was opened.
to
The most obvious is in direct response to an application's execution
of posix_fadvise.  The arguments from posix_fadvise would be translated
directly to IO_ADVISE and sent immediately to the server. The posix_fadvise
hints specified in Section 3.2 translate to IO_ADVISE hints by replacing
the prefix POSIX_FADV with IO_ADVISE4. In this case,
IO_ADVISE4_WRITE and IO_ADVISE4_READ
may be set based upon the type of file access specified when the file
was opened.

Fifth, while not required, I noticed that section 3.3 doesn't have any 
motivational text for IO_ADVISE4_INIT_PROXIMITY.  Just a thought, but 
might be worth considering.

Dean
> **
> *END *
> *Begin next meeting at: 4069*
>
> Cheers!
>
> *Talia Reyes-Ortiz*
> Program Manager
> CTO Office
>
> *NetApp Inc.*
> Description: 
> http://media.netapp.com/images/esigs-innovative-company-480x75.png 
> <http://www.forbes.com/special-features/innovative-companies.html>
>
>
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: <[email-masked]>
Date: Fri, 29 Jun 2012 17:12:13 -0400

I think that what is there now is fine but if you want some elaboration, I would prefer something more like the following.

POSIX_FADV_NOREUSE -  Specifies that the application expects to
access the specified data once and then not reuse it thereafter.
In previous NFS versions, the NFS client had no way of passing information about such
application expectations to the server. This hint allows the application to specify its expectations about
forthcoming data use and so allow the server to improve its caching policy regarding the specified data.
For example, with forthcoming reads and writes, the server may decide to make the data read or written
more likely to be selected for cache reuse than it would be otherwise. This may allow the server
to keep data in its cache that has a higher probability of re-access.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Dean
Sent: Friday, June 29, 2012 4:45 PM
To: [email-masked]; Eisler, Michael
Subject: Re: [nfsv4] Scribe 6/28

START

First, I propose to change the following text:
  1466       POSIX_FADV_NOREUSE -  Specifies that the application expects to
  1467          access the specified data once and then not reuse it thereafter.
to
POSIX_FADV_NOREUSE -  Specifies that the application expects to
access the specified data once and then not reuse it thereafter.
Currently, the NFS client has no way of passing caching hints to the server for
forthcoming I/O requests.  This hint allows the application to specify data access information
that will allow the server to optimize its caching policy regarding the specified data.
For example with forthcoming writes, the server may decide to bypass its cache and
use direct I/O to place the data directly on disk.  This may allow the server
to avoid polluting its cache and keep data in the cache that has a greater possibly of re-access.
Another example is with forthcoming read requests,  where the server may decide to either use direct I/O, avoiding
its cache altogether, or use a separate cache that will avoid evicting data
that has a greater possibly of re-access.

Second, I propose to remove the following:
  1498       Another use case is "Direct I/O", which allows a stated holder to
  1499       inform the server that it does not wish to cache data.  Today, for
  1500       applications that only intend to read data once, the use of direct
  1501       I/O disables client caching, but does not affect server caching.  By
  1502       caching data that will not be re-read, the server is polluting its
  1503       cache and possibly causing useful cached data to be evicted.  By
  1504       informing the server of its expected I/O access, this situation can
  1505       be avoid.  Direct I/O can be used in Linux and AIX via the open()
  1506       O_DIRECT parameter, in Solaris via the directio() function, and in
  1507       Windows via the CreateFile() FILE_FLAG_NO_BUFFERING flag.

Third, I propose to remove the following text:
  4060       Another useful point would be when an application indicates it is
  4061       using direct I/O. Direct I/O may be specified at file open, in which
  4062       case a IO_ADVISE may be included in the same compound as the OPEN
  4063       operation with the IO_ADVISE4_NOREUSE flag set.  Direct I/O may also
  4064       be specified separately, in which case a IO_ADVISE operation can be
  4065       sent to the server separately.  As above, IO_ADVISE4_WRITE and
  4066       IO_ADVISE4_READ may be set based upon the type of file access
  4067       specified when the file was opened.

Fourth, I propose to change the following text:
  4055       The most obvious is in direct response to an application's execution
  4056       of posix_fadvise.  In this case, IO_ADVISE4_WRITE and IO_ADVISE4_READ
  4057       may be set based upon the type of file access specified when the file
  4058       was opened.
to
The most obvious is in direct response to an application's execution
of posix_fadvise.  The arguments from posix_fadvise would be translated
directly to IO_ADVISE and sent immediately to the server. The posix_fadvise
hints specified in Section 3.2 translate to IO_ADVISE hints by replacing
the prefix POSIX_FADV with IO_ADVISE4. In this case,
IO_ADVISE4_WRITE and IO_ADVISE4_READ
may be set based upon the type of file access specified when the file
was opened.

Fifth, while not required, I noticed that section 3.3 doesn't have any motivational text for IO_ADVISE4_INIT_PROXIMITY.  Just a thought, but might be worth considering.

Dean

END

Cheers!
Talia Reyes-Ortiz
Program Manager
CTO Office
NetApp Inc.
408.822.1566 Direct Phone
[email-masked]<mailto:[email-masked]>
<http://www.forbes.com/special-features/innovative-companies.html>

nfsv4 mailing list

[email-masked]<mailto:[email-masked]>

---

From: Peter Staubach <[email-masked]>
Date: Fri, 29 Jun 2012 17:17:29 -0400

I am not sure that I see the value in comparing NFSv4.X to previous versions of NFS.  Does that sentence really add value or could it be removed and keep the paragraph more succinct?

                Thanx...

                                ps

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Friday, June 29, 2012 5:12 PM
To: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [nfsv4] Scribe 6/28

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of Dean
Sent: Friday, June 29, 2012 4:45 PM
To: [email-masked]<mailto:[email-masked]>; Eisler, Michael
Subject: Re: [nfsv4] Scribe 6/28

START

Dean

END

Cheers!
Talia Reyes-Ortiz
Program Manager
CTO Office
NetApp Inc.

nfsv4 mailing list

---

From: Chuck Lever <[email-masked]>
Date: Fri, 29 Jun 2012 17:17:47 -0400

> Fifth, while not required, I noticed that section 3.3 doesn't have any motivational text for IO_ADVISE4_INIT_PROXIMITY.  Just a thought, but might be worth considering.

I agree, this would be valuable, especially considering that there does not appear to be an equivalent posix_fadvise(2) flag.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 29 Jun 2012 14:29:11 -0700

> I agree, this would be valuable, especially considering that there does not appear to be an equivalent posix_fadvise(2) flag.

One of the two of you pointed this out last week. :->

-- 
[email-masked], ex-cfb

---

From: <[email-masked]>
Date: Fri, 29 Jun 2012 17:28:32 -0400

Good point.  It could and should be removed.  So how about:

POSIX_FADV_NOREUSE -  This hint specifies that the application expects to
access the specified data once and then not reuse it thereafter.
The intention is to allow the server to improve its caching policy regarding the specified data.
For example, with forthcoming reads and writes, the server may decide to make the data read or written
more likely to be selected for cache reuse than it would be otherwise. This may allow the server
to keep data in its cache that has a higher probability of re-access.

If people still want more succinctness, we could go with what is currently in the document.

From: Peter Staubach [mailto:[email-masked]]
Sent: Friday, June 29, 2012 5:17 PM
To: Noveck, David; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [nfsv4] Scribe 6/28

                Thanx...

                                ps

START

Dean

END

Cheers!
Talia Reyes-Ortiz
Program Manager
CTO Office
NetApp Inc.

nfsv4 mailing list

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 29 Jun 2012 14:36:13 -0700

> that will allow the server to optimize its caching policy regarding the 

So these are generic statements about why we want to use IO_ADVISE.
How do they apply specifically just to POSIX_FADV_NOREUSE/IO_ADVISE_NOREUSE?

> that has a greater possibly of re-access.

I don't see any value add here either.  And the text certainly belongs in
the IMPLEMENTATION (13.8.4) notes.

>From what I can tell, your use case is a client extending O_DIRECT to
the caching policies on the server. Nothing in the draft prevents this
from occurring. But I heard others yesterday wanting
to preserve the ability for O_DIRECT to not dictate what happens on the server.

And that choice is in the API on the client, not in the protocol.

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 29 Jun 2012 14:41:30 -0700

> If people still want more succinctness, we could go with what is currently in the document.

Except that text does not belong in a discussion of POSIX_FADV_NOREUSE, but rather in the one
for IO_ADVISE_NOREUSE.

I also thought we agreed last week that there was no need to restate either the posix_fadvise()
documentation nor the Linux man page for it. I.e., I have made chapter 3 very succinct in the
document:

3.  Support for Application IO Hints

3.1.  Introduction

   Applications currently have several options for communicating I/O
   access patterns to the NFS client.  While this can help the NFS
   client optimize I/O and caching for a file, it does not allow the NFS
   server and its exported file system to do likewise.  This chapter
   adds a new IO_ADVISE operation to communicate the client file access
   patterns to the NFS server.  The NFS server upon receiving a
   IO_ADVISE operation MAY choose to alter its I/O and caching behavior,
   but is under no obligation to do so.

   Applications can issue client I/O hints via the posix_fadvise() [6]
   operation.  For example, on Linux, when an application uses
   posix_fadvise() to specify a file will be read sequentially, Linux
   doubles the readahead buffer size.

   Application specific NFS clients such as those used by hypervisors
   and databases can also leverage application hints to communicate
   their specialized requirements.

3.2.  IANA Considerations

   The IO_ADVISE_type4 will be extended through an IANA registry.

And I think 3.2 goes as well, see my discussion on the WG alias about why the
IANA registry does not work with the current XDR.

---

From: Peter Staubach <[email-masked]>
Date: Fri, 29 Jun 2012 17:41:40 -0400

This looks fine to me.

                                ps

From: [email-masked] [mailto:[email-masked]]
Sent: Friday, June 29, 2012 5:29 PM
To: Peter Staubach; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [nfsv4] Scribe 6/28

From: Peter Staubach [mailto:[email-masked]]<mailto:[mailto:[email-masked]]>
Sent: Friday, June 29, 2012 5:17 PM
To: Noveck, David; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: RE: [nfsv4] Scribe 6/28

                Thanx...

                                ps

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Friday, June 29, 2012 5:12 PM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [nfsv4] Scribe 6/28

START

Dean

END

Cheers!
Talia Reyes-Ortiz
Program Manager
CTO Office
NetApp Inc.

nfsv4 mailing list

---

From: Dean <[email-masked]>
Date: Fri, 29 Jun 2012 16:54:55 -0700

> documentation nor the Linux man page for it. I.e., I have made chapter 3 very succinct in the
Hmmm, I specifically remember you stating that there was no point of 
removing the information as we are not hurting for space.  I had 
originally thought that we should remove it, but now that there seems to 
be some confusion on how to implement posix_fadvise, explicitly stating 
its requirements may not be such a bad idea.
Dean

---

From: Dean <[email-masked]>
Date: Fri, 29 Jun 2012 17:22:16 -0700

On 6/29/12 2:28 PM, [email-masked] wrote:
>
> *Good point. It could and should be removed.  So how about:*
>
> *If people still want more succinctness, we could go with what is 
> currently in the document.
> *
>

 From the call, it seems there is some confusion on how posix_fadvise 
would map to IO_ADVISE, so I thought this would help.

One thing I tried to capture is that this hint acts on forthcoming I/O, 
and is not intended as a hint regarding past I/O (that is what DONTNEED 
is for).   So one thing we could do is be more clear about this in the 
IO_ADVISE section instead.

Dean

> *
> ***
>
> *From:*Peter Staubach [mailto:[email-masked]]
> *Sent:* Friday, June 29, 2012 5:17 PM
> *To:* Noveck, David; [email-masked]; [email-masked]; 
> [email-masked]
> *Subject:* RE: [nfsv4] Scribe 6/28
>
> I am not sure that I see the value in comparing NFSv4.X to previous 
> versions of NFS.  Does that sentence really add value or could it be 
> removed and keep the paragraph more succinct?
>
> Thanx...
>
> ps
>
> *From:[email-masked] [mailto:[email-masked]] *On 
> Behalf Of [email-masked]
> *Sent:* Friday, June 29, 2012 5:12 PM
> *To:* [email-masked]; [email-masked]; [email-masked]
> *Subject:* Re: [nfsv4] Scribe 6/28
>
> *I think that what is there now is fine but if you want some 
> elaboration, I would prefer something more like the following. *
>
> application expectations to the server. This hint allows the 
> application to specify its expectations about
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] 
> <mailto:[mailto:[email-masked]]> *On Behalf Of *Dean
> *Sent:* Friday, June 29, 2012 4:45 PM
> *To:* [email-masked] <mailto:[email-masked]>; Eisler, Michael
> *Subject:* Re: [nfsv4] Scribe 6/28
>
> Another example is with forthcoming read requests,  where the server 
> may decide to either use direct I/O, avoiding
> [email-masked] <mailto:[email-masked]>
>
> Description: 
> [email-masked]  <mailto:[email-masked]>

---

From: Dean <[email-masked]>
Date: Fri, 29 Jun 2012 17:27:26 -0700

> And that choice is in the API on the client, not in the protocol.

I think you have misread my statement.  This has nothing to do with 
O_DIRECT.

This text just gives motivation for why we want to support posix_fadvise 
NOREUSE in the NFS protocol.
Dean