Subject: [nfsv4] dense and sparse with IO_ADVISE (was Re: [NFSv4.2] Meeting Minutes for 09/22/2011

From: "Mike Eisler" <[email-masked]>
Date: Fri, 30 Sep 2011 13:56:20 -0700

On Thu, September 22, 2011 10:25 pm, Thomas Haynes wrote:

> + IO_ADVISE
>
> Dean summarized Mike's new text.
>
> He had some questions about dense versus sparse.

I believe the issue is this. If both of the following are true:
- the pNFS server is allowing IO_ADVISE to a DS
- the layout is sparse

then the client can potentially specify a byte range that hits an illegal
hole in the DS's file. We should examine two operations, READ and WRITE,
that also take a byte range, and can be sent to DSes.

If we sent a WRITE to a sparse DS file that hits an illegal hole, we would
return an error, NFS4ERR_PNFS_IO_HOLE. If we sent a READ to a sparse DS
file that hits an illegal hole, then if the offset were in the hole, we
would return NFS4ERR_PNFS_IO_HOLE. If the offset were not in the hole, but
the byte range, due to length, hit a hole, the server could simply return
a short read instead of NFS4ERR_PNFS_IO_HOLE, but nonetheless it could
return NFS4ERR_PNFS_IO_HOLE.

So the question is under what conditions the server return
NFS4ERR_PNFS_IO_HOLE to IO_ADVISE on sparse layouts? There are two
extremes
we can take here, and a spectrum of approaches in between.

At one extreme, which I will call the strict extreme, the DS rejects with
NFS4ERR_PNFS_IO_HOLE any IO_ADVISE that specifies a byte range that hits a
hole. In general this is bad because if the client wants to for example
indicate that sequential access is going to be used, then it just wants to
send an offset of zero, and a length of zero, which indicates sequential
access for the entire file. The strict extreme would require many
IO_ADVISE requests, one per stripe unit. For small stripe units, this is
going to be really bad.

At the other extreme, which will call the laid back extreme, the DS
never rejects any byte range with the error NFS4ERR_PNFS_IO_HOLE.

I suggest the laid back extreme because it produces the fewest IO_ADVISE
operations.

So I propose the following text to be added to the I-D:

"If neither of the flags NFL42_UFLG_IO_ADVISE_THRU_MDS and NFL4_UFLG_DENSE
are set in the layout, then any IO_ADVISE request sent to the data server
with a byte range that overlaps stripe unit that the data server does not
serve MUST NOT result in the status NFS4ERR_PNFS_IO_HOLE. Instead, the
response SHOULD be successful and if the server applies IO_ADVISE hints on
any stripe units that overlap with the specified range, those hints SHOULD
be indicated in the response.