Subject: [nfsv4] NFSv4.2 behaviour for SEEK when sa_offset == file_size (or at EOF, if you prefer)

From: Rick Macklem <[email-masked]>
Date: Sat, 10 Aug 2019 21:38:22 +0000

Hi,

I have read this multiple times and I'll admit it is still not clear to me what the
correct reply to a SEEK operation is when sa_offset is at EOF (or equal to the
file's size, if you prefer):
   From the given sa_offset, find the next data_content4 of type sa_what
   in the file.  If the server cannot find a corresponding sa_what, then
   the status will still be NFS4_OK, but sr_eof would be TRUE.  If the
   server can find the sa_what, then the sr_offset is the start of that
   content.  If the sa_offset is beyond the end of the file, then SEEK
   MUST return NFS4ERR_NXIO.

Although it isn't 100% clear to me, it sounds like the correct reply for a
request with sa_offset == file_size is:
NFS_OK plus sr_eof == True and sr_offset == sa_offset
(The key phrases are "cannot find a corresponding sa_what" and
 "beyond the end of the file".)

Is that consistent with how others interpret this?

I'll note that doesn't seem to agree with the FreeBSD implementation of
lseek(SEEK_DATA/SEEKHOLE), which returns ENXIO if the offset argument
is at EOF (or equal to the file's size, which is what I mean by EOF).
The FreeBSD implementation for lseek() seems to agree with the POSIX draft
(at least the version I found), which states:
 [ENXIO] The whence argument is SEEK_HOLE or SEEK_DATA, and offset is
 greater or equal to the file size; or the whence argument is SEEK_DATA
 and the offset falls beyond the last byte not within a hole.

I'll admit that I don't have a Linux system handy to do a test on, but the
attached trivial program could be run on a Linux system to find out.
(Just run it with a valid file name as argument.)

Thanks for any help with this, rick

---

From: David Noveck <[email-masked]>
Date: Sun, 11 Aug 2019 06:07:19 -0400

> file's size, if you prefer):

Why is it unclear?

> Is that consistent with how others interpret this?

It agrees with my interpretation.  I can't think of anything else one can
do.

> is at EOF (or equal to the file's size, which is what I mean by EOF).

That's not surprising given that lseek cannot return an eof indication (e.
g. sr_eof) without reporting a failure.  Also, lseek, when it does not
fail, moves the seek pointer, unike the corresponding v4.2 operation. As a
result, when faced with a potentially dubious request, returning an error
is a better choice than moving the seek pointer somewhere that might not be
right.

>  greater or equal to the file size;

Just as you found.

or the whence argument is SEEK_DATA
>  and the offset falls beyond the last byte not within a hole.

I believe that the v4.2 operation would also return sr_eof in that case as
well

It seems that implementations of the draft POSIX lseek would always map
sr_eof to ENXIO and leave the seek pointer unchanged.

---

From: Rick Macklem <[email-masked]>
Date: Sun, 11 Aug 2019 22:49:58 +0000

David Noveck wrote:
>On Sat, Aug 10, 2019, 5:44 PM Rick Macklem >><[email-masked]<mailto:[email-masked]>> wrote:
>>Hi,
>>
Well, I suppose it is because the Linux lseek(2) man page says:
    In both of the above cases, lseek() fails if offset points past the
       end of the file.
and returns the error for the case where offset == file_size.
This suggests some ambiguity w.r.t. the meaning of
"past the end of file/beyond the end of file".

It also appears that the Linux knfsd returns NFS4ERE_NXIO for the case
where offset == file_size. I haven't tested this, but the code appears to use
generic_file_llseek_size() which returns ENXIO for this case.
(I will test this someday soon, but it requires me to hack the FreeBSD client
 to make this request, since the lseek(2) in the FreeBSD client will check for
 and return ENXIO without doing an RPC for the offset == file_size case.
 I also don't happen to have a system running Linux at this time.)

>It agrees with my interpretation.  I can't think of anything else one can do.
Well, returning ENXIO/NFS4ERR_NXIO is the other possibility, which is what the
Linux knfsd appears to do.
>That's not surprising given that lseek cannot return an eof indication (e. g. sr_eof) >without reporting a failure.  Also, lseek, when it does not fail, moves the seek >pointer, unike the corresponding v4.2 operation. As a result, when faced with a >potentially dubious request, returning an error is a better choice than moving the >seek pointer somewhere that might not be right.
Agreed. However people tend to make POSIX-like assumptions when it isn't
crystal clear. It also means that most POSIX-like clients (Linux, FreeBSD) will
return ENXIO and never attempt an NFSv4.2 operation/RPC for this case,
leaving it untested.
>I believe that the v4.2 operation would also return sr_eof in that case as well
>
Hopefully others can tell us what extant NFSv4.2 servers like Linux's knfsd does
for this case. (I'll wager ENXIO for the Linux knfsd but, again, I have no convenient
way to test this at this time.)

>It seems that implementations of the draft POSIX lseek would always map sr_eof to >ENXIO and leave the seek pointer unchanged.
I'll assume you meant "for all the above cases", since the case that satisfies:
   All files MUST have a virtual hole at the end of the file.  That is,
   if a file system does not support sparse files, then a COMPOUND with
   {SEEK 0 NFS4_CONTENT_HOLE;} would return a result of {SEEK 1 X;},
   where "X" was the size of the file.
would update the offset to file_size, despite sr_eof == true { 1 in the RFC snippet }.
(The POSIX draft lseek(2) does have the "virtual hole" at EOF as I understand it.)

I am interested in finding out what other NFSv4.2 server implementations do?

Thanks for your comments, rick

---

From: "Frank Filz" <[email-masked]>
Date: Mon, 12 Aug 2019 08:05:06 -0700

I went through this when implementing SEEK for FSAL_VFS for Ganesha.

Yes, always translating ENXIO to eof is true seems to work, here’s the code in Ganesha:

        ret = lseek(my_fd, offset, what);

        if (ret < 0) {

                if (errno == ENXIO) {

                        info->io_eof = TRUE;

                } else {

                        status = posix2fsal_status(errno);

                }

        } else {

                info->io_eof = (ret >= attrs.filesize);

                info->io_content.hole.di_offset = ret;

        }

Status is otherwise ERR_FSAL_NO_ERROR

And here’s how the FSAL function is invoked which shows the rest of the setup for the NFSv4.2 SEEK response:

                info.io_advise = state_found->state_data.io_advise;

                info.io_content.what = arg_SEEK->sa_what;

                if (arg_SEEK->sa_what == NFS4_CONTENT_DATA ||

                                arg_SEEK->sa_what == NFS4_CONTENT_HOLE) {

                        info.io_content.hole.di_offset = arg_SEEK->sa_offset;

} else

                        info.io_content.adb.adb_offset = arg_SEEK->sa_offset;

                fsal_status = obj->obj_ops->seek2(obj, state_found, &info);

                if (FSAL_IS_ERROR(fsal_status)) {

                        res_SEEK->sr_status = NFS4ERR_NXIO;

                        goto done;

                }

                res_SEEK->sr_resok4.sr_eof = info.io_eof;

                res_SEEK->sr_resok4.sr_offset = info.io_content.hole.di_offset;

        }

done:

I’m not sure that code is perfectly clean, but it passes the pynfs tests at least. I didn’t really touch our protocol layer code, I just implemented the FSAL_VFS code.

Frank

From: nfsv4 [mailto:[email-masked]] On Behalf Of David Noveck
Sent: Sunday, August 11, 2019 3:07 AM
To: Rick Macklem <[email-masked]>
Cc: NFSv4 <[email-masked]>
Subject: Re: [nfsv4] NFSv4.2 behaviour for SEEK when sa_offset == file_size (or at EOF, if you prefer)

On Sat, Aug 10, 2019, 5:44 PM Rick Macklem <[email-masked] <mailto:[email-masked]> > wrote:

Hi,

Why is it unclear?

That's not surprising given that lseek cannot return an eof indication (e.. g. sr_eof) without reporting a failure.  Also, lseek, when it does not fail, moves the seek pointer, unike the corresponding v4.2 operation. As a result, when faced with a potentially dubious request, returning an error is a better choice than moving the seek pointer somewhere that might not be right.

Just as you found.

It seems that implementations of the draft POSIX lseek would always map sr_eof to ENXIO and leave the seek pointer unchanged.

---

From: "Frank Filz" <[email-masked]>
Date: Mon, 12 Aug 2019 10:08:06 -0700

Ok, I looked into what I did last fall...

There actually isn't a pynfs test case, instead, we just used a simple test
program. Here's the Ganesha e-mail chain discussing the eof situation:

https://lists.nfs-ganesha.org/archives/list/[email-masked]/thre
ad/UIJLKD66O4BYKFS23UFKYVPUO33XPN5B/

Hmm, from that it looks like Linux knfsd always returns ENXIO, but then I
didn't change the Ganesha implementation...

So really I'm not quite sure what the best thing to do is other than test
whatever you do with all the clients you expect to use SEEK...

Note that for Ganesha it isn't reliable to compare sa_offset and filesize
since we can't guarantee atomicity between lseek and stat.

I think our implementation passes the tests we've run using the Linux
client, but since merging it last September, we haven't done any additional
testing.

Frank

> -----Original Message-----
> From: Rick Macklem [mailto:[email-masked]]
> Sent: Monday, August 12, 2019 9:05 AM
> To: Frank Filz <[email-masked]>; 'David Noveck'
> <[email-masked]>
> Cc: 'NFSv4' <[email-masked]>
> Subject: Re: [nfsv4] NFSv4.2 behaviour for SEEK when sa_offset ==
file_size (or
> at EOF, if you prefer)
> Frank Filz wrote:
> >Yes, always translating ENXIO to eof is true seems to work, here's the
code in
> >Ganesha:
> Note that I don't think the Linux client will ever do a Seek operation
where
> sa_offset == filesize. Does pynfs test this case?
> 
> >
> If the Linux lseek(2) ever returns ENXIO when the offset < filesize, this
would
> create a reply with eof == true but sr_offset < filesize, which seems
bogus to
> me?
> (Actually, for whence == SEEK_DATA, I don't see io_content.hole.di_offset
> getting  set at all.) Clients probably don't care/notice what sr_offset is
when eof
> == true.
> 
> (This may never happen in practice. For UFS on FreeBSD, there is always
data
> after  the last hole. I don't know about other file systems.)  --> In
other words,
> for UFS on FreeBSD, ENXIO never gets returned when
>        offset < attrs.filesize
> 
> >And here's how the FSAL function is invoked which shows the rest of the
setup
> for >the NFSv4.2 SEEK response:
> >
> >                        info.io_content.hole.di_offset =
> >arg_SEEK->sa_offset; } else
> >I'm not sure that code is perfectly clean, but it passes the pynfs tests
at least. I
> didn't >really touch our protocol layer code, I just implemented the
FSAL_VFS
> code.
> It would be interesting to know if pynfs tests the case of sa_offset ==
filesize?
> rick
> Subject: Re: [nfsv4] NFSv4.2 behaviour for SEEK when sa_offset ==
file_size (or
> at EOF, if you prefer)
> 
> 
> On Sat, Aug 10, 2019, 5:44 PM Rick Macklem
> <[email-masked]<mailto:[email-masked]>> wrote:
> Hi,
> 
request
> with sa_offset == file_size is:
> NFS_OK plus sr_eof == True and sr_offset == sa_offset (The key phrases are
> "cannot find a corresponding sa_what" and  "beyond the end of the file".)
> 
do.
> lseek(SEEK_DATA/SEEKHOLE), which returns ENXIO if the offset argument is
at
> EOF (or equal to the file's size, which is what I mean by EOF).
> 
> That's not surprising given that lseek cannot return an eof indication
(e.. g.
> sr_eof) without reporting a failure.  Also, lseek, when it does not fail,
moves the
> seek pointer, unike the corresponding v4.2 operation. As a result, when
faced
> with a potentially dubious request, returning an error is a better choice
than
> moving the seek pointer somewhere that might not be right.
> 
(at
> least the version I found), which states:
well
> It seems that implementations of the draft POSIX lseek would always map
sr_eof
> to ENXIO and leave the seek pointer unchanged.

---

From: Rick Macklem <[email-masked]>
Date: Mon, 12 Aug 2019 16:04:41 +0000

Frank Filz wrote:
>Yes, always translating ENXIO to eof is true seems to work, here’s the code in >Ganesha:
Note that I don't think the Linux client will ever do a Seek operation where
sa_offset == filesize. Does pynfs test this case?

>                info->io_content.hole.di_offset = ret;
If the Linux lseek(2) ever returns ENXIO when the offset < filesize, this would
create a reply with eof == true but sr_offset < filesize, which seems bogus to me?
(Actually, for whence == SEEK_DATA, I don't see io_content.hole.di_offset getting
 set at all.)
Clients probably don't care/notice what sr_offset is when eof == true.

(This may never happen in practice. For UFS on FreeBSD, there is always data after
 the last hole. I don't know about other file systems.)
 --> In other words, for UFS on FreeBSD, ENXIO never gets returned when
       offset < attrs.filesize

>And here’s how the FSAL function is invoked which shows the rest of the setup for >the NFSv4.2 SEEK response:
>
>I’m not sure that code is perfectly clean, but it passes the pynfs tests at least. I didn’t >really touch our protocol layer code, I just implemented the FSAL_VFS code.
It would be interesting to know if pynfs tests the case of sa_offset == filesize?

rick

On Sat, Aug 10, 2019, 5:44 PM Rick Macklem <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Why is it unclear?

Just as you found.

---

From: Rick Macklem <[email-masked]>
Date: Tue, 13 Aug 2019 03:24:54 +0000

>whatever you do with all the clients you expect to use SEEK...
The problem are the clients I won't be able to do testing against.
Also, by the time I hear about problems from others that run into problems,
both the FreeBSD and other client code is shipping to users.

For this case, if it can't be resolved what the correct reply needs to be, a
client author (like me for FreeBSD) can avoid the problem by making sure
the client never sends a Seek with sa_offset == filesize to a server.

>since we can't guarantee atomicity between lseek and stat.
I don't think this matters. Even if you have atomicity between filesize and
doing the Seek op., there is no guarantee that a concurrent writer won't change
the file before the client receives the reply.
Clients/applications using the Seek op. will not work correctly if there is a
concurrent writer and will have to deal with that.
All a server can do is create a correct reply assuming no concurrent writers for
the file.

>client, but since merging it last September, we haven't done any additional
At the least, I'd suggest you look at packet traces in wireshark and try to make
sure what is on the wire conforms to the RFC.
If you think the RFC is ambiguous, then I suggest you drive the poor RFC authors
crazy by asking questions like I do. (Dave Noveck has suffered through many of
these from me over the years. Sorry Dave.)

rick

Frank

file_size (or
code in
where
would
bogus to
when eof
data
other words,
setup
at least. I
FSAL_VFS
filesize?
file_size (or
request
do.
at
(e.. g.
moves the
faced
than
(at
well
sr_eof

---

From: Rick Macklem <[email-masked]>
Date: Thu, 22 Aug 2019 20:50:26 +0000

Since I haven't seen further responses to this, it seems that Frank and
David agree that, for Seek with offset == file_size, the correct reply is
NFS_OK, eof == true, offset == file_size.

I have coded the FreeBSD NFSv4.2 server that way, but will try to make sure
that the FreeBSD NFSv4.2 client also handles a NFS4ERR_NXIO reply for this
case, since that is what the Linux knfsd appears to do.

Thanks for the comments, rick

________________________________________
From: nfsv4 <[email-masked]> on behalf of Rick Macklem <[email-masked]>
Sent: Monday, August 12, 2019 12:04:41 PM
To: Frank Filz; 'David Noveck'
Cc: 'NFSv4'
Subject: Re: [nfsv4] NFSv4.2 behaviour for SEEK when sa_offset == file_size (or at EOF, if you prefer)

rick

Why is it unclear?

Just as you found.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4