Subject: [nfsv4] re: byte range delegations

From: [email-masked]
Date: Thu, 14 Jul 2005 11:28:26 -0400 (EDT)

I have to question the wisdom of more complex delegations at this time.
(I'm not referring to the details of the proposal, but the generic concept.)

The read and write delegations we currently have were described in
RFC3010 dated Dec. 2000. At this point clients are just starting to
use those delegations (and some might argue that they haven't figured
out how to use them effectively yet).

Does it make sense to start defining a more complex delegation model
at this point?

Are people clammering for better cache coherency and arguing that byte
range locks aren't sufficient for the cases they care about?

I'm just a big believer in the KISS principal and would like to see
further complexity avoided unless there is a really good argument for it.

rick

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 14 Jul 2005 13:51:03 -0400

> range locks aren't sufficient for the cases they care about?

That's a very rhetorical question.

        The people that are happy to use NFSv2/v3 today? Probably not so
        much.

        The people that chose to go with a clustered filesystem instead?
        Yes, they probably feel quite strongly about the issue.

8-)
---

More seriously:

In the draft we list two commonly used operations that NFS currently
cannot support properly and that are directly linked to the coherency
issue:

 - shared mmap()
 - append writes

There are quite a few applications out there that rely on those working
correctly in a multi-threaded environment. If we want to help port those
applications to a clustered environment without forcing people to
redesign their entire program around the I/O quirks of the filesystem we
happen to have on offer, then we need to be able to present them with
something that goes beyond close-to-open cache consistency.

We also make the point that byte range locks cause you to take a huge
performance hit. They basically only work in an environment where you
are using uncached I/O. If you are the sole user of the file, then file
delegations help. If you are not, then irrespective of whether the
processes are behaving in a manner that causes caching conflicts, you
will still be reduced to doing synchronous I/O.

Finally, I assume that pNFS with non-files based back ends (e.g. objects
and blocks) may want to be able to do atomic read-modify-write cycles on
their data in order to support direct writes and such.

Cheers,
  Trond

---

From: David Robinson <[email-masked]>
Date: Thu, 14 Jul 2005 15:22:23 -0500

> something that goes beyond close-to-open cache consistency.

While it is true that multi-threaded applications will rely on
byte range consistency, I have yet to find any significant cases
where there are multi-client applications that need this
level of consistency. Generally speaking moving a single process
application to a multi-process or multi-client application will
need to have some sort of inter-node synchronization beyond the
filesystem. I am very interested in the existence of any interesting
number of such applications.

> will still be reduced to doing synchronous I/O.

There is no requirement that files that are locked must do uncached
I/O. Most implementations are written that way because that is
the easiest way to maintain correctness. For many years Solaris
does cached I/O for files that contain byte range locks that
encompass the entire file. It would not be difficult to increase
the granularity to caching data for any lock ranges that were
multiples of a page size. Just never had the demand to justify
the work. I personally have designed sub-page sized cached I/O
using some of the features of Solaris, but it is more complex
and the demand isn't there and the performance gains are
questionable.

While byte range delegations are interesting from a theoretical
perspective, I tend to back Mike and strongly question if there is
demand to justify the effort.

	-David

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Thu, 14 Jul 2005 15:22:04 -0700

David Robinson wrote:
> > correctly in a multi-threaded environment.
> 
> level of consistency.
> . . .

All true. But, that NFS cache coherency has historically
been so weak does adequetely explains why there are no
such applications in NFS environments.

Once upon a time there were lots of such applications
in DOS/LANManager/Netware environments because there
was adequate coherency (and attendant performance
penalties) and it was the only way to do multiuser
applications.

It's a bit of chicken-and-egg.

All that said, the page has turned for multiuser
applications. It is an era of SQL servers and such.
Multiple simultaneous actors on a single file,
especially on a networked/remote file system,
seem rather rare.

I share doubts there are broad-based requirements
for the sorts of features byte-range delegations
provide. There may be niche requirements, but
broad-based.....ehhh. I don't think the history
of NFS makes a compelling case that such requirements
do not or would not exist. I do agree that representative
requirements should be presented and analyzed to
be sure the right thing -- if anything -- gets built.

Regards,
	-gww

---

From: David Robinson <[email-masked]>
Date: Thu, 14 Jul 2005 17:42:42 -0500

> It's a bit of chicken-and-egg.

True. But while NFS can be used as a way to present data
to a mail application (think /var/mail), IMAP is
clearly the better protocol. So it is not just
chicken-and-egg, but I fear it may be I've got
a hammer and all the world is a nail.

> especially on a networked/remote file system,

I am no database expert, but my understanding is that
applications such as Oracle RAC already have the need
to coordinate between instances who owns what data
and go to great lengths to never cause conflicts that
would need to be resolved by a lower layer.

NFS has been successful because it has not tried to solve
all of the hard niche problems that others like DCE/DFS
did tackle. I am just not anxious to start back down that
path without some compelling need.

	-David