Subject: [nfsv4] re: byte range delegations

From: [email-masked]
Date: Sat, 16 Jul 2005 14:40:27 -0400 (EDT)

When you have multiple readers and writers of the same file, I see three
major variants w.r.t. byte ranges:
1 - non-overlapping byte ranges (or all readers). Your proposed protocol would
    seem fine for that. However, since the byte ranges are non-overlapping,
    there isn't a consistency problem anyhow. (The protocol does, however,
    provide the assurance that there is no consistency problem.)
2 - multiple writers (and possibly also readers) of the same byte range.
    In this case, your protocol would result in what I believe Mike was
    concerned about. That is, the clients will repeatedly ask for Write
    delegations, either waiting for the other writer or forcing the
    server to recall the other writer's delegation. To me, the best
    thing that can happen in this case is that the clients are told to
    do write-through by the protocol and will need to use some other
    mechanism, such as byte range locks, to synchronize ordering of
    the write operations.
3 - one writer, multiple readers. This is where I think a cache consistency
    protocol can be a win. It can notify the readers that there is a writer,
    thus requiring them to check with either the server or the writer to
    see if cached data is "up to date". (Unfortunately, "Change" changes for
    attribute changes as well, but shouldn't work too badly for this.)
    I believe DFS notified the readers who the writer was, so they could
    check with it. Alternately, the simpler approach is for the server to
    keep "up to date" by the protocol requiring the writer to use
    synchronous write operations. (ie write-through)

    As I understood your draft, it doesn't seem to handle case 3 this way.
    It states that, when a Read delegation is recalled, that it must flush
    cached data. (A better approach might be a callback that downgrades
    the read delegation to "must keep checking to see if cached data is
    up to date", then another callback that reverses that when the write
    delegation is released.)

In other words, I think it might be better if the delegations indicated
to the server what it "wished to cache" and the server provide callback
notifications that let the clients know if:
- for readers, check for currency before using cached data
- for writers, switch to "write-through", since there are other readers
  and/or writers

If you want to see what I tried to do w.r.t. cache consistency, you can
take a look at http://snowhite.cis.uoguelph.ca/nfsv4/nqnfs-paper.pdf.
Part of the reason I asked the retorical question is that "no one seemed
to give a rats ass about consistency then and I'm not sure things have
changed much since then".:-)

just mho, rick

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Sat, 16 Jul 2005 18:13:29 -0400

>     synchronous write operations. (ie write-through)

How did one client authenticate to another in the DFS scheme?

>     cached data. (A better approach might be a callback that downgrades

The server doesn't have to recall the delegation on the full set of
cached data. Only the area where there is a conflict.

>     up to date", then another callback that reverses that when the write

That scheme very much relies on clients writing out data synchronously.
My intention was to allow the writer to cache until a reader actually
needs to page in that data.

> - for writers, switch to "write-through", since there are other readers

I'm not sure that I understand the scheme. When would the server want to
notify the client without the data actually changing?

> changed much since then".:-)

I recently had to help debug an "NFS client problem" that turned out to
be due to an incorrect implemention of dotlocking. As far as ensuring
exclusive access to the file was concerned, the implementation was
perfectly correct, but it was locking immediately _after_ opening the
file instead of before...
My point is that even application programmers that recognise the need
for synchronisation are often hazy on the exact mechanics behind the
close-to-open consistency model.

Cheers,
  Trond

---

From: Craig Everhart <[email-masked]>
Date: Mon, 18 Jul 2005 15:01:19 -0400

Trond Myklebust wrote on 07/16/2005 06:13:29 PM:

> >     protocol can be a win. It can notify the readers that there isa
writer,
> How did one client authenticate to another in the DFS scheme?

It doesn't.  The DFS scheme doesn't have clients communicating directly
with
each other.  They communicate through the server, tending to the thrashing
behavior that several folks alluded to.  Let a thousand flowers bloom:
recommending that the contention be resolved further downstream, on the
server,
for this case, is a pretty good idea.

The DFS server doesn't insist that a client do write-through.  But it does
do "delegations" for aspects of the file separately: data cache, status
cache.
This leaves room for improvement over the proper caching of EOF...

>   Trond

            Craig

Craig Everhart
+1 919 543 2169 (tie 441 2169)