Subject: [nfsv4] rfc5661 errata 3901, was: Re: [PATCH 09/18] nfsd: implement pNFS operations

From: Christoph Hellwig <[email-masked]>
Date: Fri, 9 Jan 2015 19:49:43 +0100

On Fri, Jan 09, 2015 at 09:43:41AM -0800, Trond Myklebust wrote:
> >     [nfsv4] NFSv4.1 errata id 3226 (the return of return-on-close layouts)
> >
> > ...on thee [email-masked] mailing list. Trond mentions it there.
> > Perhaps we need to revise that errata?
> >
> 
> Please see:
>      http://www.rfc-editor.org/errata_search.php?eid=3901

I think the language in this errata is very confusing, especially:

"After the client has closed all open stateids and returned the
delegation stateids for a file for which logr_return_on_close
was set to TRUE, the server MUST  invalidate all layout segments
that were  issued to the client for that file."

While the idea that return on close layouts should be valid as
long as the "parent" stateid is around make a lot of sense, this
requirement to track all open / delegation stateids per file/client
combination seems insane.

The only logical way to extend the original text is to require
layouts to be implicitly returned when:

 - the file is close for a layout stateid that is created based on
   the open or lock stateid
 - the delegation is returned for a layout stateid that is created
   based on the delegation stateid.

That is, only keep layouts opened based on the delegation stateid
alive over a close if they are hanging off that delegation stateid.

---

From: Tom Haynes <[email-masked]>
Date: Wed, 14 Jan 2015 11:16:27 -0800

> combination seems insane.

Christoph,

I don't understand this concern. Section 8.2.1:

open stateids: OPEN state for a given client ID/open-owner/filehandle triple

delegation stateids: A stateid represents a single delegation held by a client for a
      particular filehandle.

By definition, open/delegation stateids are tracked per
file/client combination.

>  - the file is close for a layout stateid that is created based on

I read you as saying that on the first CLOSE, the layout must
be returned. This seems very burdensome in that the client is aware
that other OPENs may have occured and expects to be able to still
utilize the layout. Under this new model, it would need to get
the layout again.

I.e., Trond's orginal concern with Section 18.43.3 is just that,
the text states that:

   The logr_return_on_close result field is a directive to return the
   layout before closing the file.

Paraphrasing the errata, this could be rewritten as

   The logr_return_on_close result field is a directive to return the
   layout before the last close of the file.

>    based on the delegation stateid.

This agrees with the paragraph above.

> alive over a close if they are hanging off that delegation stateid.

Thanks,
Tom

---

From: Christoph Hellwig <[email-masked]>
Date: Thu, 15 Jan 2015 17:26:18 +0100

> By definition, open/delegation stateids are tracked per

My concern is that the language in errata 3901 says the server should
invalidate all layouts after all open stateids are closed, and all
delegation stateids are returned for a given file, which means the
server needs to add another object just to track the layouts.  If
on the other hand we say the lifetime of the layouts is tied to
the open stateid or the delegation stateid used to create the
layout stateid we can track the outstanding layouts in those
open/delegation stateid (and the lock stateid as well).

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 15 Jan 2015 11:30:14 -0500

> open/delegation stateid (and the lock stateid as well).

The problem, in my mind, is that defeats the main purpose of
return-on-close, which is to reduce the on-the-wire chattiness of the
protocol. How one implements that internally on the server is not
really a concern that should be reflected in the protocol.

Cheers
  Trond