Subject: Re: [nfsv4] layout return - final update ?

From: Benny Halevy <[email-masked]>
Date: Thu, 01 Mar 2012 12:03:56 +0200

On 2012-03-01 05:16, Rick Macklem wrote:
> Marc Eshel wrote:
>> [email-masked] wrote on 02/29/2012 04:53:47 PM:
>>>
>>> Trond Myklebust, nfsv4
>>>
>>> Marc Eshel wrote:
>>>> Lets try to keep it simple. Can we agree that once the server
>>>> returns
>>>> a
>>>> layout with return_on_closed == true for a file A
>>>
>>>> (what ever way it is represented)
>>> If, by "what ever way it is represented" you mean a file
>>> handle, then that sounds fine to me.
>>
>> Yes, any fh that maps to the same file (inode). btw, files might also
>> have
>> different fh becuse of hard links not just rename.
>>
> Well, I think this is too hard for a client to handle. (ie.
> Recognizing that multiple FH's are for the same file.)
> 
> When I said FH, I meant that the rule would apply to one FH,
> not all FHs that map to the same file.

Agreed.  The layout state is tied to a particular filehandle using
which the client sends LAYOUTGET, LAYOUTCOMMIT, LAYOUTRETURN, and
it is the same filehnadle referred by CB_LAYOUTRECALL via lor_fh.

I expect that the server will issue different layout stateids if LAYOUTGET
is called on different filehandles even if they refer to the same file
(identified with a unique fsid:fileid tuple) on the server.
To simplify matters I believe we can require that the server MUST do so.

Benny

> 
> rick
> 
>>>
>>> If you mean "any file with the same fileid and fsid from
>>> the same server", I'm not so sure, since it would be difficult
>>> to find all cases of this, imho. Yes, fileid and fsid are
>>> attributes,
>>> but a client tends to cache them (with no guarantee that
>>> they are up-to-date) and even an exhaustive search of the attribute
>>> cache might not find them all. (I believe this is only an issue for
>>> cases where a file has multiple FH's, possibly after a RENAME
>>> against certains servers?)
>>>
>>> rick
>>>
>>>> to client B the serve will be allowed to invalidate the
>>>> layouts on the last close of file A by client B. If you disagree
>>>> please
>>>> propose a clear and complete alternative.
>>>> Marc.
>>>>
>>>>
>>>>
>>>> From:
>>>> Rick Macklem <[email-masked]>
>>>> To:
>>>> Trond Myklebust <[email-masked]>
>>>> Cc:
>>>> [email-masked], [email-masked], david noveck
>>>> <[email-masked]>
>>>> Date:
>>>> 02/29/2012 03:48 PM
>>>> Subject:
>>>> Re: [nfsv4] layout return - final update ?
>>>>
>>>>
>>>>
>>>> Trond Myklebust wrote:
>>>>> On Wed, 2012-02-29 at 15:30 -0500, [email-masked] wrote:
>>>>>>> Previously what we discussed was the last CLOSE for the
>>>>>>> file/client
>>>>>> pair.
>>>>>>
>>>>>>
>>>>>> OK, but the proposed text you supplied is not clear on that
>>>>>> point.
>>>>>>
>>>>>>
>>>>>>> That said, I'm not sure about multiple filehandles
>>>>>>> representing
>>>>>>> the
>>>>>> same file.
>>>>>>
>>>>>> Another issue that needs to be clarified in the text.
>>>>>
>>>>> We should do the same as we do for other state: LAYOUTs for
>>>>> different
>>>>> filehandles should be represented by different stateids and
>>>>> should
>>>>> not
>>>>> be combined.
>>>>>
>>>>> IOW: since you can't choose to CLOSE the file using either
>>>>> filehandle,
>>>>> but MUST use the filehandle that you received when you did the
>>>>> OPEN,
>>>>> I
>>>>> suggest that we extend that principle to layouts too.
>>>>>
>>>> Isn't it also possible to have multiple OPENs for the same FH, but
>>>> different open_owner4's?
>>>>
>>>> (Just fyi, this doesn't affect the FreeBSD
>>>> client much, since it holds onto all OPENs for a given FH until
>>>> all
>>>> of the file descriptors are closed and the CLOSEes them all at the
>>>> same time. As such, the layout can be returned at that time, if
>>>> any
>>>> case of return_on_closed == true has been received for the
>>>> layout.)
>>>>
>>>> rick
>>>>
>>>>>>> This approach seems valid but it complicates the case of
>>>>>>> layouts
>>>>>>> not
>>>>>> marked as return_on_close that survived a previous last CLOSE
>>>>>> as
>>>>>> these
>>>>>> will not be affected now.
>>>>>>
>>>>>>
>>>>>> If we are talking about layout not marked return-on-close that
>>>>>> were
>>>>>> sent before the first return-on-close layout, then I think
>>>>>> that
>>>>>> the
>>>>>> text you proposed is exactly the same in this respect. Or am I
>>>>>> missing something here?
>>>>>>
>>>>>>
>>>>>>> Since we are talking about losing the layout on the last
>>>>>>> CLOSE
>>>>>>> from
>>>>>> the client of that file a catch all approach would be simpler.
>>>>>>
>>>>>> I'm not sure that that is so. First of all, it is very
>>>>>> natural/required for clients and servers to have data
>>>>>> structures
>>>>>> representing open files, and linking return-on-close layout to
>>>>>> these
>>>>>> seems relatively straightforward. on the other hand, I would
>>>>>> not
>>>>>> expect clients and servers to always have an infrastructure to
>>>>>> keep
>>>>>> track of client/fh or client/file pairs and thus always be
>>>>>> able to
>>>>>> see
>>>>>> when a last close is being done. To top it all off, you may
>>>>>> have racing closes so last-close-at-issue may not be the same
>>>>>> as
>>>>>> last-close-at-receipt. Also closes and opens can race. Those
>>>>>> sorts
>>>>>> of considerations led to me thinking associating a layout with
>>>>>> a
>>>>>> specific open file was simpler or at least it made my head
>>>>>> hurt
>>>>>> less.
>>>>>
>>>>>
>>>>> --
>>>>> Trond Myklebust
>>>>> Linux NFS client maintainer
>>>>>
>>>>> NetApp
>>>>> [email-masked]
>>>>> www.netapp.com
>>>>>