Subject: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 17 Dec 2007 10:29:34 -0500

I think we need to figure out what we are going to do to address the
various remaining issues regarding NFS4ERR_FILE_OPEN.
 
First I think there is agreement that when, because of an open file, we
don't allow a REMOVE to delete a file entry, the error NFS4ERR_FILE_OPEN
should be returned.
 
Although there has not been any disagreement expressed, I am not sure
there is consensus as to Peter Varga's idea that FILE_OPEN is intended
to be used when non-NFSv4 open's prevent a remove/rename/link.  The
reason that this issue is important for the spec is that if that is so,
then servers won't ever have to recall delegations to ascertain whether
such opens exists.  So the question remains whether servers may choose
to exclude such operation because of the presence NFSv4 opens other than
due to DENY-{WRITE,BOTH} opens on the target of a REMOVE or a
renamed-over file or whether the spec should limit FILE_OPEN to that
specific case plus any cases necessary for a multi-protocol server to be
compliant with requirements of other protocols.  
 
The second issue has to do with the semantics mentioned in REMOVE (and
by implication I think it applies to files removed due to rename-over)
regarding NFSv4 opens that are DENY-{WRITE,BOTH}.  Peter has given some
good arguments about why this was a poor choice made by RFC3530, but my
feeling is that they are not so strong as to justify changing things at
this late date.  I'd like to hear other opinions on this issue, though.
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 17 Dec 2007 11:11:54 -0500

On Mon, Dec 17, 2007 at 10:29:34AM -0500, Noveck, Dave wrote:
> such opens exists.

Servers have to recall delegations on remove, rename, and unlink anyway,
don't they?  I'm not sure if I understand your point.

--b.

> So the question remains whether servers may choose

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 17 Dec 2007 18:04:15 -0600

> don't they?  I'm not sure if I understand your point.

Server should recall delegations when they would otherwise allow
the remove, rename, unlink.  I believe Dave is thinking of the case
where the server knows a-priori that the remove/rename/link would
not succeed without having recall a delegation.

No reason to recall a delegation if it isn't going to succeed in any  
case.

Spencer

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 17 Dec 2007 19:07:40 -0500

On Mon, Dec 17, 2007 at 06:04:15PM -0600, Spencer Shepler wrote:
> No reason to recall a delegation if it isn't going to succeed in any case.

Hm, so the concern is that server might have to recall delegations
just to decide *which* error to return on the remove/rename/link?

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 17 Dec 2007 18:15:15 -0600

> just to decide *which* error to return on the remove/rename/link?

It appears that this is the essence of what Dave is attempting to
determine and that Peter's opinion is that the NFSv4 protocol is
insufficient to obstruct such a remove of open file meaningfully.
Dave would just like to have the language/interpretation clear.

Of course, this is my interpretation. :-)

Spencer

---

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 17 Dec 2007 20:41:55 -0500

Those cases normally concern recalling them when the operation actully
happens, i.e. succeeds, and that part is written anc check im.

The case where you may have to recall delegations to determine whether
they are allowed to proceed or to return an NFS4ERR_FILE_OPEN error is
diferent, or least it seems to require spearate discussion in the text
of the spec. 

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Monday, December 17, 2007 11:12 AM
To: Noveck, Dave
Cc: [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

On Mon, Dec 17, 2007 at 10:29:34AM -0500, Noveck, Dave wrote:
> First I think there is agreement that when, because of an open file, 
> we don't allow a REMOVE to delete a file entry, the error 
> NFS4ERR_FILE_OPEN should be returned.
>  

> reason that this issue is important for the spec is that if that is 
> so, then servers won't ever have to recall delegations to ascertain 
> whether such opens exists.

--b.

> So the question remains whether servers may choose to exclude such 
> operation because of the presence NFSv4 opens other than due to 
> DENY-{WRITE,BOTH} opens on the target of a REMOVE or a renamed-over 
> file or whether the spec should limit FILE_OPEN to that specific case 
> plus any cases necessary for a multi-protocol server to be compliant 
> with requirements of other protocols.
>  

> regarding NFSv4 opens that are DENY-{WRITE,BOTH}.  Peter has given 
> some good arguments about why this was a poor choice made by RFC3530, 
> but my feeling is that they are not so strong as to justify changing 
> things at this late date.  I'd like to hear other opinions on this
issue, though.

---

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 17 Dec 2007 21:02:59 -0500

Or whether to return an error at all.  If there is a delegation and the
semantics are such that NFSv4 opens that can prevent the
remove/link/rename may exist and not be known to the server (but only
the client with the delegation), then the delegation may to be recalled
for you to determine whether the operation will go through.  If it does
go through, then upu have recalled delegation in adavnce, as it were.

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Monday, December 17, 2007 7:08 PM
To: Spencer Shepler
Cc: Noveck, Dave; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

>>> Although there has not been any disagreement expressed, I am not 
>>> sure there is consensus as to Peter Varga's idea that FILE_OPEN is 
>>> intended to be used when non-NFSv4 open's prevent a 
>>> remove/rename/link.  The reason that this issue is important for the

>>> spec is that if that is so, then servers won't ever have to recall 
>>> delegations to ascertain whether such opens exists.
>>
>> Servers have to recall delegations on remove, rename, and unlink 
>> anyway, don't they?  I'm not sure if I understand your point.
>
> Server should recall delegations when they would otherwise allow the 
> remove, rename, unlink.  I believe Dave is thinking of the case where 
> the server knows a-priori that the remove/rename/link would not 
> succeed without having recall a delegation.
>
case.

Hm, so the concern is that server might have to recall delegations just
to decide *which* error to return on the remove/rename/link?

--b.

>>> to exclude such operation because of the presence NFSv4 opens other 
>>> than due to DENY-{WRITE,BOTH} opens on the target of a REMOVE or a 

>>> specific case plus any cases necessary for a multi-protocol server 
>>> to be compliant with requirements of other protocols.
>>>
>>> The second issue has to do with the semantics mentioned in REMOVE 
>>> (and by implication I think it applies to files removed due to 
>>> rename-over) regarding NFSv4 opens that are DENY-{WRITE,BOTH}.  
>>> Peter has given some good arguments about why this was a poor choice

>>> made by RFC3530, but my feeling is that they are not so strong as to

>>> justify changing things at this late date.  I'd like to hear other
opinions on this issue, though.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 17 Dec 2007 21:39:02 -0500

On Mon, Dec 17, 2007 at 09:02:59PM -0500, Noveck, Dave wrote:
> Or whether to return an error at all.

I'd assumed that delegations by definition conflicted with rename, link,
and remove.  Certainly they have to on v4.0, since CLAIM_DELEG_CUR
requires a pathname.  And on 4.1 the server must recall delegations on
link and remove on a server that supports the numlinks attribute.

But on 4.1, with rename, or with link/remove on a server that doesn't
support numlinks, I guess there's no conflict any more--I hadn't
realized that.

That's a subtle change in the delegation semantics.  The rationale would
be worth documenting (maybe in OPEN documentation?).  (Or maybe it's
there and I've missed it.)

Hm--there's a typo in the table of open types--see appended diff.

--b.

diff --git a/nfsv41_middle_op_open.xml b/nfsv41_middle_op_open.xml
index e5608b6..aa266f8 100755
--- a/nfsv41_middle_op_open.xml
+++ b/nfsv41_middle_op_open.xml
@@ -127,7 +127,7 @@
       </c>
       <c>
         CLAIM_DELEGATE_CUR
-        CLAIM_DELEG_PREV_FH
+        CLAIM_DELEG_CUR_FH
       </c>
       <c>
         The client is claiming a delegation for OPEN as granted by the

---

From: Mike Eisler <[email-masked]>
Date: Mon, 17 Dec 2007 20:34:37 -0800 (PST)

> Hm--there's a typo in the table of open types--see appended diff.

Thx. Fixed it just now.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Tue, 18 Dec 2007 16:21:04 -0500

> link and remove on a server that supports the numlinks attribute.

By the way, despite that I should admit that the Linux server isn't
actually doing that right now.  I've resisted fixing this in nfsd
(ignoring renames, links, and removes from other sources) in hopes of
fixing the vfs so it will work for all users, and David Richter's done
some work on this, but it's proved harder than expected.  (But clearly
it is a bug and I hope it'll be fixed soon....)

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 18 Dec 2007 15:24:57 -0600

> it is a bug and I hope it'll be fixed soon....)

FYI.

The Solaris server will recall delegations prior to these operations
success and this accounts for local access as well.

Spencer

---

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 18 Dec 2007 16:27:26 -0500

> there and I've missed it.)

Thanks for pointing this out.  I am writing text that describes when
delegations
get recalled so I think the case of numlinks not being supported would
need to be 
mentioned there.

I'm not inclined to try to stress the differentce with 4.0 because they
are 
subtle and complicated.  For example, if I have a file with links A and
B and
I have a delegation for B, and I rename A to C, then I would think there
would
not be a recall in 4.0 either.  It's hard enough to describe exactly
what happens
in v4.1.

I have to think about this some more but I think that if you open A and
get 
a delegation as a result, then if A is renamed that the delegation
should be
recalled.  On the other hand, if the filehandle of A is passed to
WANT_DELEGATION
and I get a delegation then if A is renamed then the delegation is not
recalled.
Or is it?  Should the semantics be that if either A or B is renamed,
then the
delegation is recalled? 

Issue 24 in the issue tracker seems to bringing forth more subtleties
than I 
had expected.  I guess that's a good thing, or at least I will think so
then 
the writing is finished and draft-18 is submitted.

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Monday, December 17, 2007 9:39 PM
To: Noveck, Dave
Cc: Spencer Shepler; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 18 Dec 2007 15:43:32 -0600

> Or is it?  Should the semantics be that if either A or B is renamed,

I would prefer that the delegation is recalled.  I may have a client
that chooses to do recovery by filename but also chooses to use
the WANT_DELEGATION.  It would want to be notified of the name change
even if it appears to be using the filehandle.  This eases the  
implementation
burden on a client that is attempting to do both 4.0 and 4.1.

Spencer

---

From: Trond Myklebust <[email-masked]>
Date: Tue, 18 Dec 2007 16:45:02 -0500

> Or is it?  Should the semantics be that if either A or B is renamed,

If the filehandle is the same for A and B, then clients will certainly
expect to have the delegation recalled if either filename is
deleted/renamed. Otherwise we ought to make CLAIM_DELEGATE_CUR mandatory
to not implement since we'd be making it impossible for the client to
know when it is safe to use.

Trond

---

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 18 Dec 2007 16:55:06 -0500

I've been thinking about this a bit more and I think we have three
choices but only one good one or perhaps I should say a maximum of one
good one:

    1) A delegation gotten with open by path component protects
       that path component's mapping to fh (in the sense that 
       RENAME results in delegation recall), while a delegation 
       gotten by fh (e.g. WANT_DELEGATION) does not.

    2) Neither way of getting a delegation protects the path
       component's mapping to fh and RENAME does not result in
       delegation recall.

    3) Whenever a delegation is gotten (by either method) all the
       names of the fh are protetcted and RENAME of any of them 
       result in delegation recall.

I think (1) is too complex and error-prone in that you have two sorts of
delegation that are very subtly different.

I think a delegation that doesn't protect the name-fh binding is subject
to subtle bugs, so (2) is not good.

This leaves (3) as the right choice, although I would not have assumed
so a priori.  It does have the counterinutive effect that if I have
delegation gotten under some other name and I link that file to have
name C and then rename C to be D, the delegation is recalled, but I'm
not inclinded to make things more complicated to deal specially with
that issue.

Comments?

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Monday, December 17, 2007 9:39 PM
To: Noveck, Dave
Cc: Spencer Shepler; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

--b.

---

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 18 Dec 2007 16:57:52 -0500

It appears that Spencer and Trond are respectively 9 and 12 minutes
ahead of me.

-----Original Message-----
From: Noveck, Dave 
Sent: Tuesday, December 18, 2007 4:55 PM
To: Noveck, Dave; J. Bruce Fields
Cc: Spencer Shepler; [email-masked]
Subject: RE: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

Comments?

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Monday, December 17, 2007 9:39 PM
To: Noveck, Dave
Cc: Spencer Shepler; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

--b.

---

From: Trond Myklebust <[email-masked]>
Date: Tue, 18 Dec 2007 17:10:26 -0500

> It appears that Spencer and Trond are respectively 9 and 12 minutes

That's not necessarily a bad thing, as long as we all agree :-)

Trond

---

From: "J. Bruce Fields" <[email-masked]>
Date: Tue, 18 Dec 2007 17:25:19 -0500

> It appears that Spencer and Trond are respectively 9 and 12 minutes

Uh, and I think I'm just now catching up.  OK, so it sounds like a
delegation should cover all links to that file, got it.

That still leaves me a little confused as to what the rationale for
adding CLAIM_DELEG_CUR_FH and CLAIM_DELEG_PREV_FH was, though.  Could we
get a sentence or two in the spec explaining their use?  (I assume
servers are required to support them?  Any advice on when clients should
use them?)

By the way, in 10.4, it says:

	" When a client has a read open delegation, it may not make any
	changes to the contents or attributes of the file but it is
	assured that no other client may do so."

But I've seen clients modify files without returning delegations first,
and that seems completely reasonable to me.  If a server knows this is
the only client with a read delegation, but doesn't want to give it a
write delegation for whatever reason, why shouldn't it let the one
client with a read delegation modify the file?  Any chance we could make
that just:

	"When a client has a read open delegation, it is assured that no
	other clients may make changes to the contents or attributes of
	the file."

Also it's not just other clients that are prohibited from making
changes, so perhaps:

	"While a client has a read open delegation, it is assured that
	neither the contents nor the attributes of the file will change."

And with the above modification to cover links:

	"While a client has a read open delegation, it is assured that
	neither the contents, nor the attributes, nor the names of any
	links to the file will changes."

Is that reasonable?

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Tue, 18 Dec 2007 17:34:35 -0500

> 	links to the file will changes."

Eh, I just contradicted myself a little.  Maybe something like:

	"While a client has a read open delegation, it is assured that
	neither the contents, nor the attributes, nor the names of any
	links to the file will change without its knowledge."

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 18 Dec 2007 17:11:32 -0600

Just consider a pre-comment. :-)

---

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 19 Dec 2007 10:02:16 -0500

> client with a read delegation modify the file?  

You're right.  In fact the SETATTR IMPLEMENTATION section now ("now"
here means in my CVS directory :-) says.

    <t>
      If the object whose attributes are being changed has a file
delegation 
      which is held by a client other than the one doing the SETATTR,
      the delegation(s) must be recalled, and the 
      operation cannot proceed to actually change an attribute 
      until each such delegation is returned 
      or revoked.  
      In all cases in which delegations are recalled, the server
      is likely to return one or more NFS4ERR_DELAY error while the
      delegation(s) remains outstanding, although it may, if the
      returns happen quickly, not do that.
    </t>

I think WRITE is consistent with that but I'll check.

> Any chance we could make

Will do.

> Eh, I just contradicted myself a little.  

I know the feeling.  The spec does that to you.

> Maybe something like:

>	links to the file will change without its knowledge."

That sounds right to me.

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Tuesday, December 18, 2007 5:35 PM
To: Noveck, Dave
Cc: Spencer Shepler; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 27 Dec 2007 15:26:58 -0500

> >	links to the file will change without its knowledge."

Stupid nitpick--should there be a statement somewhere that time_access
is an exception?  I couldn't find one on a quick skim.

--b.

---

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 27 Dec 2007 15:37:56 -0500

Nitpick but not a stupid one.  I think people have been assuming this is
an exception but the spec should state it clearly. 

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Thursday, December 27, 2007 3:27 PM
To: Noveck, Dave
Cc: Spencer Shepler; [email-masked]
Subject: Re: [nfsv4] Getting to closure to NFS4ERR_FILE_OPEN

--b.