Subject: RE: [nfsv4] more re: client re-using lock_owner

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 20 May 2005 19:23:10 -0400

David Robinson wrote:
> Noveck, Dave wrote:
> > OK, I see.  Since the spec allows the server to forget owner state
> > when there are no associated opens, it follows that returning OK
> > and requiring open-confirm cannot be illegal, since a server which
> > has dropped that open state will do precisely that.
> > 
> > However, even though it is not illegal, it still strikes me as
> > a very bad thing to do.  You have enough information to figure
> > out that the seqid is wrong so that it just seems better to tell 
> > the client that, even though you might not be able to if you had
> > dropped that info.  With door #2 you have a server in which 
> > getting a bad owner seqid when you have no opens brings on an 
> > attack of amnesia, so you always respond as if you had dropped
> > the owner info.   
> 
> This seens unduly harsh. Section 8.1.8 says that the server upon
> seeing a lock_owner for the "first time" will require an
> OPEN_CONFIRM. But the spec fails to talk very clearly
> about what is the "first time". It is clear that while there
> is active state for that lock_owner it can't be the first
> time and the server MUST return back NFS4ERR_BADSEQID, the
> client has screwed up state that it is required to maintain.

> However, if there is no active state, how long a time period or
> what events must occur before the reuse of a lock_owner
> qualifies to be the "first time"? 
 
The spec gives no constraints on this.  A server may drop the
owner state immediately after the last close of an open for this
owner if it wants.  The only consequence of doing that would be
a lot of extra OPEN_CONFIRM's and thus bad performance.  It would
be legal however.  

In this case, we are talking about, where the hypothesis is that 
the server does indeed have information about the openowner, it 
is clear this is not the "first time" the server is seeing that
owner and the server knows that.  The only reason that this is 
not illegal is that judgments of spec legality are subject to 
an epistemological constraint.  They may not refer to details
of the internals of the server, even though we may very well
know about them.  If something is illegal it must determinable
from behavior visible over the wire.  But note that I am not
saying this is illegal.  I'm just saying it seems bad to me
and I think I may take into account what we know about the 
actual situation when we discuss those questions. 

> And how is a client expected
> to be able to reliably determine that fact?  

He doesn't need to.  He is free to drop owners at any time as long
as he doesn't reuse the owner string after dropping that owner,
and that isn't very hard to arrange.  If he is referring to a new
owner, and it would necessarily be a new owner if there is a new
sequence space, then the new owner must be associated with a new
opaque string.  You can't use the same opaque string for two
different owners.

> The other factor is
> that if the client had done something major like rebooted and
> lost any history, it may not have any way to construct a valid
> seqid so it must start with a new lock_owner. 

What does it use for a clientid?  If it does a SETCLIENTID, the
owner state is gone and he has a fresh slate.  If he wants to 
use the same clientid, then he can write it to disk, and pick it
up after the reboot. but if he wants to do that, he'd better also 
write the owner state persistently as well.  If he can't do that, 
he has to start fresh with a SETCLIENTID.

> Seemingly harmless
> to do such a thing, but I can imagine scenerios where a client
> would rather not keep trying new values until it hit one that
> worked. Eventually the server will forget that lock_owner and
> will think it is the "first time", so why not make that a
> more pro-active event?

It is easy to create a new lockowner all the time.  Just append the
time you created it and it will be unique.

The point here is that the client is not helpless.  The server will
only know about owners that the client himself created.  While there
are not an infinite set of possible owner strngs, there are eight to
the two to the thirtysecond, which is pretty close to infinity for
practical purposes.

> So I would like Door #2 Monty...

Is is legal.  You may go through the door.  If I say overly harsh
things about your choice, don't take it personally.  I just don't
think it is the right choice, even though I agree it is legal.

> Reasonable behavior should be:

We need to separate what someone thinks is reasonable from what
is in RFC3530.  If you think the spec should have allowed clients
to forget state at will, then that is one thing.  But I don't think 
the spec actually did that.

> 1) Client MUST use the correct next seqid if there is active state
>    for that lock_owner.
> 2) Client SHOULD use the correct next seqid if there is no
>    active state for that lock_owner.

Strictly speaking, the client may send any seqid he wants but if it
is not the correct one, he has to be prepared for BADSEQID.  If there
is open state and he uses an incorrect seqid, then he may rely on 
getting BADSEQID.  If there is no open, he may not rely on getting that
error, since the server may have forgotten the owner state.

> 3) The server MUST return NFS4ERR_BADSEQID if the OPEN lock_owner
>    has an incorrect seqid and there is active state for that
>    lock_owner.

Agree.

> 4) The server SHOULD allow the OPEN if the lock_owner has the
>    correct seqid and there is no active state for that lock_owner.

He must process the open normally.  Whether it allowed depends on
lots of stuff.

> 5) The server SHOULD (MUST?) return OPEN4_RESULT_CONFIRM if the
>    lock_owner has an incorrect seqid and there is no active state
>    for that lock_owner.

Why?  If you assert that this is somehow required by RFC3530, then
where exactly does it say that?  I think it says that if it knows
a seqid is invalid it should return BADSEQID.  I don't think the fact
that it might sometimes not know that changes that.  It may be
that 5) could have been added to the spec without harm, but it
wasn't.

As far as the (MUST?) alternative, how can you say you "SHOULD"
do the open if he has the correct seqid and "MUST" do it if he
has an incorrect seqid?

> So in normal processing everyone SHOULD make a reasonable effort
> to maintain enough information to minimize the need to use
> OPEN_CONFIRM. But in the absence of active state, if one side
> forgets, it is useful to allow the lock_owner to be reused
> by sending OPEN4_RESULT_CONFIRM instead of NFS4ERR_BADSEQID.

I don't see how it is useful.  The client should not reuse the
same opaque string for different owners and if he forgets the
seqid, then these are different owners. 

You may think so but the fact is that the spec discusses the
case of the server forgetting the stateid and never mentions
the case of the client forgetting it.  If you think that that is
an oversight, then you need an argument to show why this is a
necessary case to deal with.  I assert that it is not, since the
cases of the server and the client are not parallel.  The client
decides on the lifetime of owners and the server has no way of
of knowing when the client is done with one.  This requires special
handling to deal with the case of a server who is forced to drop
state.  The client doesn't have that problem.  He may drop owners
at will with no special protocol provision, as long as he doesn't
try to forget the seqid and then use that same owner string again.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: David Robinson <[email-masked]>
Date: Fri, 20 May 2005 19:46:41 -0500

> actual situation when we discuss those questions.

As the specification is written, it is perfectly legal to return
either NFS4ERR_BADSEQID or OPEN4_RESULT_CONFIRM. The client must
be prepared to handle either. I am not advocating that either
method is illegal, just that OPEN4_RESULT_CONFIRM would be
preferred.

>>to be able to reliably determine that fact?

The spec's use of 'first_time' may lead the reader to
expect that the client somehow tracks the server's notion
of 'first time', that doesn't seem reasonable nor reliable
to do. The whole issue of 'first_time' is simply an optimization
by the server to minimize OPEN_CONFIRM traffic. The minimal
required functionality is to OPEN4_RESULT_CONFIRM if there
is no active state.

My argument is that a server that cannot perform the optimization
because the seqid is not the correct next seqid is used, it should
should act as it were the starting from scratch and had not kept
track of the last seqid and reply with OPEN4_RESULT_CONFIRM.

The difference is that in one case the server replaces the notion
of the next valid seqid and in the other the client must create
a new unique open_owner. This would seem to be easier for the
server as it is not leaving behind 'old' open_owners to track
and better for the client as it is one less round trip by
saving the OPEN with new open_owner request.

[ignoring my bad and withdrawn example]

> think it is the right choice, even though I agree it is legal.

We agree that both are legal. I just find that if the server is
nicer the client has less work to do. Asking for an OPEN_CONFIRM
feels friendlier than issuing an NFS4ERR_BADSEQID.

> the spec actually did that.

I believe that the spec is agnostic to what the client and server
do with respect to maintaining open_owner seqid numbers when
there is no active state. The server can maintain nothing
always requiring an OPEN_CONFIRM and the client can use
any value but must be willing to accept a NFS4ERR_BADSEQID
error.

> error, since the server may have forgotten the owner state.

Agreed.

> He must process the open normally.  Whether it allowed depends on

Agreed, "allowed" is dependant on all the other stuff involved
in OPEN.  This case is the strong SHOULD that a server ought to
implement the optimization eliminating OPEN_CONFIRMs.

> do the open if he has the correct seqid and "MUST" do it if he

The (MUST?) is more of an editorial comment that the RFC doesn't require
this, but if I had thought of this scenerio a few years ago I would
have pushed to make this a MUST. But it is not required anywhere,
just as the server is not required to maintain the last valid seqid
when there is no active state just so it can return BADSEQID.

> seqid, then these are different owners.

A simple client may choose to use a simple value as the open_owner
and want to maintain very minimal state and do something trivial
like restart the seqid at zero if there are no open files.

In the BADSEQID case the client will need to do the following
operations:
	OPEN oo1 seq0 ->
		<- BADSEQID
	OPEN oo2 seq0 ->
		<- OPEN4_RESULT_CONFIRM
	OPEN_CONFIRM ->
		<- OK
This could be even worse if the choice of 'oo2' also conflicted
with some previous open_owner that the server is still tracking
the seqid for, triggering multiple trys by the client to get
a valid open_owner.

If the server is nicer, it can be reduced to:
	OPEN oo1 seq0 ->
		<- OPEN4_RESULT_CONFIRM
	OPEN_CONFIRM ->
		<- OK

The client doesn't need to be as aggressive at creating new
open_owners and the server doesn't have a growing list
of open_owner/seqid pairs to track to optimize a case
that is very unlikely to ever occur. Even a server that
returns BADSEQID ought to just choose to forget that
open_owner as it has effectively just told that client to
never use it again.  So it might as well just allow reuse
by issuing an OPEN4_RESULT_CONFIRM.

> try to forget the seqid and then use that same owner string again.

I don't think there is an oversight.  The RFC chose to describe how
a server can be optimal and remember old seqid's when there is no
active state. That does not mean that it is incorrect to be silent
about how a client may or may not reuse an open_owner. Functionally
as written the RFC is complete, I am just proposing that the server
can be friendlier in the case where it chooses to remember old
seqids.

All of this is really picking nits on a very rare edge case. The server
is always correct to return BADSEQID when there is no active
state for an open_owner. But it can be more optimal or friendly.

	-David