Subject: [nfsv4] more re: client re-using lock_owner

From: [email-masked]
Date: Thu, 19 May 2005 20:01:26 -0400 (EDT)

Well, I'm happy to continue using Door #1, since that's what my code already
does. However, I'm a little nervous that I've missed some fundamental
concept, since I can't see what the problem with Door #2 would be.

Here's an example scenario in more detail:
The client does...
1	Opens "xx" with lock_owner "P1" and seqid#0, gets Stateid0
2	Does an OpenConfirm (as required by the server) with seqid#1, so the
		seqid# is properly initialized, using Stateid0
3	Opens "yy" with lock_owner "P1" and seqid#2, gets Stateid1
4	Opens "zz" with lock_owner "P1" and seqid#3, gets Stateid2
5	Closes "xx" with seqid#4 and Stateid0
6	Closes "yy" with seqid#5 and Stateid1
7	Closes "zz" with seqid#6 and Stateid2

Now, I can see why #2 is needed and I can see why the seqid# has to
be correct for #2-#7.

But, it now seems that lock_owner "P1" is back to what I might call State0.
(No open/lock state associated with lock_owner "P1", which has an
 OpenConfirm'd seqid#.)
Continuing on...
8a	Server throws away lock_owner "P1" (allowed, but not required by
		Sec. 8.1.7 as I understand it)
8	Client opens "aa" with lock_owner "P1" and seqid#0

Now, if 8a has been done, the server would allow 8, requiring an
OpenConfirm for the new seqid# sequence starting at 0.

However, if 8a hasn't been done by the server, 8 will not be allowed and
the server will reply NFS4ERR_BADSEQID, for what I meant by Door #1.

Whereas what I meant by Door #2 was to allow 8, re-initializing the
seqid# sequence at 0 and requiring another OpenConfirm.
(Note that this case only occurs after at least the sequence
of Open, OpenConfirm and Close have occurred. I am not talking about the
case of just an unconfirmed Open.) Since only one of the Ops in sequence
could be replayed, I just can't see what the problem with doing this is?

Am I making sense? rick

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: [email-masked]
Date: Fri, 20 May 2005 11:40:16 -0700 (PDT)

> From: David Robinson [mailto:[email-masked]] 

> Noveck, Dave wrote:

> > However, even though it is not illegal, it still strikes me as
> > a very bad thing to do.  You have enough information to figure
> > out that the seqid is wrong so that it just seems better to tell 
> > the client that, even though you might not be able to if you had
> > dropped that info.  With door #2 you have a server in which 
> > getting a bad owner seqid when you have no opens brings on an 
> > attack of amnesia, so you always respond as if you had dropped
> > the owner info.   
> 
> This seens unduly harsh. Section 8.1.8 says that the server upon
> seeing a lock_owner for the "first time" will require an
> OPEN_CONFIRM. But the spec fails to talk very clearly
> about what is the "first time". It is clear that while there
> is active state for that lock_owner it can't be the first
> time and the server MUST return back NFS4ERR_BADSEQID, the
> client has screwed up state that it is required to maintain.
> 
> However, if there is no active state, how long a time period or
> what events must occur before the reuse of a lock_owner
> qualifies to be the "first time"? And how is a client expected
> to be able to reliably determine that fact?  The other factor is
> that if the client had done something major like rebooted and
> lost any history, it may not have any way to construct a valid
> seqid so it must start with a new lock_owner. Seemingly harmless
> to do such a thing, but I can imagine scenerios where a client
> would rather not keep trying new values until it hit one that
> worked. Eventually the server will forget that lock_owner and
> will think it is the "first time", so why not make that a
> more pro-active event?

I don't see how this particular situation applies since the
rebooted client wil have issued a SETCLIENTID sequence and thus caused the
server to destroy the open_owner, lock_owner, etc. state associated
with it.

We have a RELEASE_LOCKOWNER operation, but no RELEASE_OPENOWNER
operation. And despite the subject of this thread, the issue is open_owners,
lock_owners.

> 
> So I would like Door #2 Monty...

Here's what bothers me about #2. Let's say reason for
the re-used open_owner is not because the client has
forgotten, but because of a retry of an OPEN,
but the client still remembers the open_owner state.
If we return NFS4ERR_BADSEQID,
then this doesn't perturb the existing sequence number state
for the open_owner. If we request open confirmation, then
thr server has two choices:

1. Perturb the state. So if the client in fact had not forgotten
   about the open_owner's previous sequence number, the next time
   client goes to use a sequence number from the previous use of
   the open_owner, he gets NFS4ERR_BADSEQID. And gets very
   confused.

2. Provisionally perturb the state to address the issue of (1)
   above. The server has to maintain two parallel states 
   (similar to Schrodinger's Cat experiment), one
   of which collapses based on a subsequence use. 
   The original sequence number state collapses
   when the client follows up with the OPEN_CONFIRM to be replaced with
   the new sequence. Or, the provisional new sequence number state collapses,
   with the original state continued, once the the original sequence number
   is used for another stated changing operating.

---

From: Mike Eisler <[email-masked]>
Date: Fri, 20 May 2005 18:34:14 -0700 (PDT)

> I assume this is Mike's obfuscated reply address.
                   ^^^^
                   Eisler

This new address will self destruct once the spammers
find it. (My established correspondents should be able
to use my previous email address, unless they are 
asking me to buy mortgages from Nigerian banks backed by
shareholders who made it big in Viagra sales. Oops,
I've just tripped all your spamassassin filters and you
won't see this).

> [email-masked] wrote:
> If the previous OPEN suceeded the server will have state

Not sure you understand what Trond and I are saying.

Let's say the sequence of events is:

time t: OPEN file 1 (seq 1) -->

   times out/held up somewhere

time t+1: OPEN file 1 [retry] (seq 1) -->

time t+2                     <-- OPEN resp from time t

time t+3: client requences response sent a t+1.

time t+4: OPEN_CONFIRM (seq 2) -->

time t+5:                    <-- OPEN_CONFIRM resp

time t+6: OPEN file2(seq 3) -->

...

time t+n ... time t+n+m (all files for the open_owner closed, all locks released)

sequence number for open_owner is now 1000

time t+n+m+1: OPEN retry from t+1 OPEN retry from t+1 finally reaches server.

          at same time:

time t+n+m+2: OPEN file100 seq (1000) -->

The client, at t+n+m+2 thinks his seq is 1000. If the
server implements door #2, then the open for file100 gets BADSEQ,
because the OPEN at t+n+m+1 is accepted without a BADSEQ.
I believe this will confuse most clients, and perhaps all of
the clients that show up at bakeathons.

>  > the server has two choices:
> 
> This would only happen if we have a buggy client. If the server
> requests an open_confirm he is telling the client that they

I should have said if the server "requests open_confirm and resets the sequence
number to 2"

> are establishing new state. If a client holds onto
> its old state  after getting a OPEN4_RESULT_CONFIRM it is
> just broken.

This was a retried OPEN that the client has long since
forgotten about, because he has long since achieved success
with that operation.

The client doesn't care if the server is requesting OPEN_CONFIRM;
he's going to drop the OPEN response. He's going to drop it because
the xid of that response does not correspond to any outstanding
request.

The problem is that short of an infinite duplicate request
cache, the server cannot disinguish (1) retried OPENs from (2)
a reset of a sequence of sequence numbers from a 
new OPEN caused by a client that has forgotten about his 
unused open_owner. So he has to maintain the multiple
quantum-mechanics-like states I mentioned, until the next operation.
If the next operation is OPEN_CONFIRM with seq #2 and the matching stateid X,
then the other state associated with sequence #1000 is disposed of. If the
next operation is OPEN with sequence #1000, then the state with sequence#2
is disposed off. So implementing door #2, requires Schrodinger states.

(The point of all this sequence number stuff we (you actually :-) added
to NFSv4 was to dispense with the dup request cache for nasty
non-idempotent operations like open and locks.)

What is missing here is a RELEASE_OPENOWNER operation. With that,
the issue of clients forgetting about unused OPEN_OWNERS would be moot.

The other thing is that OPEN_CONFIRM is a kludge 
(courtesy me), but it was put it save a separate round 
trip to establish an open_owner. Given the separation
between open_owners and lock_owners a separate operation 
for creating open_owners might not be a bad thing. But anyway, 
draft-ietf-nfsv4-sess-01.txt kills OPEN_CONFIRM.

---

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 21 May 2005 10:19:28 -0400

Even supposing that a server could have complicated
Schrodinger logic that works, I don't think this idea
(client resetting seqids when there is no open state)
would work.

Suppose a client, with a given lockowner opens a file,
does a little stuff and closes with the owner at the 
close being at seqid 3.

Now suppose that later the client sends seqid 0 (same
owner string which is sent as part of a "new" openowner
and the server accepts it and ask for an open confirm and
it is duly sent with seqid 1.

Now you receive seqid 2 and the question arises whether it
is part of the "new" openowner sequence or is some that has
just been disgorged by a router.  The server can't tell so
if he gets seqid 2 he will execute it, correctly or not.

The issue here is that the spec requires ascending seqid's
for a given owner for a reason.  If they can be reset to zero
then you can have different messages with the same seqid/
owner-string/clientid triple and that is *bad*.

If a client tries to reset the seqid, the result shoud be
BADSEQID.  Under some circumstances, the server may be 
unable to detect this situation (dropping apparently unused
state), and so the spec cannot require this in all situations,
but it is something the server should try to do and it 
certainly should do that when it has the information available,
as in the case we are talking about.

If the client resets the seqid he has to expect BADSEQID.  If
the server is unable to make the check, then OPEN_CONFIRM is
used to determine whether the potentially bad seqid received is
OK or not.  If the client's response is to send OPEN_CONFIRM,
then he is saying that the seqid for the open is the correct one,
i.e. it is in the correct sequence.  If he does that when it 
isn't, then he is deliberately violating the protocol and the
server will not be able to reliably detect replays.

> draft-ietf-nfsv4-sess-01.txt kills OPEN_CONFIRM.

It's a brief for the prosecution.  The v4.1 spec will be the 
death sentence, but since sessions are optional, we will have
to wait at least until v4.2 to actually kill it.  Call me
bloodthirsty but I'd like to get on with it.

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]]
Sent: Friday, May 20, 2005 9:34 PM
To: [email-masked]
Subject: RE: [nfsv4] more re: client re-using lock_owner

                   ^^^^
                   Eisler

...

          at same time:

---

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 23 May 2005 15:32:32 -0400

David Robinson wrote:
> > Noveck, Dave wrote:
> > The spec gives no constraints on this.  A server may drop the
> > owner state immediately after the last close of an open for this
> > owner if it wants.  The only consequence of doing that would be
> > a lot of extra OPEN_CONFIRM's and thus bad performance.  It would
> > be legal however.  
> > 
> > In this case, we are talking about, where the hypothesis is that 
> > the server does indeed have information about the openowner, it 
> > is clear this is not the "first time" the server is seeing that
> > owner and the server knows that.  The only reason that this is 
> > not illegal is that judgments of spec legality are subject to 
> > an epistemological constraint.  They may not refer to details
> > of the internals of the server, even though we may very well
> > know about them.  If something is illegal it must determinable
> > from behavior visible over the wire.  But note that I am not
> > saying this is illegal.  I'm just saying it seems bad to me
> > and I think I may take into account what we know about the 
> > actual situation when we discuss those questions.
>
> As the specification is written, it is perfectly legal to return
> either NFS4ERR_BADSEQID or OPEN4_RESULT_CONFIRM. The client must
> be prepared to handle either. I am not advocating that either
> method is illegal, just that OPEN4_RESULT_CONFIRM would be
> preferred.

I know I said that returning OPEN4_RESULT_CONFIRM is legal, but if
so it is  barely legal.  Perhaps "not provably illegal" would be a 
better way to summarize my view of this.  I certainly cannot understand
how you could describe this as "preferred".

> >>And how is a client expected
> >>to be able to reliably determine that fact?
>
> The spec's use of 'first_time' may lead the reader to
> expect that the client somehow tracks the server's notion
> of 'first time', that doesn't seem reasonable nor reliable
> to do. 

The fact that there is no way for the client to do this should
probably suggest to someone interpreting the spec this way, that
he is headed down the wrong path.

> The whole issue of 'first_time' is simply an optimization
> by the server to minimize OPEN_CONFIRM traffic. The minimal
> required functionality is to OPEN4_RESULT_CONFIRM if there
> is no active state.

> My argument is that a server that cannot perform the optimization
> because the seqid is not the correct next seqid is used, it should
> should act as it were the starting from scratch and had not kept
> track of the last seqid and reply with OPEN4_RESULT_CONFIRM.

But why would you do that?

> The difference is that in one case the server replaces the notion
> of the next valid seqid 

Exactly.  He treats a seqid that he knows to be incorrect as if
it were correct.  And the justification, that there are cases in
which the server might not have the information to validate the
sequence id, and depends on the client to do that, does not justify
the server in acting as if he can't.

The concept of the next valid sequence id is that it is the one
which is one greater than the last valid sequence id.  If you look
at the wire, that is what you should see.  For a given owner, the
id's should increase.  In this case, if the server replaces his
notion of the next valid sequence id with one that is different
from that value, he is doing something wrong.  The only reason it is
approximately legal is that you can't prove, just looking at the
wire that the server has the necessary information.  Since you 
don't know that the server was not the innocent victim here (and 
the client just lied to him), you just can't convict.  But it 
is a far cry from that sort of legality to simply assume that 
when the server changes his notion of what the next valid seqid, 
that has any effect of that the real next valid seqid is.  You 
have a situation in whch the seqid ascends and then descends when
the spec says it should ascend, except for wraparound at 4 billion,
I suppose.

> and in the other the client must create
> a new unique open_owner. 

He can either create a new open_owner or, if he doesn't want to do
that, he can use an old one but if he uses an old one then the protocol
has a notion of what the next valid seqid is, and the client should
conform to it.

> This would seem to be easier for the
> server as it is not leaving behind 'old' open_owners to track

If the client wants to use a small set of owners and not leave old
ones around, it may do so.  What it may not do is change the next
seqid whenever it feels like, even if it only feels like it when
there is no open state.

> and better for the client as it is one less round trip by
> saving the OPEN with new open_owner request.

If you make a request with an old open_owner and you have no idea
what the next valid seqid is, then it is very likely that you will
get BADSEQID.  My suggestion is, if you don't like the extra round 
trip, don't do that.  You are in exactly the same position after 
getting the BADSEQID as you were before.  All the server has done
is obey the protocol, rather than cut you the sort of slack that
is going to do you an injury in the end.

> [ignoring my bad and withdrawn example]

> > Is is legal.  You may go through the door.  If I say overly harsh
> > things about your choice, don't take it personally.  I just don't
> > think it is the right choice, even though I agree it is legal.
> 
> We agree that both are legal. I just find that if the server is
> nicer the client has less work to do. Asking for an OPEN_CONFIRM
> feels friendlier than issuing an NFS4ERR_BADSEQID.

It is not friendlier to accept what is invalid from the client and
treat it as if it were valid.  Error checking is part of the protocol
too.

Rather than argue about the precise characterization of the server's
behavior on a legality-illegality continuum, I think the critical
issue is the legality of what you assume the client will be doing.

You are supposing that a client will send a request with a seqid 
that is not valid, and then when the server asks it to confirm that
that is valid (since it finds itself without the necessary information),
the client confirms the one it sent, changing the server's notion of 
the next valid seqid to one which does not match RFC3530's notion of 
the next valid seqid, which is the next one in numerical sequence.  
That is where we have a stark disagreement about what is legal and 
illegal.

> >>Reasonable behavior should be:
> 
> > We need to separate what someone thinks is reasonable from what
> > is in RFC3530.  If you think the spec should have allowed clients
> > to forget state at will, then that is one thing.  But I don't think 
> > the spec actually did that.

> I believe that the spec is agnostic to what the client and server
> do with respect to maintaining open_owner seqid numbers when
> there is no active state. 

It is not agnostic about what the valid seqid is.  The spec says:

   Locking is different than most NFS operations as it requires "at-
   most-one" semantics that are not provided by ONCRPC.  ONCRPC over a
   reliable transport is not sufficient because a sequence of locking
   requests may span multiple TCP connections.  In the face of
   retransmission or reordering, lock or unlock requests must have a
   well defined and consistent behavior.  To accomplish this, each lock
   request contains a sequence number that is a consecutively increasing
   integer.  Different lock_owners have different sequences.

"that is a consecutively increasing integer" does not sound all that
agnostic to me.  I don't see any license for the client to arbitrarily
reset the value to zero at any time, whether the last operation was a
close of the only current open for that owner or not.  
 
> The server can maintain nothing
> always requiring an OPEN_CONFIRM and the client can use
> any value but must be willing to accept a NFS4ERR_BADSEQID
> error.

Correct so far, but what I don't think is valid is for the client to
use an invalid seqid and then when he gets a request to confirm to
then take advantage of the server's (presumed) lack of knowledge and
foist an invalid seqid on him.  

> >>1) Client MUST use the correct next seqid if there is active state
> >>   for that lock_owner.
> >>2) Client SHOULD use the correct next seqid if there is no
> >>   active state for that lock_owner.
> 
> > Strictly speaking, the client may send any seqid he wants but if it
> > is not the correct one, he has to be prepared for BADSEQID.  If
there
> > is open state and he uses an incorrect seqid, then he may rely on 
> > getting BADSEQID.  If there is no open, he may not rely on getting
that
> > error, since the server may have forgotten the owner state.
> 
> Agreed.
> 
> >>4) The server SHOULD allow the OPEN if the lock_owner has the
> >>   correct seqid and there is no active state for that lock_owner.
> 
> > He must process the open normally.  Whether it allowed depends on
> > lots of stuff.

> Agreed, "allowed" is dependant on all the other stuff involved
> in OPEN.  This case is the strong SHOULD that a server ought to
> implement the optimization eliminating OPEN_CONFIRMs.

> >>5) The server SHOULD (MUST?) return OPEN4_RESULT_CONFIRM if the
> >>   lock_owner has an incorrect seqid and there is no active state
> >>   for that lock_owner.

> > Why?  If you assert that this is somehow required by RFC3530, then
> > where exactly does it say that?  I think it says that if it knows
> > a seqid is invalid it should return BADSEQID.  I don't think the
fact
> > that it might sometimes not know that changes that.  It may be
> > that 5) could have been added to the spec without harm, but it
> > wasn't.
> > 
> > As far as the (MUST?) alternative, how can you say you "SHOULD"
> > do the open if he has the correct seqid and "MUST" do it if he
> > has an incorrect seqid?

> The (MUST?) is more of an editorial comment that the RFC doesn't
require
> this, but if I had thought of this scenerio a few years ago I would
> have pushed to make this a MUST. 

In which case I would have pushed for a clear statement that if the
server
does have the information necessary to decide the issue of seqid
validity,
it MUST return BADSEQID.

> But it is not required anywhere,
> just as the server is not required to maintain the last valid seqid
> when there is no active state just so it can return BADSEQID.

Well I would contrariwise strongly urge servers to do that, at least
for a considerable time, and particularly so  if there are clients
that feel that they can change the next valid seqid at will, as long
as there is no current open state.

> >>So in normal processing everyone SHOULD make a reasonable effort
> >>to maintain enough information to minimize the need to use
> >>OPEN_CONFIRM. But in the absence of active state, if one side
> >>forgets, it is useful to allow the lock_owner to be reused
> >>by sending OPEN4_RESULT_CONFIRM instead of NFS4ERR_BADSEQID.
>
> > I don't see how it is useful.  The client should not reuse the
> > same opaque string for different owners and if he forgets the
> > seqid, then these are different owners.
>
> A simple client may choose to use a simple value as the open_owner
> and want to maintain very minimal state and do something trivial
> like restart the seqid at zero if there are no open files.

He may want to initially, but as he thinks about the consequences,
I would expect him to decide to do something else, that makes more 
sense given the protocol.

The spec requires, for a given open_owner, ascending seqids.  There are
two ways, each pretty simple, to do that:

1) Use a fixed or at least only expanding set of owners and reuse the
   existing ones as necessary, and maintain the current next seqid upon
   reuse.

2) Allocate and deallocate owners as you wish but make sure that when
you
   think you are creating a new open_owner, that it really is a new 
   open_owner (i.e. that the opaque string is different from any seen
   before).  Simply adding an integer or a time to make the string
unique
   is a simple way to do that.

> In the BADSEQID case the client will need to do the following
> operations:
>	OPEN oo1 seq0 ->
>		<- BADSEQID
>	OPEN oo2 seq0 ->
>		<- OPEN4_RESULT_CONFIRM
>	OPEN_CONFIRM ->
>		<- OK
> This could be even worse if the choice of 'oo2' also conflicted
> with some previous open_owner that the server is still tracking
> the seqid for, triggering multiple trys by the client to get
> a valid open_owner.

Exactly.  You have yourself a mess.  What you thought was simpler
is not in fact simpler.  Both 1) and 2) are simpler than that since
you never will have to hunt for a valid owner.  If it hurts when you
do that, don't do that.

> If the server is nicer, it can be reduced to:
> 	OPEN oo1 seq0 ->
>		<- OPEN4_RESULT_CONFIRM
>	OPEN_CONFIRM ->
>		<- OK

Sure, but the client has to be prepared for BADSEQID and code for that
case, so it still isn't as simple as 1) and 2).  If you want a simple
client, then those are two good choices, and they don't require any
special niceness on the part of the server.

There are three consequences of doing this:

1) you have code complexity deal with BADSEQID return, which otherwise
   is just a log-this-and-report-a-serious-error type problem (applies
   no mater what server does).

2) you have to hunt and incur latency tyring to find a seqid which works
   (applies when the server is not "friendly", but note that most
existing
    servers are "friendly" in that regard).

3) you compromise RFC3530 replay detection for locking requests (applies
   to all sorts of servers since owner information may be dropped but
issue
   is more severe with "friendly" servers).

You accept that 1) is the case.

It appears that you accept that 2) is the case, but instead of
concluding 
that the client should not do this, you want the server to be more
"friendly".

You argue that 3) is not the case, but even putting that aside, I can't 
see why any client would persist in this approach given 1) and 2).  I
know
you think this is simple but as Einstein (I think) said, "Everything
should 
be as simple as possible, but not more so".

> The client doesn't need to be as aggressive at creating new
> open_owners and the server doesn't have a growing list
> of open_owner/seqid pairs to track to optimize a case
> that is very unlikely to ever occur. 

The client does not have to be aggressive at creating new owners.
It can reuse old ones as long as it uses the correct seqid when
it does.  If it doesn't want to bother with that small amount of
storage it can make sure the new ones are unique.  

> Even a server that
> returns BADSEQID ought to just choose to forget that
> open_owner as it has effectively just told that client to
> never use it again. 

You don't know that it has told the client anything.  If a client
is behaving then typically BADSEQID will be sent in response to
replays such that the client is not looking for the response. 

> So it might as well just allow reuse
> by issuing an OPEN4_RESULT_CONFIRM.

Reuse when the seqid is incorrect, is well, incorrect.  When the
client and the server reset the seqid to a value that it had before
you have broken the basic logic for at-most-once semantics, which
depends on there being for every clientid/owner/seqid only one
request with that triple.  If there are two or more instance then
some may be executed more than once and you have at-most-a-
handful-of-times semantics which isn't what the spec calls for.

> You may think so but the fact is that the spec discusses the
> case of the server forgetting the stateid and never mentions
> the case of the client forgetting it.  If you think that that is
> an oversight, then you need an argument to show why this is a
> necessary case to deal with.  I assert that it is not, since the
> cases of the server and the client are not parallel.  The client
> decides on the lifetime of owners and the server has no way of
> of knowing when the client is done with one.  This requires special
> handling to deal with the case of a server who is forced to drop
> state.  The client doesn't have that problem.  He may drop owners
> at will with no special protocol provision, as long as he doesn't
> try to forget the seqid and then use that same owner string again.

> I don't think there is an oversight.  The RFC chose to describe how
> a server can be optimal and remember old seqid's when there is no
> active state. That does not mean that it is incorrect to be silent
> about how a client may or may not reuse an open_owner. 

I agree that there is no oversight.

I don't think that the spec is incorrect in being silent about
open_owner
reuse.  I'm not saying the spec is incorrect, but I am saying that the
fact the chose to be specific about a server forgetting state and did
not mention anything about a client forgetting state is something you
can draw conclusions from (in part, because we agree that it is not an
oversight).

The spec says that the seqid is an ascending sequence (for a given
owner).

The implication of this is that the one generating requests (i.e. the 
client) must know the previous seqid so it can generate the next one.

If you think that the spec intends there to be an exception, then there
either,

     Has to be words that say that client is free to change the seqid 
     value at any point in which there is no open state for that
open_owner.
     (are there any such?).

     It is intended but not stated (i.e it is an oversight, which we
agree 
     is not the case).

     It is clearly implied because the client can't successfully operate
     without it.  (That is also not the case, although we can imagine 
     client implementations that will not work, but so what?).

The fact that none of the above aply leads me to conclude that there is 
no exception.  It is quite possible for a simple client to provide
ascending 
seqid's for a given owner.

> Functionally
> as written the RFC is complete, I am just proposing that the server
> can be friendlier in the case where it chooses to remember old
> seqids.

I think it is friendlier, when the server is capable of deciding that a 
given seqid is incorrect, to tell the client.  I don't see that
struggling
to limit error detection to the absolute minimum allowed by the spec is
friendly.

> All of this is really picking nits on a very rare edge case. The
server
> is always correct to return BADSEQID when there is no active
> state for an open_owner. But it can be more optimal or friendly.

But we are also talking about what the client may do and the behavior
that you believe the server should accommodate seems to me to be
clearly wrong.  You say the client should accept BADSEQID but it is 
not clear exactly how a client might not accept a BADSEQID response 
if that is what the server sends :-).  The issue is what conclusion
a client implementer mgiht draw from getting BADSEQID.  Either,

     The client should decide that it should try a tiny bit harder
     to generate ascending seqids for a given owner as the spec says 
     it should.

     The client decides that the server is not sufficiently friendly.

In the latter case, the client's accpetance of the BADSEQID is kind of
grudging.  I don't think that is what RFC3530 intended.

I suppose that there is little point in continuing this discussion,
except to point out that it will be rendered moot by sessions, and 
that's a good thing. 

-----Original Message-----
From: David Robinson [mailto:[email-masked]]
Sent: Friday, May 20, 2005 8:47 PM
To: [email-masked]
Subject: Re: [nfsv4] more re: client re-using lock_owner

Noveck, Dave wrote:

> The spec gives no constraints on this.  A server may drop the
> owner state immediately after the last close of an open for this
> owner if it wants.  The only consequence of doing that would be
> a lot of extra OPEN_CONFIRM's and thus bad performance.  It would
> be legal however.  
> 
> In this case, we are talking about, where the hypothesis is that 
> the server does indeed have information about the openowner, it 
> is clear this is not the "first time" the server is seeing that
> owner and the server knows that.  The only reason that this is 
> not illegal is that judgments of spec legality are subject to 
> an epistemological constraint.  They may not refer to details
> of the internals of the server, even though we may very well
> know about them.  If something is illegal it must determinable
> from behavior visible over the wire.  But note that I am not
> saying this is illegal.  I'm just saying it seems bad to me
> and I think I may take into account what we know about the 
> actual situation when we discuss those questions.

As the specification is written, it is perfectly legal to return
either NFS4ERR_BADSEQID or OPEN4_RESULT_CONFIRM. The client must
be prepared to handle either. I am not advocating that either
method is illegal, just that OPEN4_RESULT_CONFIRM would be
preferred.

>>And how is a client expected
>>to be able to reliably determine that fact?

The spec's use of 'first_time' may lead the reader to
expect that the client somehow tracks the server's notion
of 'first time', that doesn't seem reasonable nor reliable
to do. The whole issue of 'first_time' is simply an optimization
by the server to minimize OPEN_CONFIRM traffic. The minimal
required functionality is to OPEN4_RESULT_CONFIRM if there
is no active state.

My argument is that a server that cannot perform the optimization
because the seqid is not the correct next seqid is used, it should
should act as it were the starting from scratch and had not kept
track of the last seqid and reply with OPEN4_RESULT_CONFIRM.

The difference is that in one case the server replaces the notion
of the next valid seqid and in the other the client must create
a new unique open_owner. This would seem to be easier for the
server as it is not leaving behind 'old' open_owners to track
and better for the client as it is one less round trip by
saving the OPEN with new open_owner request.

[ignoring my bad and withdrawn example]

> Is is legal.  You may go through the door.  If I say overly harsh
> things about your choice, don't take it personally.  I just don't
> think it is the right choice, even though I agree it is legal.

We agree that both are legal. I just find that if the server is
nicer the client has less work to do. Asking for an OPEN_CONFIRM
feels friendlier than issuing an NFS4ERR_BADSEQID.

>>Reasonable behavior should be:

> We need to separate what someone thinks is reasonable from what
> is in RFC3530.  If you think the spec should have allowed clients
> to forget state at will, then that is one thing.  But I don't think 
> the spec actually did that.

I believe that the spec is agnostic to what the client and server
do with respect to maintaining open_owner seqid numbers when
there is no active state. The server can maintain nothing
always requiring an OPEN_CONFIRM and the client can use
any value but must be willing to accept a NFS4ERR_BADSEQID
error.

>>1) Client MUST use the correct next seqid if there is active state
>>   for that lock_owner.
>>2) Client SHOULD use the correct next seqid if there is no
>>   active state for that lock_owner.

> Strictly speaking, the client may send any seqid he wants but if it
> is not the correct one, he has to be prepared for BADSEQID.  If there
> is open state and he uses an incorrect seqid, then he may rely on 
> getting BADSEQID.  If there is no open, he may not rely on getting
that
> error, since the server may have forgotten the owner state.

Agreed.

>>4) The server SHOULD allow the OPEN if the lock_owner has the
>>   correct seqid and there is no active state for that lock_owner.

> He must process the open normally.  Whether it allowed depends on
> lots of stuff.

Agreed, "allowed" is dependant on all the other stuff involved
in OPEN.  This case is the strong SHOULD that a server ought to
implement the optimization eliminating OPEN_CONFIRMs.

>>5) The server SHOULD (MUST?) return OPEN4_RESULT_CONFIRM if the
>>   lock_owner has an incorrect seqid and there is no active state
>>   for that lock_owner.

> Why?  If you assert that this is somehow required by RFC3530, then
> where exactly does it say that?  I think it says that if it knows
> a seqid is invalid it should return BADSEQID.  I don't think the fact
> that it might sometimes not know that changes that.  It may be
> that 5) could have been added to the spec without harm, but it
> wasn't.
> 
> As far as the (MUST?) alternative, how can you say you "SHOULD"
> do the open if he has the correct seqid and "MUST" do it if he
> has an incorrect seqid?

The (MUST?) is more of an editorial comment that the RFC doesn't require
this, but if I had thought of this scenerio a few years ago I would
have pushed to make this a MUST. But it is not required anywhere,
just as the server is not required to maintain the last valid seqid
when there is no active state just so it can return BADSEQID.

>>So in normal processing everyone SHOULD make a reasonable effort
>>to maintain enough information to minimize the need to use
>>OPEN_CONFIRM. But in the absence of active state, if one side
>>forgets, it is useful to allow the lock_owner to be reused
>>by sending OPEN4_RESULT_CONFIRM instead of NFS4ERR_BADSEQID.

> I don't see how it is useful.  The client should not reuse the
> same opaque string for different owners and if he forgets the
> seqid, then these are different owners.

A simple client may choose to use a simple value as the open_owner
and want to maintain very minimal state and do something trivial
like restart the seqid at zero if there are no open files.

In the BADSEQID case the client will need to do the following
operations:
	OPEN oo1 seq0 ->
		<- BADSEQID
	OPEN oo2 seq0 ->
		<- OPEN4_RESULT_CONFIRM
	OPEN_CONFIRM ->
		<- OK
This could be even worse if the choice of 'oo2' also conflicted
with some previous open_owner that the server is still tracking
the seqid for, triggering multiple trys by the client to get
a valid open_owner.

If the server is nicer, it can be reduced to:
	OPEN oo1 seq0 ->
		<- OPEN4_RESULT_CONFIRM
	OPEN_CONFIRM ->
		<- OK

The client doesn't need to be as aggressive at creating new
open_owners and the server doesn't have a growing list
of open_owner/seqid pairs to track to optimize a case
that is very unlikely to ever occur. Even a server that
returns BADSEQID ought to just choose to forget that
open_owner as it has effectively just told that client to
never use it again.  So it might as well just allow reuse
by issuing an OPEN4_RESULT_CONFIRM.

> You may think so but the fact is that the spec discusses the
> case of the server forgetting the stateid and never mentions
> the case of the client forgetting it.  If you think that that is
> an oversight, then you need an argument to show why this is a
> necessary case to deal with.  I assert that it is not, since the
> cases of the server and the client are not parallel.  The client
> decides on the lifetime of owners and the server has no way of
> of knowing when the client is done with one.  This requires special
> handling to deal with the case of a server who is forced to drop
> state.  The client doesn't have that problem.  He may drop owners
> at will with no special protocol provision, as long as he doesn't
> try to forget the seqid and then use that same owner string again.

I don't think there is an oversight.  The RFC chose to describe how
a server can be optimal and remember old seqid's when there is no
active state. That does not mean that it is incorrect to be silent
about how a client may or may not reuse an open_owner. Functionally
as written the RFC is complete, I am just proposing that the server
can be friendlier in the case where it chooses to remember old
seqids.

All of this is really picking nits on a very rare edge case. The server
is always correct to return BADSEQID when there is no active
state for an open_owner. But it can be more optimal or friendly.

	-David