Subject: Re: OPEN_CONFIRM question

From: "William A.(Andy) Adamson" <[email-masked]>
Date: Fri, 18 Aug 2000 14:08:53 -0400

well, this should be on the list, so here goes. i'm asking dave noveck a 
question concering OPEN_CONFIRM, and this is his response.

comments?

-->Andy

> i'm confused by some of the OPEN_CONFIRM comments in draft07, and thought you 
> could help shed some light on the subject.

I don't know.  More typical is adding further confusion :-)

> 
> in the IMPLEMENTATION comments of the OPEN_CONFIRM call description 14.2.18, 
> there is a paragraph that starts
> 
>     'The server must hold unconfirmed OPEN state until one of three 
>      events occur.'
> 
> it's the second event, and the other unmentioned possible events that cause my

> confusion:
> 
>     ' Second, the client sends another OPEN request with a sequence id
>      that is incorrect for the nfs_lockowner (out of sequence).  In this
>      case, the server assumes the second OPEN request is valid and the
>      first one is a replay.  The server cancels the OPEN state of the
>      first OPEN request, establishes an unconfirmed OPEN state for the
>      second OPEN request, and responds to the second OPEN request with
>      an indication that an OPEN_CONFIRM is needed.  The process then
>      repeats itself.  While there is a potential for a denial of service
>      attack on the client, it is mitigated if the client and server
>      require the use of a security flavor based on Kerberos V5, LIPKEY,
>      or some other flavor that uses cryptography.'
> 
> 
> So, my server receives the first OPEN for a lockowner (for now i'm a lazy 
> server implementation that requires an OPEN_CONFIRM on each new lockowner).

You call that lazy?  Mine ignores the whole issue and never requires
OPEN_CONFIRM.  Now that's lazy.

> I create the lockowner state, and mark it as UNCONFIRMED. in my response to 
> the client, i indicate the need for an OPEN_CONFIRM.
> 
> now before the server receives an OPEN_CONFIRM, the server receives an OPEN 
> request from the same lockowner:
> 
> case 1: the sequence id is the same as the sequence id from the first OPEN.
> is this an  'out of sequence' sequenceid as refered to above? right now, i'm 
> treating this as a retransmission - and use the lockowner retrans cache.

I don't think that is right.  The idea of unconfirmed opens is that you
don't really know yet what the sequence number environment is, so a 
retrans cache probably won't work correctly.

So I would treat this is "out of sequence" but this leads to another
issue.  See below.

> 
> case 2: the sequence id is one more than the sequence id from the first OPEN.
> is this an 'out of sequence' sequence id as refered to above?

I believe that it has to be considered that it is and you treat the
second OPEN as provisionally correct but requiring confirmation. 

> 
> case 3: the sequence id is less than the first OPEN sequence id 
> i consider this an 'out of sequence' sequence id.

For the same reasons as above, yes.

> 
> case 4:  the sequence id is greater than one plus the first OPEN sequence id.

> i consider this an 'out of sequence' sequence id.

For the same reasons as above, yes.

OK, so any sequence number is treated as out of sequence making either
our understand or the spec suspect.  Since my understanding is perfect,
it must be the spec that's wrong :-)  I think it should say that any OPEN
is handled that way.  The thing that sequencing (in the broad sense)
requires is that an OPEN_CONFIRM follow an OPEN that requires confirmation.

It seems to me that the paragraph that follows the one you quoted
also has problems.  I think NFS4ERR_BAD_STATEID should instead be
NFS4ERR_BAD_SEQID.  Otherwise you have to return NFS4ERR_BAD_STATEID
to an OPEN_CONFIRM that doesn't send a stateid.  But wait a minute,
if you have a read which has a stateid, what do you return?
Maybe "has a stateid" should be changed to "has a seqid"?

> 
> unless the client serializes it's per lockowner OPEN's, it's going to be tough

> for the client to be able to match OPENS with OPEN_CONFIRMS because the client

> and the server could then have a different view of the current seq # for the 
> lockowner.

Right.  There are lots of places where you find out that the sequence logic
depends on the fact that requests that use sequence numbers have to be
serialized at the client on a per-lockowner basis.

This should be stated up front to reduce confusion.  I have proposed that
that be done numerous times, but for some reason, it has never been done.

---

From: mre <[email-masked]>
Date: Sun, 20 Aug 2000 21:34:39 -0700 (PDT)

Since I claim the blame for OPEN_CONFIRM, hopefully the following
addresses the issues raised.

> >     'The server must hold unconfirmed OPEN state until one of three 
[...]
> > server implementation that requires an OPEN_CONFIRM on each new lockowner).
[...]
> So I would treat this is "out of sequence" but this leads to another

I don't agree.

If the ONC RPC duplicate request cache caught this, (i.e. the
RPC xids matched), then the client would get back
the same response.

The intent of this text

"If a duplicate of
   last request (r == L) is received, the stored response is returned."
   
is that the ONC RPC duplicate cache, with no guarantee of
sequential xids and no deterministic way to keep unconfirmed
responses in the cache, is not good enough for locking. So
what the NFSv4 server does is maintain an industrial strength
cache for locks. 

"  It is critical the server maintain the last response sent to the
   client to provide a more reliable cache of duplicate non-idempotent
   requests than that of the traditional cache described in [Juszczak].
   The traditional duplicate request cache uses a least recently used
   algorithm for removing unneeded requests. However, the last lock
   request and response on a given nfs_lockowner must be cached as long
   as the lock state exists on the server."
   
Given that the ONC RPC duplicate request cache will acknowledge the the
request with the same response, I think that if the second OPEN has the
same sequence number as the the first OPEN, AND if the request looks
exactly the same, then the correct behavior is to return the "stored
response."

> second OPEN as provisionally correct but requiring confirmation. 

Right:

	"In this
     first one is a replay."

> For the same reasons as above, yes.

Right.

> For the same reasons as above, yes.

Right.

> NFS4ERR_BAD_SEQID.  Otherwise you have to return NFS4ERR_BAD_STATEID

If the seqid is good, BAD_SEQID is the wrong error. 

> Maybe "has a stateid" should be changed to "has a seqid"?

Well perhaps I committed one of two errors when I contributed the
OPEN_CONFIRM stuff:

	(1) OPEN_CONFIRM4args should have had a stateid as an
		argument, or
		
	(2) there should have been a distinct NFS4ERR_BAD_OPENVERF
		error. However I may have thought at the time
		that it wasn't worth adding a distinct error code for 
		such a rare event.
		
Regardless, I don't think these are fatal errors, and don't see harm in
overloading NFS4ERR_BAD_STATEID for the OPEN_CONFIRM scenario described
above.  However, if Spencer can negotiate with the RFC editor and the
Area Advisor to add a new NFS4ERR_BAD_OPENVERF error, and correct the
text, then this would be fine with me.

> that be done numerous times, but for some reason, it has never been done.

It is in section 8.1.5, though awkwardly:

   Note that for requests that contain a sequence number, the client,     |
   for each nfs_lockowner, there should be no more than one outstanding   |
   request.                                                               |

"should be" --> "have". Definitely something that
the authors and the RFC Editor can fix during the 24 hour 
authors' review period.

---

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 21 Aug 2000 07:32:36 -0700

> exactly the same, then the correct behavior is to return the "stored

The case I was worried about was if the sequence numbers were the same
but the requests did not look exactly the same.  Are we saying that 
that for sequence-number-based caching we have to save the entire
request to verify this (or perhaps the OPEN and all of its args)?
I'm hoping not.

In the case xid-based reply caching, you assume that if the client is
playing by the rules, then equal xids imply equal contents.   Similarly,
for the normal case of sequnce-number-based caching.  If the client is
playing by the rules, equal sequence numbers imply identical requests.

In the environment we are dealing with here, the server doesn't have 
a reliable view of the sequence number yet.  After thinking about this 
some more, I've decided that this doesn't affect the fact that the 
client still has to obey the same rule: that if he sends two requests
with the same lockowner and sequnce number, they have to be the same.

So I agree that you can use the lockowner retrans cache in this case.

> text, then this would be fine with me.

Either way is OK with me.

> the authors and the RFC Editor can fix during the 24 hour 

Sorry I missed this.