Subject: [nfsv4] Callback race(s) and v4/sessions

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 03 Aug 2005 10:27:47 -0400

After the working group meeting here in Paris yesterday, some
of us were discussing the callback races that pNFS layouts
(as well as ordinary v4 delegation recalls) can fall prey to.
I have a new proposal using sessions.

Briefly, this is where a client requests a delegation or layout,
and the reply from the server is delayed in the network. If a
conflicting event occurs, causing the server to send a recall
on the callback channel, the client may reply to the callback
with an error, rather than an acknowledgement. This forces
the server to perform a timeout/retry of unknown duration,
which may result in a forcible revocation.

For pNFS, it was thought at first that the presence of a v4
session on the channels would prevent this, however this is
not the case. Sessions cannot impose ordering across multiple
connections, though they do provide reliable semantics on them.

Here's the idea: add an identifier to the sessions's CB_SEQUENCE
op which carries the { slotid, sequenceid } pair associated with
the operation that requested the layout or delegation, if it is still
"active" (not retired in the server's session DRC). These values
uniquely identify each request in the session, and are provided by
the client - in effect, each request is named. The client can inspect
them to determine whether they match a request which it knows
to be outstanding, and can delay its reply to the server, or return
NFS4ERR_DELAY perhaps.

The presence of the session's reliable DRC at the server will
also prevent any issues of retries affecting this across client
reconnect (which is often associated with this type of scenario),
so the session is important for this reason as well. The unique
naming of each request by the { slotid, sequenceid } pair
is what guarantees that the server will know when the reply
to the conflicting requests are seen by the client, as the client
explicitly drains the server's DRC entries during normal activity (by
reusing the slot, or reducing max_slot).

This is just a sketch of the idea, to see what folks think and
see if it's appropriate to dig further. In yesterday's WG meeting
there was some belief that adding a sequence number to the
layout operations was a solution, I believe that's the wrong
approach because it's specific to the layout ops, and is an
upper layer bandaid over a lower layer issue. I would greatly
prefer to fix this problem right.

Reactions? And, are there other callbacks that would benefit
from having additional context such as this (other uses of this
field)?

Tom.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Jim Zelenka <[email-masked]>
Date: Wed, 03 Aug 2005 10:39:04 -0400

>from having additional context such as this (other uses of this

I like this idea. It seems equally applicable to similar issues
for both data and layout delegations. There's been some discussion
as well about using callbacks to invalidate or update device_id
info cached in the clients (although there is not universal
agreement that we should do so :-). If that plan were adopted, this
could help sequence invalidations versus cache repopulations.

-jim

-- 
Jim Zelenka
Software Engineer, Panasas, Inc.
Accelerating Time to Results(TM) with Clustered Storage
www.panasas.com
412.323-3500