Subject: RE: [nfsv4] Eliminating stateid's from pnfs IO

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 22 Sep 2005 10:38:01 -0400

> The second problem should also be solveable using sessions: the
session
> allows the server to determine the client that sent a given I/O
request,
> thus allowing it to recognize inter-client conflicts.  

I assume that the server being referred to is the data server.  So the
data server can recognize conflicts among its clients but conflicts 
we are worried about are conflicts between a client of a meta-data
server, where a lock is done and an IO by a client of data server.

So if you knew when such conflicts existed and had a notion of
identity that allowed you to say that this client of the metadata
server and that client of the data server were the same, then 
you could determine if such a conflict existed, but if you don't
have both of these things, then I don't think you can.

> If the client
> itself can be relied on to handle conflicts between local processes,
> then we're done.

Then the data servers deal with coarser-grained entities, but they
still have to deal with conflicts among those coarser-garined entitites.

> What am I missing?
> 
> The current draft's stateid requirement necessitates a
high-performance
> nfssv4-specific server-to-server protocol, which seems likely to
> complicate pNFS implementation.  So it would obviously be nice if we
> could eliminate that requirement.

Right but if the data server is going to be making decisions that
require
knowledge that the metadata server has, it has to get that knowledge.
Whether that is in the form of stateid or client conflict imformattion,
it does have to be propagated to the data servers.

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]] 
Sent: Wednesday, September 21, 2005 2:20 PM
To: [email-masked]
Subject: [nfsv4] Eliminating stateid's from pnfs IO

Why shouldn't pNFS clients use only special stateid's for all IO?

>From http://www.ietf.org/internet-drafts/draft-welch-pnfs-ops-03.txt:

	"The client MUST use the stateid representing open or lock state
	as returned by an earlier metadata operation (e.g., OPEN, LOCK),
	or a special stateid to perform I/O on the data-servers (as in
	regular NFSv4)."

I don't understand why we're requiring this.

Stateid's perform two purposes that I can think of. First, they provide
protection against replays that cross state-transition boundaries.  (For
example, if a client flushes some write data and then sends an unlock,
there is a risk those writes could be replayed after the unlock.  The
stateid change on unlock allows the server to reject those replays.)

Second, stateid's tie I/O to lockowners, allowing the server to
determine whether a given I/O request conflicts with an existing
mandatory file lock.

I believe sessions solve the replay problem completely.

The second problem should also be solveable using sessions: the session
allows the server to determine the client that sent a given I/O request,
thus allowing it to recognize inter-client conflicts.  If the client
itself can be relied on to handle conflicts between local processes,
then we're done.

What am I missing?

The current draft's stateid requirement necessitates a high-performance
nfsv4-specific server-to-server protocol, which seems likely to
complicate pNFS implementation.  So it would obviously be nice if we
could eliminate that requirement.

At the very least I think the draft needs a clear statement of the
problem that stateid's are meant to solve.  Maybe I'm missing it....

--b.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 22 Sep 2005 11:21:48 -0400

On Thu, Sep 22, 2005 at 10:38:01AM -0400, Noveck, Dave wrote:
> > session allows the server to determine the client that sent a given
> > I/O request, thus allowing it to recognize inter-client conflicts.  
> I assume that the server being referred to is the data server.

Right, sorry.

> So the data server can recognize conflicts among its clients but
> conflicts we are worried about are conflicts between a client of a
> meta-data server, where a lock is done and an IO by a client of data
> server.

Right.  But some setups (e.g. cluster filesystem backends) already have
some way of handling lock conflicts that would need little if any
modification to handle this case.

> > high-performance nfssv4-specific server-to-server protocol, which
> > seems likely to complicate pNFS implementation.  So it would
> > obviously be nice if we could eliminate that requirement.
> 
> require knowledge that the metadata server has, it has to get that
> knowledge.  Whether that is in the form of stateid or client conflict
> imformattion, it does have to be propagated to the data servers.

There has to be some sort of propagation, but I'm not convinced it has
to propagate stateid's specifically.  (What I'm complaining about the is
the need for a high-performance *nfsv4-specific* server-to-server
protocol.)

Also, I should have said that it's not the client requirement that's the
problems--clients can use real stateid's easily enough, and if that
helps some implementations, fine--it's the requirement that data servers
check them.

--b.