Subject: [nfsv4] dCache and NFSv4.1

From: Tigran Mkrtchyan <[email-masked]>
Date: Thu, 13 Apr 2006 16:54:08 +0200

Benny, Garth,

   please forward this email to right persons in case if
I sent it to wrong list.

Thanks,
	Tigran.

Dear NFSv4.1 (pNFS) Gurus,

I am working for dCache project at DESY ( http://www.dcache.org ).

Briefly, our system is a front end to tape robot
with metadata server based on nfs2 and data movers distributed over a 
number of nodes. the metadata server is a database with nfs2 interface.
All nfs operations is allowed except IO. For io we have own data 
transfer protocol. Since pNFS ideally fits into our need we would like 
to implement it for out storage system. The NFSv4 based metadata server 
prototype is ready, but our goal is 4.1. And now actual questions:

a)

As in pNFS draft, paragraph 14.3 :
----------------------------
  Once the layout has been retrieved, it can be held across multiple 
OPEN and CLOSE sequences.
----------------------------

Is it possible to have in the result from  LAYOUTGET the layout a flag, 
which describes layout persistently. While we have to get files from 
tapes it may happen ( and will happen due to load sharing ) that each 
time client gets different tape mover - different node.

As in pNFS draft, paragraph 14.5.1
--------------------------------------------------
Since a layout protects a client's access to a file via a direct 
client-storage-device path, a layout need only be recalled when it is 
semantically unable to serve this function. Typically, this occurs when 
the layout no longer encapsulates the true location of the file over the 
byte range it represents. Any operation or action (e.g., server driven 
restriping or load balancing) that changes the layout will result in a 
recall of the layout. A layout is recalled by the CB_LAYOUTRECALL 
callback operation (see Section 24.9 (CB_LAYOUTRECALL)).
-----------------------------------------------------------

For our implementations it's means, that we have to recall layout after 
each CLOSE, which is a unnecessary load in a metadata server.

b)
   Where we can find ,x file of pNFS protocol?

Thank you in advance,
	Tigran.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 13 Apr 2006 11:25:31 -0500

>   Where we can find ,x file of pNFS protocol?

I hope to have a first draft of the NFS4.1 .x file available next week.

Spencer

---

From: "William A.(Andy) Adamson" <[email-masked]>
Date: Thu, 13 Apr 2006 12:46:42 -0400

> As in pNFS draft, paragraph 14.3 :

no. the layout refers to the 'node' via an indirection; the deviceid. if the 
mapping between the deviceid and the storage node changes, the layout must be 
recalled; but this is not so bad.

> As in pNFS draft, paragraph 14.5.1

well, that's the worst case - you need to recall only if the deviceid-node 
mapping changes. plus, other layouttypes (block) will be recalling layouts a 
lot, so it's YTD how layout recall/layoutget on top of the existing OPEN/CLOSE 
state management effects the metadata server performance.

what happens in your current system when the file location changes?

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Thu, 13 Apr 2006 20:41:58 +0200

>what happens in your current system when the file location changes? 

Currently, when a client wants to access a file, we find out the 
location of file and depending on access protocol ( dCache supports dcap 
- the 'native one', ftp, gssftp, gridftp, http and xrootd ) data node 
connects to client or client get actual location of data. In case of 
pNFS we would like to return location, internally notify data mover to 
accept client, but after CLOSE layout becomes invalid. There is a 
several reason for it - internal load management, different policies for 
write and read, file replications on load, garbage collector.

Regards,
    Tigran.

---

From: "Talpey, Thomas" <[email-masked]>
Date: Thu, 13 Apr 2006 15:23:55 -0400

At 02:41 PM 4/13/2006, Tigran Mkrtchyan wrote:
>connects to client or client get actual location of data.

Are you interested in defining layouts for any of these? I think a pNFS
metadata server with ftp or http layouts would be a wonderful thing
to define in the IETF.

Tom.

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Thu, 13 Apr 2006 21:57:40 +0200

>metadata server with ftp or http layouts would be a wonderful thing

Currently we interested in FILE layout and, possibly, OSD. To be able to 
talk to standard client is the primary goal.
If am  I right (  I am not a expert in NFSv4 ) , for other layout types 
you need a special client. We have installations with Solaris (sparc amd 
x86), AIX, Linux, Mac X OS. As soon as we move into non standard kernel 
based client there is no way to support this. Our current implementation 
is a user level API for various platforms. All 'server' code written in 
Java ( including upcoming NFSv4.1 ).

Regards,
    Tigran.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 13 Apr 2006 14:42:01 -0500

> metadata server with ftp or http layouts would be a wonderful thing

Why?  Because it would go through existing firewalls?  :-)