Subject: [nfsv4] Layout Range.

From: "Anatoly Pinchuk" <[email-masked]>
Date: Tue, 23 Oct 2007 13:06:05 -0700

According to the description of the LAYOUTGET operation

(Page 453,
http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-minorversion1-14.tx
t
<http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-minorversion1-14.t
xt> )

   As well, the metadata server may adjust the range of the returned

   layout based on striping patterns and usage implied by the

   loga_iomode.  The client must be prepared to get a layout that does

   not line up exactly with its request; there MUST be at least an

   overlap of loga_minlength between the layout returned by the server

   and the client's request, or the server SHOULD reject the request.

   See Section 13.5.2 for more details.

there is a possibility that the server returns layout for just a
sub-range of the range

requested by the client. In other words, if the client requested a
layout for [min, max]

range, the server, generally, can return layout applicable to only [min
+ p, max - q] one,

where p>=0, q>=0; min, p, max, and q have to satisfy some consistency
rules.

For example, if the client requested the layout for range [1027,
0x40000000],

the server can return one for [0x10000000, 0x30000000] range.

If so, the returned layout should be limited to the range indicated by
the server

([0x10000000, 0x30000000] in the example above ). It is not clear how

nfsv4_1_file_layout4 structure does that. It would be great if somebody
provides a clarification.

Thanks,

Anatoly

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Mike Eisler <[email-masked]>
Date: Tue, 23 Oct 2007 17:59:17 -0700 (PDT)

Great minds apparently worry about the same issues at the same time.
:-)

It was pointed out to me that some file systems or pNFS servers
might have multiple, very distinct striping patterns per file.

To take Anatoly's example:

[1027, 0x0x99999999] might be striped over 4 data servers,
[0x10000000, 0x30000000] might striped over 6 servers, and
[0x30000001, 0x40000000] might striped over 5 servers, and each
group of data servers might be different.

We have at leats two ways to go:

- Explicitly allow LAYOUTGET to return a subset of the requested range,
  and include some information about why this was done (e.g. two
  flags that indicates that the low end and/or high end of the
requested
  range has a different striping pattern

- change LAYOUTGET to return an array of what it currently returns.

Regarding:

> 0x40000000], the server can return one for [0x10000000, 0x30000000]
range.
> 
> If so, the returned layout should be limited to the range indicated
> by the server
> 
> nfsv4_1_file_layout4 structure does that. It would be great if
> somebody provides a clarification.

I didn't understand the question ... or rather didn't understand
what had to be clarified. 

/* Encoded in the loc_body field of type layout_content4: */
struct nfsv4_1_file_layout4 {
         deviceid4      nfl_deviceid;
         nfl_util4      nfl_util;
         uint32_t       nfl_first_stripe_index;
         nfs_fh4        nfl_fh_list<>;
};

struct layout4 {
    offset4                 lo_offset;
    length4                 lo_length;
    layoutiomode4           lo_iomode;
    layout_content4         lo_content;
};

The idea is that from lo_offset though lo_offset + lo_length - 1.
In this case we chop that range into stripe unit sizes as encoded
in nfl_util.

---

From: Benny Halevy <[email-masked]>
Date: Wed, 24 Oct 2007 12:17:52 +0200

Interestingly enough we discussed this issue also during the Bakeathon.
The common interpretation of the current spec amongst the people I
asked was that if the server returns a different range than what was
requested then it must cover [loga_offset, loga_offset + loga_minlength - 1]
i.e. loga_minlength refers to bytes following the requested offset.
This will provide for a simple client implementation that sweeps
through the desired range.

Benny

On Oct. 24, 2007, 2:59 +0200, Mike Eisler <[email-masked]> wrote:
>> by

---

From: Garth Goodson <[email-masked]>
Date: Wed, 24 Oct 2007 09:21:00 -0700

> through the desired range.

Right.  LAYOUTGET was designed to return a single range starting at the 
offset returned covering the length returned.  I believe it is stated 
that the returned layout must overlap by at least loga_minlength bytes 
(which must be at least 1) the requested range.  If the minlength 
requirement can not be met in a single layout an error is returned and 
the client is currently responsible for splitting that range and retrying.

If there are different striping patterns these should be returned via 
multiple layoutgets in separate layouts.

-Garth

---

From: Robert Gordon <[email-masked]>
Date: Wed, 24 Oct 2007 16:18:04 -0500

On Oct 24, 2007, at 11:21 AM, Garth Goodson wrote:

> Right.  LAYOUTGET was designed to return a single range starting
> at the offset returned covering the length returned.  I believe
> it is stated that the returned layout must overlap by at least
> loga_minlength bytes (which must be at least 1) the requested range.
> If the minlength requirement can not be met in a single layout an
> error is returned and the client is currently responsible for  
> splitting
> that range and retrying.

I think that error would be NFS4ERR_LAYOUTTRYLATER, however we
do not indicate it this is a transient issue or if the client
needs to split the range and try again.

>
> If there are different striping patterns these should be returned
> via multiple layoutgets in separate layouts.

It would seem easier to make LAYOUTGET4resok.logr_layout an array,
as mike suggested.

Robert.

---

From: Garth Goodson <[email-masked]>
Date: Wed, 24 Oct 2007 14:37:13 -0700

Robert Gordon wrote:
>> error is returned and the client is currently responsible for splitting
>> that range and retrying.
> 
TRYLATER is a transient error and I think should not be returned if a 
range needs to be split due to striping issues, rather than some type of 
locking issue over a portion of the range.

> It would seem easier to make LAYOUTGET4resok.logr_layout an array,

I have no objection to returning an array, however I believe that a 
client will still have to handle the corner case of not retrieving the 
entire range at once (even if multiple layouts are returned), since 
there could be other factors; e.g., max_count is too small to return all 
layouts that cover that range.

-Garth

---

From: Benny Halevy <[email-masked]>
Date: Thu, 25 Oct 2007 15:32:27 +0200

> layouts that cover that range.

Agreed.

Benny