Subject: [nfsv4] Question on stripe unit size in file layout

From: "Suchit Kaura" <[email-masked]>
Date: Wed, 24 Oct 2007 14:22:45 -0700

Hi,

I just wanted to get some more clarifications in this regard.

Looking at the issue from a pNfs client perspective:

The client has a set of ranges AKA layouts that define parts of the user
defined file, that it has obtained from the metadata server.

Thus, a clients view of a file could be:

0 to  Oa-1   : No Layout Available

Oa to Ob -1 : Layout 1

Ob to Oc -1 : Layout 2

Oc to Od -1 : No Layout Available

Od to EOF : Layout 3, loga_length field with all bits set to 1 (one) per
specification.

Oa, Ob, Oc, Od represent offsets within the file. EOF is end of file.

An IO to a file has a file offset and a length. The first thing the
client needs to do is to map the IO i.e. Offset O Length L into the
several (1 to n) layouts that are needed to satisfy the request (This
could involve requesting some layouts from the Metadata server). The
client would then need to process the applicable offset, length
contained within a given layout to determine where exactly to send the
IO.

Thus, in general a client would need to resolve a offset, length range
present within a layout to several ranges AKA stripe units. 

For a given file a pNfs client can indicate the preferred stripe unit
size using the layouthint structure. The metadata server however can
return a different size for the requested layout.

http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-13
<blocked::http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-13>
page 272

----

/* Encoded in the loc_body field of type layout_content4: */

struct nfsv4_1_file_layout4 {

deviceid4 nfl_deviceid;

nfl_util4 nfl_util;

uint32_t nfl_first_stripe_index;

nfs_fh4 nfl_fh_list<>;

};

The nfsv4_1_file_layout4 data type represents the layout. It is

composed of four fields:

1. nfl_deviceid: The device ID which maps to a value of type

nfsv4_1_file_layout_ds_addr4.

2. nfl_util: Like the nflh_util field of data type

nfsv4_1_file_layouthint4, a compact representation of how the

data on a file on each data server is packed, whether the client

should send COMMIT operations to the metadata server or data

server, and the stripe unit size.

----

A couple of questions:

Do we imagine then that different layouts can have different stripe unit
sizes or is this explicitly forbidden for a metadata server to return
different stripe unit sizes for the same file?  

In general if we are writing a client can we use file_offset -
layout->lo_offset instead of file_offset for dense layouts calculation?

layout->lo_offset represents the base offset within a particular layout
returned by the metadata server.

http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-13
<blocked::http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion1-13>
page 276

-------------

   The calculation to determine the octet offset within the data file
   for dense data server layouts is:

      stripe_width = stripe_unit_size * N;
         where N = number of elements in nflda_stripe_indices.

      data_file_offset = floor(file_offset / stripe_width)
         * stripe_unit_size
         + file_offset % stripe_unit_size
--------------

Once again thanks for all the clarifications.

Regards,

Suchit

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Mike Eisler <[email-masked]>
Date: Mon, 29 Oct 2007 14:55:10 -0700 (PDT)

> different stripe unit sizes for the same file?  

As currently specified, in the same filer, different layouts 
can have different stripe unit sizes. 

If a server uses different stripe unit sizes for different layouts
on the file, and the layouts overlap, the consequences are
unspecified.
 
> layout->lo_offset instead of file_offset for dense layouts

I'm not understanding the question. The offset calculation for
dense packing is:

and file_offset comes from the argument to the client's 
programming interfsce (e.g. a vnode switch's _read or _write entry
point, or what POSIX lseek() returns).

---

From: "Suchit Kaura" <[email-masked]>
Date: Mon, 29 Oct 2007 15:24:27 -0700

I guess an example would help.

Let's imagine a user file of length 7KB defined with the following 2
layouts

1st layout
File Offset = 0, length = 3KB, stripe unit size = 1KB, stripe width =
3KB 

2nd layout 
File Offset = 3KB length = 4KB stripe unit size = 2KB, stripe width =
4KB

Now if we consider the formula provided 

It really would start giving us incorrect results based on which unit
size and width we applied to it. It is probably better to have a formula
that uses a relative offset from the start of the layout.

For e.g. if we use the stripe width of 4KB and stripe unit size of 2KB
for a file offset of 3KB the result would be a data file offset of 1KB.
This seems incorrect. 

Regards,
Suchit

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]] 
Sent: Monday, October 29, 2007 2:55 PM
To: Suchit Kaura; [email-masked]
Subject: Re: [nfsv4] Question on stripe unit size in file layout

--- Suchit Kaura <[email-masked]> wrote:

> A couple of questions:
> 
> Do we imagine then that different layouts can have different stripe 
> unit sizes or is this explicitly forbidden for a metadata server to 
> return different stripe unit sizes for the same file?

As currently specified, in the same filer, different layouts can have
different stripe unit sizes. 

If a server uses different stripe unit sizes for different layouts on
the file, and the layouts overlap, the consequences are unspecified.
 
> layout->lo_offset instead of file_offset for dense layouts

I'm not understanding the question. The offset calculation for dense
packing is:

and file_offset comes from the argument to the client's programming
interfsce (e.g. a vnode switch's _read or _write entry point, or what
POSIX lseek() returns).