Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661

From: <[email-masked]>
Date: Sat, 26 Nov 2011 19:56:39 -0500

What I think is the final errata text for this change to RFC 5661 is attached - sorry it took so
long to get back to this.  In the absence of objection, I'll file the errata request.

The next item is to do something for NFSv4.2 - I will go write up some change text for the
minorversion2 draft along the lines of what was done for the LAYOUTRETURN changes there.
I don't plan to change the arguments to LAYOUTCOMMIT.

Thanks,
--David

> -----Original Message-----
> From: Benny Halevy [mailto:[email-masked]]
> Sent: Monday, July 11, 2011 1:58 AM
> To: Black, David
> Cc: [email-masked]; [email-masked]
> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
> 
> On 2011-07-11 04:47, [email-masked] wrote:
> > My rationale for the client to put EOF in loca_length was to get reasonable
> > behavior if a new client happens to talk to an old server that hasn't implemented
> > the "MUST ignore" requirement.
> >
> > If loca_length is set to EOF (or some client view of what EOF is) by the client,
> > this new-client-old-server scenario only causes issues for the pNFS block layout.
> > If loca_length is set to 0 by the client, then this is also a concern for the
> > file and object layouts.
> >
> > A subtlety here is that this is about errata for the existing NFSv4.1 RFC (5661).
> > I have no problem with NFS4.2 using "MUST" for everything (client MUST set both
> > values to 0, server MUST ignore them), based on the visible bump in the minor
> > version number.
> >
> > How confident are people that the "MUST ignore" requirement for the byte range
> > will get into all the 4.1 servers so that this new-client-old-server scenario
> > can't happen?
> 
> We either commit to this change or not.
> A requirement for backward compatibility means server SHOULD ignore rather than
> MUST ignore, which we don't want to do.
> 
> Since there are no commercial pnfs clients nor servers out in the market I'm
> not worried about server vendors that do not follow the errata.
> I expect all interested vendors to follow the spec + errata and the working
> group discussion (as well as participate in the testing events, but this is
> a separate issue).
> 
> Benny
> 
> >
> > Thanks,
> > --David
> >
> >> -----Original Message-----
> >> From: Benny Halevy [mailto:[email-masked]]
> >> Sent: Sunday, July 10, 2011 7:13 AM
> >> To: Black, David
> >> Cc: Welch, Brent; Mike Eisler; [email-masked]
> >> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
> >>
> >> On 2011-07-09 20:53, Welch, Brent wrote:
> >>> Yeah, LAYOUTCOMMIT already takes loca_last_write_offset which is all the client knows.
> loca_length
> >> should be 0.
> >>
> >> Agreed.
> >>
> >> Benny
> >>
> >>> 	Brent
> >>>
> >>> -----Original Message-----
> >>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Mike Eisler
> >>> Sent: Saturday, July 09, 2011 7:53 AM
> >>> To: [email-masked]
> >>> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
> >>>
> >>> Apologies if there is a discussion I've missed, but:
> >>>
> >>>    The
> >>>    client SHOULD set loca_offset to 0 and SHOULD set loca_length to the
> >>>    offset that the client expects for EOF after the LAYOUTCOMMIT.
> >>>    The server MUST ignore the loca_offset and loca_length arguments.
> >>>
> >>> If the server MUST ignore loca_length, then why burden the client with a
> >>> mandate to determine the "offset that the client expects for EOF after the
> >>> LAYOUTCOMMIT."
> >>>
> >>> And BTW, I'm not sure what "offset that the client expects for EOF after
> >>> the LAYOUTCOMMIT" is.
> >>>
> >>> Regardless, I suggest,
> >>>
> >>>    The
> >>>    client SHOULD set both loca_offset and loca_length to 0.
> >>>
> >>>
> >>> On Fri, July 8, 2011 5:39 pm, [email-masked] wrote:
> >>>> A day late, but here's the revised errata text for this that includes
> >>>> everything Benny
> >>>> found.  See attached file.
> >>>>
> >>>> I've tried to deal with the variation of commit effect among layout types
> >>>> (file and
> >>>> object commit the entire layout, block doesn't) by using a concept of the
> >>>> "scope" of
> >>>> a LAYOUTCOMMIT operation to replace references to the byte range defined
> >>>> by the
> >>>> to-be-deprecated loca_offset and loca_length arguments.
> >>>>
> >>>> Thanks,
> >>>> --David
> >>>> ----------------------------------------------------
> >>>> David L. Black, Distinguished Engineer
> >>>> EMC Corporation, 176 South St., Hopkinton, MA  01748
> >>>> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> >>>> [email-masked]        Mobile: +1 (978) 394-7754
> >>>> ----------------------------------------------------
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> nfsv4 mailing list
> >>>> [email-masked]
> >>>> https://www.ietf.org/mailman/listinfo/nfsv4
> >>>>
> >>>
> >>>
> >
> > _______________________________________________
> > nfsv4 mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/nfsv4

---

From: <[email-masked]>
Date: Sat, 26 Nov 2011 20:24:56 -0500

And here's an initial draft of text for the minorversion2 draft.

Thanks,
--David

> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Saturday, November 26, 2011 7:57 PM
> To: [email-masked]

---

From: Benny Halevy <[email-masked]>
Date: Sun, 27 Nov 2011 12:31:52 +0200

Thanks for getting back to this.
Overall the proposed changes look good to me.
I have a few comments listed below:

>   The LAYOUTCOMMIT operation commits changes in the layout represented
>   by the current filehandle, client ID (derived from the session ID in
>   the preceding SEQUENCE operation), and stateid.  As a layout-independent
>   operation, LAYOUTCOMMIT commits the entire layout; layout-specific data

nit: it's more correct to define this as "layout type-specific" rather than
"layout-specific"

>   (loca_layoutupdate) may specify a smaller scope of data that is to
>   be committed (e.g., for the block layout, see RFC 5663 [41]).

>   The loca_offset and loca_length arguments have been deprecated.  The
>   The client MUST hold one or more existing layouts
>   previously granted via LAYOUTGET (Section 18.43), with an iomode of
>   LAYOUTIOMODE4_RW.

In case layout type-specific data restrict the scope of the LAYOUTCOMMIT
operation, the client MUST hold one or more existing layouts with an iomode
of LAYOUTIOMODE4_RW fully covering the committed byte ranges.

>   In the case where no previously granted layout
>   has an iomode of LAYOUTIOMODE4_RW, the server should return the error
>   NFS4ERR_BAD_IOMODE.  For the case where the client does not hold
>   any previously granted layout, the server should return the error
>   NFS4ERR_BAD_LAYOUT.

>   The loca_last_write_offset field specifies the offset of the last
>   byte written by the client previous to the LAYOUTCOMMIT.  Note that
>   this value is never equal to the file's size (at most it is one byte
>   less than the file's size) and MUST be less than or equal to
>   NFS4_MAXFILEOFF.  Also, loca_last_write_offset MUST be within the
>   scope of data committed by the LAYOUTCOMMIT.

Out of curiosity, how does the client tell the server about a
new loca_last_write_offset
when writing in-place?
Can it send LAYOUTCOMMIT for extents already in the
PNFS_BLOCK_READ_WRITE_DATA
state? (It seems to be allowed by RFC5663)

Benny

---

From: <[email-masked]>
Date: Sun, 27 Nov 2011 19:52:11 -0500

Hi Benny,

I've corrected the nit and add the sentence that you suggested for layout coverage.

On this question:

> Out of curiosity, how does the client tell the server about a new loca_last_write_offset
> when writing in-place?
> Can it send LAYOUTCOMMIT for extents already in the PNFS_BLOCK_READ_WRITE_DATA

It's allowed, and a check w/our implementers suggests that "yes" is a good answer.
I've deleted the offending sentence.

Revised (v3) text is attached.

Thanks,
--David

From: Benny Halevy [mailto:[email-masked]]
Sent: Sunday, November 27, 2011 5:32 AM
To: Black, David
Cc: [email-masked]; [email-masked]
Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661

>   operation, LAYOUTCOMMIT commits the entire layout; layout-specific data

nit: it's more correct to define this as "layout type-specific" rather than "layout-specific"

>   scope of data committed by the LAYOUTCOMMIT.

Out of curiosity, how does the client tell the server about a new loca_last_write_offset
when writing in-place?
Can it send LAYOUTCOMMIT for extents already in the PNFS_BLOCK_READ_WRITE_DATA
state? (It seems to be allowed by RFC5663)

Benny

On Sun, Nov 27, 2011 at 2:56 AM, <[email-masked]<mailto:[email-masked]>> wrote:
What I think is the final errata text for this change to RFC 5661 is attached - sorry it took so
long to get back to this.  In the absence of objection, I'll file the errata request.

Thanks,
--David

> From: Benny Halevy [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
> On 2011-07-11 04:47, [email-masked]<mailto:[email-masked]> wrote:
> >> From: Benny Halevy [mailto:[email-masked]<mailto:[email-masked]>]
> >> Cc: Welch, Brent; Mike Eisler; [email-masked]<mailto:[email-masked]>
> >>> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Mike Eisler
> >>> To: [email-masked]<mailto:[email-masked]>
> >>> On Fri, July 8, 2011 5:39 pm, [email-masked]<mailto:[email-masked]> wrote:
> >>>> +1 (508) 293-7953<tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786<tel:%2B1%20%28508%29%20293-7786>
> >>>> [email-masked]<mailto:[email-masked]>        Mobile: +1 (978) 394-7754<tel:%2B1%20%28978%29%20394-7754>
> >>>> [email-masked]<mailto:[email-masked]>
> > [email-masked]<mailto:[email-masked]>

---

From: <[email-masked]>
Date: Sun, 27 Nov 2011 19:55:32 -0500

Benny's comments on the 4.1 errata text also affect the 4.2 (minorversion2) text.  New version (v2) is attached.

Thanks,
--David

> From: Black, David
> Sent: Saturday, November 26, 2011 8:25 PM
> To: Black, David; [email-masked]
> Subject: [nfsv4] New LAYOUTCOMMIT text for v4.2
>

---

From: Benny Halevy <[email-masked]>
Date: Mon, 28 Nov 2011 13:41:02 +0200

Thanks.
Looks good!

Benny

On 2011-11-28 02:52, [email-masked] wrote:
> Hi Benny,
> 
>  
> 
> I’ve corrected the nit and add the sentence that you suggested for layout coverage.
> 
>  
> 
> On this question:
> 
>  
> 
> It’s allowed, and a check w/our implementers suggests that “yes” is a good answer.
> 
> I’ve deleted the offending sentence.
> 
>  
> 
> *From:*Benny Halevy [mailto:[email-masked]]
> *Sent:* Sunday, November 27, 2011 5:32 AM
> *To:* Black, David
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
> 
>  
> 
> On Sun, Nov 27, 2011 at 2:56 AM, <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> From: Benny Halevy [mailto:[email-masked] <mailto:[email-masked]>]
> 
>> Cc: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
>> On 2011-07-11 04:47, [email-masked] <mailto:[email-masked]> wrote:
>> >> From: Benny Halevy [mailto:[email-masked] <mailto:[email-masked]>]
>> >> Cc: Welch, Brent; Mike Eisler; [email-masked] <mailto:[email-masked]>
>> >>> From: [email-masked] <mailto:[email-masked]> [mailto:[email-masked] <mailto:[email-masked]>] On Behalf Of Mike Eisler
>> >>> To: [email-masked] <mailto:[email-masked]>
>> >>> On Fri, July 8, 2011 5:39 pm, [email-masked] <mailto:[email-masked]> wrote:
>> >>>> +1 (508) 293-7953 <tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786 <tel:%2B1%20%28508%29%20293-7786>
>> >>>> [email-masked] <mailto:[email-masked]>        Mobile: +1 (978) 394-7754 <tel:%2B1%20%28978%29%20394-7754>
>> >>>> [email-masked] <mailto:[email-masked]>
>> > [email-masked] <mailto:[email-masked]>

---

From: Benny Halevy <[email-masked]>
Date: Mon, 28 Nov 2011 13:45:06 +0200

Ack.

Benny