Subject: [nfsv4] Relationship between the drafts rfc5666bis-05 and rpcrdma-bidirection-02

From: David Noveck <[email-masked]>
Date: Mon, 2 May 2016 11:10:22 -0400

In the process of looking at draft-ietf-nfsv4-rpcrdma-bidirection-02, I
noticed some issues in rfc5666bis-05 that might need to be addressed as
part of last call for that latter document.  There might be some
corresponding changes needed in rpcrdma-bidirection-03 but I won't address
those now.  Instead I'll outline the issue, discuss possible changes to
address it and make a proposal for addressing it in rfc5666bis.

*Issue*

The last paragraph of section 5 is as follows:

The RPC-over-RDMA protocol passes RPC messages without regard to their
type (CALL or REPLY) or direction (forwards or backwards).  Each
endpoint of a connection MAY send any RPC-over-RDMA message header
type at any time (subject to credit limits).

The big problem with this paragraph as it stands is that it gives people
the impression that since this document describes Version One and since
messages are sent without regard to their direction, one could read this
document and expect to find out how to implement reverse direction
operation. In fact, although that might have been possible in the -04, the
-05 gives you no information on reverse direction operation. To find out
about reverse direction operation, you have to read rpcrdma-bidirection.

Later, in Section 8.1, the following paragraph appears:

Such extensions must be specified in a Standards Track document with
appropriate review by the nfsv4 Working Group and the IESG.  An
example of a conventional extension to RPC-over-RDMA Version One can
be found in [I-D.ietf-nfsv4-rpcrdma-bidirection].

While these statements are true, the fact that this is the only mention of
rpcrdma-bidirection could exacerbate the problem above since the reader might
be led to believe that defining a conventional extension is the only
function of that document.

*Alternatives*

There are a number of possible ways to address this issue:

   - Move back the material related to backwards direction operation that
   was pulled out in the transition from -04 to -05. At that point it isn't
   clear exactly what would happen to the bidirection document but it is
   likely that its only function would turn out to be the one mentioned in
   Section 8.1.
   - Leave the bidirection document with the function it currently has but
   change rfc5666bis to be clear about that functional division. The changes
   to rfc5666bis would be small and changes to rpcrdma-bidirection would be
   limited because the function of the document is not changing.

Given that we're in WGLC, it makes sense to pick the second option as it is
less disruptive and in my proposal below I will attempt to flesh that out
with regard to rfc5666bis.

*Proposed Changes*

I propose adding the following new paragraph after the existing second
paragraph in Section 1:

Forward direction operation (in which the client is the requester and the
server the responder) is discussed in this document.  Operation in the
reverse direction and with use of multiple directions is dealt with in  the
companion document [I-D.ietf-nfsv4-rpcrdma-bidirection].

I propose adding a new bullet to the list of changes to the protocol:

The protocol has been made symmetrical and thus able to handle
reverse-direction requests and responses. For details, see
[I-D.ietf-nfsv4-rpcrdma-bidirection].

I propose rewriting the last sentence of he last paragraph of section 5 is
as follows:

Apart from restrictions imposed by upper-layer bindings, each endpoint of a
connection is allowed to send any RPC-over-RDMA message header type at any
time (subject to credit limits).

I propose rewriting the last paragraph of Section 8.1 to read as follows:

Such extensions must be specified in a Standards Track document with
appropriate review by the nfsv4 Working Group and the IESG.  An
example of a conventional extension to RPC-over-RDMA Version One is
the specification of size-limited reverse-direction support to enable
NFSv4.1 callbacks, described in [I-D.ietf-nfsv4-rpcrdma-bidirection]

---

From: Chuck Lever <[email-masked]>
Date: Mon, 2 May 2016 11:15:27 -0400

> On May 2, 2016, at 11:10 AM, David Noveck <[email-masked]> wrote:
> 
> In the process of looking at draft-ietf-nfsv4-rpcrdma-bidirection-02, I noticed some issues in rfc5666bis-05 that might need to be addressed as part of last call for that latter document.  There might be some corresponding changes needed in rpcrdma-bidirection-03 but I won't address those now.  Instead I'll outline the issue, discuss possible changes to address it and make a proposal for addressing it in rfc5666bis.
> 
> Issue
> 
> The RPC-over-RDMA protocol passes RPC messages without regard to their type (CALL or REPLY) or direction (forwards or backwards).  Each endpoint of a connection MAY send any RPC-over-RDMA message header type at any time (subject to credit limits).
> The big problem with this paragraph as it stands is that it gives people the impression that since this document describes Version One and since messages are sent without regard to their direction, one could read this document and expect to find out how to implement reverse direction operation.  In fact, although that might have been possible in the -04, the -05 gives you no information on reverse direction operation.  To find out about reverse direction operation, you have to read rpcrdma-bidirection.
> 
> Such extensions must be specified in a Standards Track document with appropriate review by the nfsv4 Working Group and the IESG.  An example of a conventional extension to RPC-over-RDMA Version One can be found in [I-D.ietf-nfsv4-rpcrdma-bidirection].
> While these statements are true, the fact that this is the only mention of rpcrdma-bidirection could exacerbate the problem above since the reader might be led to believe that defining a conventional extension is the only function of that document.
> 
> Alternatives
> 
> 	• Move back the material related to backwards direction operation that was pulled out in the transition from -04 to -05.  At that point it isn't clear exactly what would happen to the bidirection document but it is likely that its only function would turn out to be the one mentioned in Section 8.1.
> 	• Leave the bidirection document with the function it currently has but change rfc5666bis to be clear about that functional division.  The changes to rfc5666bis would be small and changes to rpcrdma-bidirection would be limited because the function of the document is not changing.
> Given that we're in WGLC, it makes sense to pick the second option as it is less disruptive and in my proposal below I will attempt to flesh that out with regard to rfc5666bis. 
> 
> Proposed Changes

Instead, I could simply delete the "or direction (forwards
or backwards)." clause. Would that be adequate to address
your comments?

> I propose adding the following new paragraph after the existing second paragraph in Section 1:
> 
> Forward direction operation (in which the client is the requester and the server the responder) is discussed in this document.  Operation in the reverse direction and with use of multiple directions is dealt with in  the companion document [I-D.ietf-nfsv4-rpcrdma-bidirection].
> 
> The protocol has been made symmetrical and thus able to handle reverse-direction requests and responses.  For details, see [I-D.ietf-nfsv4-rpcrdma-bidirection].
> 
> I propose rewriting the last sentence of he last paragraph of section 5 is as follows:
> 
> Apart from restrictions imposed by upper-layer bindings, each endpoint of a connection is allowed to send any RPC-over-RDMA message header type at any time (subject to credit limits).
> 
> Such extensions must be specified in a Standards Track document with appropriate review by the nfsv4 Working Group and the IESG.  An example of a conventional extension to RPC-over-RDMA Version One is the specification of size-limited reverse-direction support to enable NFSv4.1 callbacks, described in [I-D.ietf-nfsv4-rpcrdma-bidirection]
> 

--
Chuck Lever

---

From: David Noveck <[email-masked]>
Date: Mon, 2 May 2016 12:36:39 -0400

> or backwards)." clause. Would that be adequate to address

I guess it's barely adequate.

I do have to note that bidirectional operation is one of the major advances
that you've made over rfc5666 and it is appropriate for Section 2.2 to note
that.  I think it is long past time to advance beyond the
we've-just-cleaned-up-some-details narrative that you started with.

However, I don't want this issue to hold up going forward with
this document, so I won't continue pushing the matter.

On Mon, May 2, 2016 at 11:15 AM, Chuck Lever <[email-masked]> wrote:

> type (CALL or REPLY) or direction (forwards or backwards).  Each endpoint
> of a connection MAY send any RPC-over-RDMA message header type at any time
> (subject to credit limits).
> appropriate review by the nfsv4 Working Group and the IESG.  An example of
> a conventional extension to RPC-over-RDMA Version One can be found in
> > While these statements are true, the fact that this is the only mention
> of rpcrdma-bidirection could exacerbate the problem above since the reader
> might be led to believe that defining a conventional extension is the only
> >       • Move back the material related to backwards direction operation
> that was pulled out in the transition from -04 to -05.  At that point it
> isn't clear exactly what would happen to the bidirection document but it is
> >       • Leave the bidirection document with the function it currently
> has but change rfc5666bis to be clear about that functional division.  The
> changes to rfc5666bis would be small and changes to rpcrdma-bidirection
> would be limited because the function of the document is not changing.
> > Given that we're in WGLC, it makes sense to pick the second option as it
> is less disruptive and in my proposal below I will attempt to flesh that
> out with regard to rfc5666bis.
> >
> > Proposed Changes
>
> > Forward direction operation (in which the client is the requester and
> the server the responder) is discussed in this document.  Operation in the
> > Apart from restrictions imposed by upper-layer bindings, each endpoint
> of a connection is allowed to send any RPC-over-RDMA message header type at
> any time (subject to credit limits).
> >
> appropriate review by the nfsv4 Working Group and the IESG.  An example of
> a conventional extension to RPC-over-RDMA Version One is the specification
> of size-limited reverse-direction support to enable NFSv4.1 callbacks,
> described in [I-D.ietf-nfsv4-rpcrdma-bidirection]
> >
>
> --
> Chuck Lever
>
>
>
>