Subject: Re: [nfsv4] Soliciting next steps for RPCSEC_GSSv3

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 2 Aug 2013 11:17:05 -0400

On Wed, May 15, 2013 at 05:37:07AM +0000, Spencer Shepler wrote:
> 
> My copy-paste from another email thread is mildly inappropriate but I thought it best to continue this discussion under a separate topic to ensure that everyone has had a chance to comment (see thread content in question below).
> 
> The topic is completing the NFSv4.2 work and by association the RPCSEC_GSSv3 I-D.
> 
> As my original email noted, the NFSv4.2 I-D has a normative reference on the RPCSEC_GSSv3 I-D.
> 
> At this point, we have no editor for the RPCSEC I-D.
> 
> So the first order of business is that I am soliciting an editor for
> http://tools.ietf.org/id/draft-williams-rpcsecgssv3-02.txt
> 
> You will note the I-D expired 18 months ago.
> 
> Second order of business is to decide whether to keep the dependency and if not, what can or should be done about it.
> 
> Bruce suggested three choices (enumerated below).
...
>                 a) Give up for now on some of the requirements in the
> 
>                 "Requirements for Secure inter-Server Copy" section.
> 
>                 Specifically, "The specification MUST provide guidance for using
> 
>                 NFSv4.x as a copy protocol", and "The specification MUST NOT
> 
>                 mandate a trust relationship between the source and destination
> 
>                 server."  Which leaves us having to tell the servers how to
> 
>                 authenticate to each other.
> 
> 
> 
>                 b) Specify only intra-server copy, leave inter-server for 4.3.
> 
> 
> 
>                 c) Leave all of server-to-server copy for 4.3.

(Note I mean "all of server-side copy" for c.)

Anyway, "c" seems out as we already have intra-server-copy
implementations in progress.

I haven't seen anyone explain how to do "a".

I haven't seen any strong objections to "b", and "b" looks by far
simplest.

So I took a stab at removing references to inter-server-copy, subject
labels, and RPCSEC_GSSv3 from the spec.  This eliminates a lot of text:

 20 files changed, 50 insertions(+), 1284 deletions(-)

I'm completely in favor of RPCSEC_GSSv3, but this seems to better
reflect implementation reality for now.

Detailed changes follow.  Done fairly quickly, so I may have missed a
few things.

Any opinions?

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 2 Aug 2013 11:18:50 -0400

From: "J. Bruce Fields" <[email-masked]>

RPCSEC_GSSv3 progress seems to be slower than labeled NFS progress, and
at least one implementation of the latter without the former already
exists.

Therefore, remove references to RPCSEC_GSSv3 from this section.

Some references to hypothetical future subject layer transport methods
are left in, with the idea that this document will then still be
sufficient to guide implementation of full MAC once RPCSEC_GSSv3 is
done.
---
 nfsv42_middle_lnfs.xml |   65 ++++++++++++++++++------------------------------
 1 file changed, 24 insertions(+), 41 deletions(-)

diff --git a/nfsv42_middle_lnfs.xml b/nfsv42_middle_lnfs.xml
index f667021..3cfc25d 100644
--- a/nfsv42_middle_lnfs.xml
+++ b/nfsv42_middle_lnfs.xml
@@ -47,17 +47,9 @@
     </t>
 
     <t>
-      The final change necessary is a modification to the RPC layer used
-      in NFSv4 in the form of a new version of the
-      RPCSEC_GSS <xref target='RFC2203' /> framework.
-      In order for an NFSv4 server to apply MAC checks it must obtain
-      additional information from the client.  Several methods were explored for
-      performing this and it was decided that the best approach was to
-      incorporate the ability to make security attribute assertions through
-      the RPC mechanism.  RPCSECGSSv3 <xref target='rpcsecgssv3' />
-      outlines a method to assert additional security information
-      such as security labels on gss context creation and have that data
-      bound to all RPC requests that make use of that context.
+      An additional useful change would be modification to the RPC layer
+      used in NFSv4 to allow RPC calls to carry security labels.  Such
+      modifications are outside the scope of this document.
     </t>
   </section>
 
@@ -231,15 +223,6 @@
       </t>
 
       <t>
-        As the server is always presented with the subject
-        label from the client, it does not necessarily need to
-        communicate the fact that the label has changed to
-        the client. In the cases where the change outright denies
-        the client access, the client will be able to quickly
-        determine that there is a new label in effect.
-      </t>
-
-      <t>
         Consider a system in which the clients enforce MAC checks and
         and the server has a very simple security
         system which just stores the labels. In this system, the
@@ -263,12 +246,6 @@
   </section>
 
   <section anchor='ss:lnfs:pnfs' title='pNFS Considerations'>
-    <t>
-      This section examines the issues in deploying Labeled NFS in
-      a pNFS community of servers.
-    </t>
-
-    <section toc='exclude' anchor="ss:pnfs:checks" title="MAC Label Checks">
       <t>
         The new FATTR4_SEC_LABEL attribute is metadata information
         and as such the DS is not aware of the value contained
@@ -278,23 +255,17 @@
         the subject label presented by the client, it SHOULD
         utilize this mechanism.
       </t>
-    </section>
   </section>
 
   <section anchor='ss:lnfs:discovery' title='Discovery of Server Labeled NFS Support'>
     <t>
       The server can easily determine that a client supports Labeled NFS
       when it queries for the FATTR4_SEC_LABEL label for an object.
-      Note that it cannot assume that the presence of RPCSEC_GSSv3
-      indicates Labeled NFS support. The client might need to discover
-      which LFS the server supports.
+      The client might need to discover which LFS the server supports.
     </t>
 
     <t>
-      A server which supports Labeled NFS MUST allow a client with any subject
-      label to retrieve the FATTR4_SEC_LABEL attribute for the root
-      filehandle, ROOTFH. The following compound must always succeed
-      as far as a MAC label check is concerned:
+      The following compound MUST NOT be denied by any MAC label check:
     </t>
 
     <figure>
@@ -331,10 +302,13 @@
         requires that both the client and server have an opportunity to
         perform an access control check based on all relevant information
         within the network.  The file object security attribute is provided
-        using the mechanism described in <xref target='ss:lnfs:macsec' />.  The
-        security attribute of the subject making the request is transported
-        at the RPC layer using the mechanism described in RPCSECGSSv3
-        <xref target='rpcsecgssv3' />.
+        using the mechanism described in <xref target='ss:lnfs:macsec' />.
+
+	Fully MAC-Functional NFSv4 servers are not possible in the
+	absence of RPC layer modifications to support subject label
+	transport.  However, servers may make decisions based on the
+	RPC credential information available, and future specifications
+	may provide subject label transport.
       </t>
 
       <section anchor='ss:modes:fm_ilt' title='Initial Labeling and Translation'>
@@ -376,12 +350,21 @@
         </t>
 
         <t>
+	  When the server receives the request from the client it uses
+	  any credential information along with the label of the object
+	  the client is
+
           When the server receives the request from the client it extracts the
           security attribute conveyed in the RPC request.  The server then uses
           this security attribute and the attribute of the object the client is
           trying to access to make an access control decision.  If the server's
           policy allows this access it will fulfill the client's request,
           otherwise it will return NFS4ERR_ACCESS.
+	</t>
+
+	<t>
+	  Future protocol extensions may also allow the server to factor into
+	  the decision a security label extracted from the RPC request.
         </t>
 
         <t>
@@ -400,9 +383,9 @@
           but does not enforce policies. Such a server will store and retrieve
           all object labels presented by clients, utilize the methods
           described in <xref target='ss:macss:lnfs:changes' /> to allow
-          the clients to detect changing labels,, but will not restrict access
-          via the subject label. Instead, it will expect the clients to enforce
-          all such access locally.
+          the clients to detect changing labels, but will factor the
+	  label into access decisions.  Instead, it will expect the clients
+	  to enforce all such access locally.
         </t>
       </section>
     </section>
-- 
1.7.9.5

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 2 Aug 2013 11:19:42 -0400

I'm aware of at least one implementation effort for intra-server copies,
but not of any serious effort for inter-server copies.

It seems likely that the former will be implemented long before the
latter, and therefore it is preferable not to make advancing this spec
dependent on the (more complicated) inter-server case.

Also, this allows us to remove references to the (so far unfinished)
RPCSEC_GSSv3 spec.
---
 Makefile                            |    6 -
 dotx.d/Makefile                     |    4 -
 dotx.d/copy_args.x                  |    1 -
 dotx.d/copy_notify_args.x           |    7 -
 dotx.d/copy_notify_res.x            |   12 -
 dotx.d/head.x                       |    3 -
 dotx.d/offload_revoke_args.x        |    5 -
 dotx.d/offload_revoke_res.x         |    4 -
 dotx.d/spit_types.sh                |    6 -
 nfsv42_middle_copy.xml              |  859 +----------------------------------
 nfsv42_middle_errors.xml            |   80 ----
 nfsv42_middle_new_errors.xml        |   80 ----
 nfsv42_middle_op_cb_offload.xml     |    4 +-
 nfsv42_middle_op_copy.xml           |   48 +-
 nfsv42_middle_op_copy_notify.xml    |   89 ----
 nfsv42_middle_op_mandlist.xml       |    2 -
 nfsv42_middle_op_offload_abort.xml  |    4 +-
 nfsv42_middle_op_offload_revoke.xml |   51 ---
 nfsv42_middle_op_offload_status.xml |    4 +-
 19 files changed, 26 insertions(+), 1243 deletions(-)
 delete mode 100644 dotx.d/copy_notify_args.x
 delete mode 100644 dotx.d/copy_notify_res.x
 delete mode 100644 dotx.d/offload_revoke_args.x
 delete mode 100644 dotx.d/offload_revoke_res.x
 delete mode 100644 nfsv42_middle_op_copy_notify.xml
 delete mode 100644 nfsv42_middle_op_offload_revoke.xml

diff --git a/Makefile b/Makefile
index 6704ca4..4ccdde4 100755
--- a/Makefile
+++ b/Makefile
@@ -311,10 +311,6 @@ AUTOGEN =	\
 		autogen/close_res.xml \
 		autogen/commit_args.xml \
 		autogen/commit_res.xml \
-		autogen/copy_notify_args.xml \
-		autogen/copy_notify_res.xml \
-		autogen/offload_revoke_args.xml \
-		autogen/offload_revoke_res.xml \
 		autogen/copy_args.xml \
 		autogen/copy_res.xml \
 		autogen/offload_abort_args.xml \
@@ -442,8 +438,6 @@ IDXMLSRC_BASE = \
 	${DOC_PREFIX}_middle_op_aaa.xml \
 	${DOC_PREFIX}_middle_op_copy.xml \
 	${DOC_PREFIX}_middle_op_offload_abort.xml \
-	${DOC_PREFIX}_middle_op_copy_notify.xml \
-	${DOC_PREFIX}_middle_op_offload_revoke.xml \
 	${DOC_PREFIX}_middle_op_offload_status.xml \
 	${DOC_PREFIX}_middle_op_exchange_id.xml \
 	${DOC_PREFIX}_middle_op_write_plus.xml \
diff --git a/dotx.d/Makefile b/dotx.d/Makefile
index cdd43ba..a61c351 100755
--- a/dotx.d/Makefile
+++ b/dotx.d/Makefile
@@ -112,10 +112,6 @@ FORE_OPS_AS_IS2 = \
                 copy_res.x \
                 offload_abort_args.x \
                 offload_abort_res.x \
-                copy_notify_args.x \
-                copy_notify_res.x \
-                offload_revoke_args.x \
-                offload_revoke_res.x \
                 offload_status_args.x \
                 offload_status_res.x \
 		write_plus_args.x \
diff --git a/dotx.d/copy_args.x b/dotx.d/copy_args.x
index c3346cd..b0c6c9d 100644
--- a/dotx.d/copy_args.x
+++ b/dotx.d/copy_args.x
@@ -13,6 +13,5 @@ struct COPY4args {
 	length4		ca_count;
 	uint32_t	ca_flags;
 	component4	ca_destination;
-	netloc4		ca_source_server<>;
 };
 
diff --git a/dotx.d/copy_notify_args.x b/dotx.d/copy_notify_args.x
deleted file mode 100644
index 364f53b..0000000
--- a/dotx.d/copy_notify_args.x
+++ /dev/null
@@ -1,7 +0,0 @@
-
-struct COPY_NOTIFY4args {
-        /* CURRENT_FH: source file */
-	stateid4    	cna_src_stateid;
-	netloc4    	cna_destination_server;
-};
-
diff --git a/dotx.d/copy_notify_res.x b/dotx.d/copy_notify_res.x
deleted file mode 100644
index e28cbd6..0000000
--- a/dotx.d/copy_notify_res.x
+++ /dev/null
@@ -1,12 +0,0 @@
-struct COPY_NOTIFY4resok {
-	nfstime4	cnr_lease_time;
-	netloc4		cnr_source_server<>;
-};
-
-union COPY_NOTIFY4res switch (nfsstat4 cnr_status) {
-case NFS4_OK:
-	COPY_NOTIFY4resok	resok4;
-default:
-	void;
-};
-
diff --git a/dotx.d/head.x b/dotx.d/head.x
index db21417..39a6e8f 100644
--- a/dotx.d/head.x
+++ b/dotx.d/head.x
@@ -174,9 +174,6 @@ enum nfsstat4 {
 
  /* NFSv4.2 errors start here. */
 
- NFS4ERR_PARTNER_NOTSUPP= 10088,/* s2s not supported       */
- NFS4ERR_PARTNER_NO_AUTH= 10089,/* s2s not authorized      */
- NFS4ERR_METADATA_NOTSUPP=10090,/* dest metadata diff sourc*/
  NFS4ERR_OFFLOAD_DENIED = 10091,/* dest not allowing copy  */
  NFS4ERR_WRONG_LFS      = 10092,/* LFS not supported       */
  NFS4ERR_BADLABEL       = 10093,/* incorrect label         */
diff --git a/dotx.d/offload_revoke_args.x b/dotx.d/offload_revoke_args.x
deleted file mode 100644
index 045b189..0000000
--- a/dotx.d/offload_revoke_args.x
+++ /dev/null
@@ -1,5 +0,0 @@
-struct OFFLOAD_REVOKE4args {
-        /* CURRENT_FH: source file */
-	netloc4    	ora_destination_server;
-};
-
diff --git a/dotx.d/offload_revoke_res.x b/dotx.d/offload_revoke_res.x
deleted file mode 100644
index 88609ab..0000000
--- a/dotx.d/offload_revoke_res.x
+++ /dev/null
@@ -1,4 +0,0 @@
-struct OFFLOAD_REVOKE4res {
-	nfsstat4	orr_status;
-};
-
diff --git a/dotx.d/spit_types.sh b/dotx.d/spit_types.sh
index 40a2468..114cdda 100755
--- a/dotx.d/spit_types.sh
+++ b/dotx.d/spit_types.sh
@@ -994,8 +994,6 @@ enum nfs_opnum4 {
 %
  OP_COPY		= 59,
  OP_OFFLOAD_ABORT	= 60,
- OP_COPY_NOTIFY		= 61,
- OP_OFFLOAD_REVOKE	= 62,
  OP_OFFLOAD_STATUS	= 63,
  OP_WRITE_PLUS		= 64,
  OP_READ_PLUS		= 65,
@@ -1112,8 +1110,6 @@ union nfs_argop4 switch (nfs_opnum4 argop) {
 				opreclaim_complete;
 
  /* Operations new to NFSv4.2 */
- case OP_COPY_NOTIFY:	COPY_NOTIFY4args opoffload_notify;
- case OP_OFFLOAD_REVOKE: OFFLOAD_REVOKE4args opcopy_revoke;
  case OP_COPY:		COPY4args opcopy;
  case OP_OFFLOAD_ABORT:	OFFLOAD_ABORT4args opoffload_abort;
  case OP_OFFLOAD_STATUS: OFFLOAD_STATUS4args opoffload_status;
@@ -1242,8 +1238,6 @@ union nfs_resop4 switch (nfs_opnum4 resop) {
 				opreclaim_complete;
 
  /* Operations new to NFSv4.2 */
- case OP_COPY_NOTIFY:	COPY_NOTIFY4res opcopy_notify;
- case OP_OFFLOAD_REVOKE: OFFLOAD_REVOKE4res opoffload_revoke;
  case OP_COPY:		COPY4res opcopy;
  case OP_OFFLOAD_ABORT:	OFFLOAD_ABORT4res opoffload_abort;
  case OP_OFFLOAD_STATUS: OFFLOAD_STATUS4res opoffload_status;
diff --git a/nfsv42_middle_copy.xml b/nfsv42_middle_copy.xml
index 13e9174..260c5f6 100644
--- a/nfsv42_middle_copy.xml
+++ b/nfsv42_middle_copy.xml
@@ -16,32 +16,16 @@
     </t>
 
     <t>
-      If the source object and destination object are on different file
-      servers, the file servers will communicate with one another to
-      perform the copy operation. The server-to-server protocol by which
-      this is accomplished is not defined in this document.
+      The source and destination objects must be located on the same
+      file server; server-to-server copy is not specified by this
+      document.
     </t>
+
   </section>
 
   <section title="Protocol Overview">
     <t>
-      The server-side copy offload operations support both intra-server and
-      inter-server file copies. An intra-server copy is a copy in which the
-      source file and destination file reside on the same server. In an
-      inter-server copy, the source file and destination file are on
-      different servers. In both cases, the copy may be performed
-      synchronously or asynchronously.
-    </t>
-
-    <t>
-      Throughout the rest of this document, we refer to the NFS
-      server containing the source file as the "source server"
-      and the NFS server to which the file is transferred as the
-      "destination server". In the case of an intra-server copy,
-      the source server and destination server are the same server.
-      Therefore in the context of an intra-server copy, the terms
-      source server and destination server refer to the
-      single server performing the copy.
+      Copies may be performed synchronously or asynchronously.
     </t>
 
     <t>
@@ -60,30 +44,9 @@
       procedure.
     </t>
 
-    <t>
-      For the inter-server copy, the operations are defined
-      to be compatible with the traditional copy authentication approach.
-      The client and user are authorized at the source for reading. Then
-      they are authorized at the destination for writing.
-    </t>
-
     <section title="Overview of Copy Operations">
       <t>
         <list style="hanging">
-          <t hangText="COPY_NOTIFY:">
-            For inter-server copies, the client sends
-            this operation to the source server to notify it of a future
-            file copy from a given destination server for the given user.
-            (<xref target="op:copy_notify" />)
-          </t>
-
-          <t hangText="OFFLOAD_REVOKE:">
-            Also for inter-server copies, the client
-            sends this operation to the source server to revoke permission
-            to copy a file for the given user.
-            (<xref target="op:offload_revoke" />)
-          </t>
-
           <t hangText="COPY:">
             Used by the client to request a file copy.
             (<xref target="op:copy" />)
@@ -101,7 +64,7 @@
           </t>
 
           <t hangText="CB_OFFLOAD:">
-            Used by the destination server to report the
+            Used by the server to report the
             results of an asynchronous file copy to the client.
             (<xref target="op:cb_offload" />)
           </t>
@@ -118,9 +81,9 @@
       </t>
     </section>
 
-    <section title="Intra-Server Copy">
+    <section title="Copy Overview">
       <t>
-        To copy a file on a single server, the client uses a COPY operation.
+        To copy a file, the client uses a COPY operation.
         The server may respond to the copy operation with the final results of
         the copy or it may perform the copy asynchronously and deliver the
         results using a CB_OFFLOAD operation callback. If the copy is performed
@@ -129,15 +92,15 @@
       </t>
 
       <t>
-        A synchronous intra-server copy is shown in
-        <xref target="fig:sync-intra-server" />. In this example, the NFS
+        A synchronous copy is shown in
+        <xref target="fig:sync-copy" />. In this example, the NFS
         server chooses to perform the copy synchronously. The copy operation
         is completed, either successfully or unsuccessfully, before the
         server replies to the client's request. The server's reply contains
         the final result of the operation.
       </t>
 
-      <figure title="A synchronous intra-server copy." anchor="fig:sync-intra-server">
+      <figure title="A synchronous copy." anchor="fig:sync-copy">
         <artwork><![CDATA[
   Client                                  Server
      +                                      +
@@ -162,8 +125,8 @@
       </figure>
 
       <t>
-        An asynchronous intra-server copy is shown in
-        <xref target="afig:sync-intra-server" />. In this example, the NFS
+        An asynchronous copy is shown in
+        <xref target="afig:async-copy" />. In this example, the NFS
         server performs the copy asynchronously. The server's reply to
         the copy request indicates that the copy operation was initiated
         and the final result will be delivered at a later time. The server's
@@ -173,7 +136,7 @@
         performs a callback to the client and reports the results.
       </t>
 
-      <figure title="An asynchronous intra-server copy." anchor="afig:sync-intra-server">
+      <figure title="An asynchronous intra-server copy." anchor="afig:async-copy">
         <artwork><![CDATA[
   Client                                  Server
      +                                      +
@@ -209,244 +172,6 @@
       </figure>
     </section>
 
-    <section title="Inter-Server Copy">
-      <t>
-        A copy may also be performed between two servers. The copy
-        protocol is designed to accommodate a variety of network
-        topologies. As shown in <xref target="fig:inter-server" />,
-        the client and servers may be connected by multiple networks.
-        In particular, the servers may be connected by a specialized,
-        high speed network (network 192.0.2.0/24 in the diagram)
-        that does not include the client. The protocol allows the
-        client to setup the copy between the servers (over network
-        203.0.113.0/24 in the diagram) and for the servers to
-        communicate on the high speed network if they choose to
-        do so.
-      </t>
-
-      <figure title="An example inter-server network topology." anchor="fig:inter-server">
-        <artwork><![CDATA[
-                          192.0.2.0/24
-              +-------------------------------------+
-              |                                     |
-              |                                     |
-              | 192.0.2.18                          | 192.0.2.56
-      +-------+------+                       +------+------+
-      |     Source   |                       | Destination |
-      +-------+------+                       +------+------+
-              | 203.0.113.18                        | 203.0.113.56
-              |                                     |
-              |                                     |
-              |             203.0.113.0/24          |
-              +------------------+------------------+
-                                 |
-                                 |
-                                 | 203.0.113.243
-                           +-----+-----+
-                           |   Client  |
-                           +-----------+
-        ]]></artwork>
-      </figure>
-
-      <t>
-        For an inter-server copy, the client notifies the source
-        server that a file will be copied by the destination
-        server using a COPY_NOTIFY operation. The client then
-        initiates the copy by sending the COPY operation to the
-        destination server. The destination server may perform the
-        copy synchronously or asynchronously.
-      </t>
-
-      <t>
-        A synchronous inter-server copy is shown in
-        <xref target="fig:sync-inter-server" />. In this case, the
-        destination server chooses to perform the copy before
-        responding to the client's COPY request.
-      </t>
-
-      <t>
-        An asynchronous copy is shown in
-        <xref target="fig:async-inter-server" />. In this case, the
-        destination server chooses to respond to the client's COPY
-        request immediately and then perform the copy
-        asynchronously.
-      </t>
-
-      <figure title="A synchronous inter-server copy." anchor="fig:sync-inter-server">
-        <artwork><![CDATA[
-  Client                Source         Destination
-     +                    +                 +
-     |                    |                 |
-     |--- OPEN        --->|                 | Returns os1
-     |<------------------/|                 |
-     |                    |                 |
-     |--- COPY_NOTIFY --->|                 |
-     |<------------------/|                 |
-     |                    |                 |
-     |--- OPEN ---------------------------->| Returns os2
-     |<------------------------------------/|
-     |                    |                 |
-     |--- COPY ---------------------------->|
-     |                    |                 |
-     |                    |                 |
-     |                    |<----- read -----|
-     |                    |\--------------->|
-     |                    |                 |
-     |                    |        .        | Multiple reads may
-     |                    |        .        | be necessary
-     |                    |        .        |
-     |                    |                 |
-     |                    |                 |
-     |<------------------------------------/| Destination replies
-     |                    |                 | to COPY
-     |                    |                 |
-     |--- CLOSE --------------------------->| Release open state
-     |<------------------------------------/|
-     |                    |                 |
-     |--- CLOSE       --->|                 | Release open state
-     |<------------------/|                 |
-        ]]></artwork>
-      </figure>
-
-      <figure title="An asynchronous inter-server copy." anchor="fig:async-inter-server">
-        <artwork><![CDATA[
-  Client                Source         Destination
-    +                    +                 +
-    |                    |                 |
-    |--- OPEN        --->|                 | Returns os1
-    |<------------------/|                 |
-    |                    |                 |
-    |--- LOCK        --->|                 | Optional, could be done
-    |<------------------/|                 | with a share lock
-    |                    |                 |
-    |--- COPY_NOTIFY --->|                 | Need to pass in
-    |<------------------/|                 | os1 or lock state
-    |                    |                 |
-    |                    |                 |
-    |                    |                 |
-    |--- OPEN ---------------------------->| Returns os2
-    |<------------------------------------/|
-    |                    |                 |
-    |--- LOCK ---------------------------->| Optional ...
-    |<------------------------------------/|
-    |                    |                 |
-    |--- COPY ---------------------------->| Need to pass in
-    |<------------------------------------/| os2 or lock state
-    |                    |                 |
-    |                    |                 |
-    |                    |<----- read -----|
-    |                    |\--------------->|
-    |                    |                 |
-    |                    |        .        | Multiple reads may
-    |                    |        .        | be necessary
-    |                    |        .        |
-    |                    |                 |
-    |                    |                 |
-    |--- OFFLOAD_STATUS ------------------>| Client may poll
-    |<------------------------------------/| for status
-    |                    |                 |
-    |                    |        .        | Multiple OFFLOAD_STATUS
-    |                    |        .        | operations may be sent
-    |                    |        .        |
-    |                    |                 |
-    |                    |                 |
-    |                    |                 |
-    |<-- CB_OFFLOAD -----------------------| Destination reports
-    |\------------------------------------>| results
-    |                    |                 |
-    |--- LOCKU --------------------------->| Only if LOCK was done
-    |<------------------------------------/|
-    |                    |                 |
-    |--- CLOSE --------------------------->| Release open state
-    |<------------------------------------/|
-    |                    |                 |
-    |--- LOCKU       --->|                 | Only if LOCK was done
-    |<------------------/|                 |
-    |                    |                 |
-    |--- CLOSE       --->|                 | Release open state
-    |<------------------/|                 |
-    |                    |                 |
-        ]]></artwork>
-      </figure>
-    </section>
-
-    <section title="Server-to-Server Copy Protocol" anchor="server-to-server-proto">
-      <t>
-        The source server and
-        destination server are not required to use a specific protocol to
-        transfer the file data. The choice of what protocol to use
-        is ultimately the destination server's decision.
-      </t>
-
-      <section title="Using NFSv4.x as a Server-to-Server Copy Protocol">
-        <t>
-          The destination server MAY use standard NFSv4.x (where x >= 1)
-          to read the data from the source server. If NFSv4.x is used
-          for the server-to-server copy protocol, the destination server
-          can use the filehandle contained in the COPY request with
-          standard NFSv4.x operations to read data
-          from the source server. Specifically, the destination
-          server may use the NFSv4.x OPEN operation's CLAIM_FH
-          facility to open the file being copied and obtain an
-          open stateid. Using the stateid, the destination server
-          may then use NFSv4.x READ operations to read the file.
-        </t>
-      </section>
-
-      <section title="Using an alternative Server-to-Server Copy Protocol">
-        <t>
-          In a homogeneous environment, the source and destination servers
-          might be able to perform the file copy extremely efficiently using
-          specialized protocols. For example the source and destination
-          servers might be two nodes sharing a common file system format for
-          the source and destination file systems. Thus the source and destination
-          are in an ideal position to efficiently render the image of the source
-          file to the destination file by replicating the file system formats
-          at the block level. Another possibility is that the source and
-          destination might be two nodes sharing a common storage area network,
-          and thus there is no need to copy any data at all, and instead
-          ownership of the file and its contents might simply be re-assigned
-          to the destination. To allow for these possibilities, the destination
-          server is allowed to use a server-to-server copy protocol of its choice.
-        </t>
-
-        <t>
-          In a heterogeneous environment, using a protocol other than NFSv4.x
-          (e.g., HTTP <xref target="RFC2616" /> or FTP <xref target="RFC0959" />)
-          presents some challenges. In particular, the destination
-          server is presented with the challenge of accessing the source file
-          given only an NFSv4.x filehandle.
-        </t>
-
-        <t>
-          One option for protocols that identify source files with path names
-          is to use an ASCII hexadecimal representation of the source filehandle
-          as the file name.
-        </t>
-
-        <t>
-          Another option for the source server is to use URLs to direct the
-          destination server to a specialized service. For example, the response
-          to COPY_NOTIFY could include the URL ftp://s1.example.com:9999/_FH/0x12345,
-          where 0x12345 is the ASCII hexadecimal representation of the source
-          filehandle. When the destination server receives the source server's
-          URL, it would use "_FH/0x12345" as the file name to pass to the FTP
-          server listening on port 9999 of s1.example.com. On port
-          9999 there would be a special instance of the FTP service that
-          understands how to convert NFS filehandles to an open file descriptor
-          (in many operating systems, this would require a new system call, one
-          which is the inverse of the makefh() function that the pre-NFSv4 MOUNT
-          service needs).
-        </t>
-
-        <t>
-          Authenticating and identifying the destination server to the source
-          server is also a challenge. Recommendations for how to accomplish
-          this are given in <xref target="server-to-server-no-rpc-with-gss" />
-          and <xref target="server-to-server-no-rpc-no-gss" />.
-        </t>
-      </section>
-    </section>
   </section>
 
   <section title="Requirements for Operations" anchor="operations">
@@ -458,39 +183,20 @@
     </t>
 
     <t>
-      If a client desires an intra-server file copy, then it MUST
+      A client supporting server-side copy MUST
       support the COPY and CB_OFFLOAD operations. If COPY returns a
       stateid, then the client MAY use the OFFLOAD_ABORT and OFFLOAD_STATUS
       operations.
     </t>
 
     <t>
-      If a client desires an inter-server file copy, then it MUST
-      support the COPY, COPY_NOTICE, and CB_OFFLOAD operations, and
-      MAY use the OFFLOAD_REVOKE operation. If COPY returns a stateid,
-      then the client MAY use the OFFLOAD_ABORT and OFFLOAD_STATUS
-      operations.
-    </t>
-
-    <t>
-      If a server supports intra-server copy, then the server MUST
+      If a server supports server-side copy, then the server MUST
       support the COPY operation. If a server’s COPY operation
       returns a stateid, then the server MUST also support these operations:
       CB_OFFLOAD, OFFLOAD_ABORT, and OFFLOAD_STATUS.
     </t>
 
     <t>
-      If a source server supports inter-server copy, then the source
-      server MUST support all these operations: COPY_NOTIFY and
-      OFFLOAD_REVOKE. If a destination server supports inter-server
-      copy, then the destination server MUST support the COPY
-      operation. If a destination server’s COPY operation returns
-      a stateid, then the destination server MUST also support these
-      operations: CB_OFFLOAD, OFFLOAD_ABORT, COPY_NOTIFY, OFFLOAD_REVOKE,
-      and OFFLOAD_STATUS.
-    </t>
-
-    <t>
       Each operation is performed in the context of the user identified
       by the ONC RPC credential of its containing COMPOUND or CB_COMPOUND
       request. For example, a OFFLOAD_ABORT operation issued by a given user
@@ -504,36 +210,6 @@
       copy operations using an implementation specific interface.
     </t>
 
-    <section title="netloc4 - Network Locations">
-      <t>
-        The server-side copy operations specify network locations using
-        the netloc4 data type shown below:
-      </t>
-
-      <?rfc include='autogen/type_netloc_type4.xml'?>
-
-      <t>
-        If the netloc4 is of type NL4_NAME, the nl_name field MUST be
-        specified as a UTF-8 string. The nl_name is expected to be
-        resolved to a network address via DNS, LDAP, NIS, /etc/hosts,
-        or some other means. If the netloc4 is of type NL4_URL, a
-        server URL <xref target="RFC3986" /> appropriate for the
-        server-to-server copy operation is specified as a UTF-8 string.
-        If the netloc4 is of type NL4_NETADDR, the nl_addr field MUST
-        contain a valid netaddr4 as defined in Section 3.3.9 of
-        <xref target="RFC5661" />.
-      </t>
-
-      <t>
-        When netloc4 values are used for an inter-server copy as
-        shown in <xref target="fig:inter-server" />, their values
-        may be evaluated on the source server, destination server,
-        and client. The network environment in which these systems
-        operate should be configured so that the netloc4 values
-        are interpreted as intended on each system.
-      </t>
-    </section>
-
     <section title="Copy Offload Stateids" anchor="ss:copy:stateid">
       <t>
         A server may perform a copy offload operation asynchronously.  An
@@ -575,508 +251,5 @@
       may be used to secure the protocol described in this chapter.
     </t>
 
-    <t>
-      NFSv4 clients and servers supporting the inter-server copy
-      operations described in this chapter are REQUIRED to implement
-      <xref target="rpcsecgssv3" />, including the RPCSEC_GSSv3
-      privileges copy_from_auth and copy_to_auth. If the server-to-server
-      copy protocol is ONC RPC based, the servers are also REQUIRED to
-      implement the RPCSEC_GSSv3 privilege copy_confirm_auth. These
-      requirements to implement are not requirements to use. NFSv4 clients
-      and servers are RECOMMENDED to use <xref target="rpcsecgssv3" />
-      to secure server-side copy operations.
-    </t>
-
-    <section title="Inter-Server Copy Security" anchor="server-to-server-sec">
-      <section title="Requirements for Secure Inter-Server Copy">
-        <t>
-          Inter-server copy is driven by several requirements:
-        </t>
-
-        <t>
-          <list style="symbols">
-            <t>
-              The specification MUST NOT mandate an inter-server copy
-              protocol. There are many ways to copy data. Some will be
-              more optimal than others depending on the identities of
-              the source server and destination server. For example
-              the source and destination servers might be two nodes sharing
-              a common file system format for the source and destination
-              file systems. Thus the source and destination are in an
-              ideal position to efficiently render the image of the source
-              file to the destination file by replicating
-              the file system formats at the block level. In other cases, the
-              source and destination might be two nodes
-              sharing a common storage area network, and thus there is no
-              need to copy any data at all, and instead
-              ownership of the file and its contents simply gets re-assigned
-              to the destination.
-            </t>
-
-            <t>
-              The specification MUST provide guidance for using NFSv4.x as
-              a copy protocol. For those source and destination servers
-              willing to use NFSv4.x there are specific security considerations
-              that this specification can and does address.
-            </t>
-
-            <t>
-              The specification MUST NOT mandate pre-configuration between the
-              source and destination server. Requiring that the source and
-              destination first have a "copying relationship" increases the
-              administrative burden. However the specification MUST NOT
-              preclude implementations that require pre-configuration.
-            </t>
-
-            <t>
-              The specification MUST NOT mandate a trust relationship between
-              the source and destination server. The NFSv4 security model
-              requires mutual authentication between a principal on an NFS
-              client and a principal on an NFS server. This model MUST continue
-              with the introduction of COPY.
-            </t>
-          </list>
-        </t>
-      </section>
-
-      <section title="Inter-Server Copy with RPCSEC_GSSv3">
-        <t>
-          When the client sends a COPY_NOTIFY to the source server to expect
-          the destination to attempt to copy data from the source server, it
-          is expected that this copy is being done on behalf of the principal
-          (called the "user principal") that sent the RPC request that encloses
-          the COMPOUND procedure that contains the COPY_NOTIFY operation. The
-          user principal is identified by the RPC credentials. A mechanism that
-          allows the user principal to authorize the destination server to perform
-          the copy in a manner that lets the source server properly authenticate
-          the destination's copy, and without allowing the destination to
-          exceed its authorization is necessary.
-        </t>
-
-        <t>
-          An approach that sends delegated credentials of the client's user
-          principal to the destination server is not used for the following
-          reasons. If the client's user delegated its credentials, the
-          destination would authenticate as the user principal. If the
-          destination were using the NFSv4 protocol to perform the copy,
-          then the source server would authenticate the destination server
-          as the user principal, and the file copy would securely proceed.
-          However, this approach would allow the destination server to
-          copy other files. The user principal would have to trust the
-          destination server to not do so. This is counter to the requirements,
-          and therefore is not considered. Instead an approach using
-          RPCSEC_GSSv3 <xref target="rpcsecgssv3" /> privileges is proposed.
-        </t>
-
-        <t>
-          One of the stated applications of the proposed RPCSEC_GSSv3
-          protocol is compound client host and user authentication
-          [+ privilege assertion]. For inter-server file copy, we require
-          compound NFS server host and user authentication
-          [+ privilege assertion]. The distinction between the two is one
-          without meaning.
-        </t>
-
-        <t>
-          RPCSEC_GSSv3 introduces the notion of privileges. We define three privileges:
-        </t>
-
-        <t>
-          <list style="hanging">
-
-            <t hangText="copy_from_auth:">
-              A user principal is authorizing a source principal
-              ("nfs@&lt;source&gt;") to allow a destination principal
-              ("nfs@&lt;destination&gt;") to copy a file from the source
-              to the destination. This privilege is established on the
-              source server before the user principal sends a COPY_NOTIFY
-              operation to the source server.
-
-              <vspace blankLines="1" />
-              <?rfc include='autogen/copy_from_auth.xml'?>
-              <vspace blankLines="1" />
-
-              cfp_shared_secret is a secret value the user
-              principal generates.
-            </t>
-
-            <t hangText="copy_to_auth:">
-              A user principal is authorizing a destination principal
-              ("nfs@&lt;destination&gt;") to allow it to copy a file
-              from the source to the destination. This privilege is
-              established on the destination server before the user
-              principal sends a COPY operation to the destination server.
-
-              <vspace blankLines="1" />
-              <?rfc include='autogen/copy_to_auth.xml'?>
-              <vspace blankLines="1" />
-
-              ctap_shared_secret is a secret value the user principal
-              generated and was used to establish the copy_from_auth
-              privilege with the source principal.
-            </t>
-
-            <t hangText="copy_confirm_auth:">
-              A destination principal is confirming with the
-              source principal that it is authorized to copy data
-              from the source on behalf of the user principal. When
-              the inter-server copy protocol is NFSv4, or for that
-              matter, any protocol capable of being secured via
-              RPCSEC_GSSv3 (i.e., any ONC RPC protocol), this privilege
-              is established before the file is copied from the
-              source to the destination.
-
-              <vspace blankLines="1" />
-              <?rfc include='autogen/copy_confirm_auth.xml'?>
-            </t>
-          </list>
-        </t>
-
-        <section title="Establishing a Security Context">
-          <t>
-            When the user principal wants to COPY a file between two servers, if it has
-            not established copy_from_auth and copy_to_auth privileges on the servers,
-            it establishes them:
-          </t>
-
-          <t>
-            <list style="symbols">
-
-              <t>
-                The user principal generates a secret it will share with the two
-                servers. This shared secret will be placed in the cfap_shared_secret
-                and ctap_shared_secret fields of the appropriate privilege data types,
-                copy_from_auth_priv and copy_to_auth_priv.
-              </t>
-
-              <t>
-                An instance of copy_from_auth_priv is filled in with the shared
-                secret, the destination server, and the NFSv4 user id of the
-                user principal. It will be sent with an RPCSEC_GSS3_CREATE
-                procedure, and so cfap_seq_num is set to the seq_num of the
-                credential of the RPCSEC_GSS3_CREATE procedure. Because
-                cfap_shared_secret is a secret, after XDR encoding
-                copy_from_auth_priv, GSS_Wrap() (with privacy) is invoked
-                on copy_from_auth_priv. The RPCSEC_GSS3_CREATE procedure's
-                arguments are:
-
-                <vspace blankLines="1" />
-
-                <figure>
-                  <artwork>
-   struct {
-      rpc_gss3_gss_binding    *compound_binding;
-      rpc_gss3_chan_binding   *chan_binding_mic;
-      rpc_gss3_assertion      assertions&lt;&gt;;
-      rpc_gss3_extension      extensions&lt;&gt;;
-   } rpc_gss3_create_args;
-                 </artwork>
-                </figure>
-
-                <vspace blankLines="1" />
-
-                The string "copy_from_auth" is placed in assertions[0].privs. The
-                output of GSS_Wrap() is placed in extensions[0].data. The field
-                extensions[0].critical is set to TRUE. The source server calls
-                GSS_Unwrap() on the privilege, and verifies that the seq_num matches
-                the credential. It then verifies that the NFSv4 user id being asserted
-                matches the source server's mapping of the user principal. If it
-                does, the privilege is established on the source server as:
-                &lt;"copy_from_auth", user id, destination&gt;. The successful
-                reply to RPCSEC_GSS3_CREATE has:
-
-                <vspace blankLines="1" />
-
-                <figure>
-                  <artwork>
-   struct {
-      opaque                  handle&lt;&gt;;
-      rpc_gss3_chan_binding   *chan_binding_mic;
-      rpc_gss3_assertion      granted_assertions&lt;&gt;;
-      rpc_gss3_assertion      server_assertions&lt;&gt;;
-      rpc_gss3_extension      extensions&lt;&gt;;
-   } rpc_gss3_create_res;
-                  </artwork>
-                </figure>
-
-                <vspace blankLines="1" />
-
-                The field "handle" is the RPCSEC_GSSv3 handle that the client
-                will use on COPY_NOTIFY requests involving the source and
-                destination server. granted_assertions[0].privs will be equal to
-                "copy_from_auth". The server will return a GSS_Wrap() of
-                copy_to_auth_priv.
-              </t>
-
-              <t>
-                An instance of copy_to_auth_priv is filled in with the
-                shared secret, the source server, and the NFSv4 user id.
-                It will be sent with an RPCSEC_GSS3_CREATE procedure, and
-                so ctap_seq_num is set to the seq_num of the credential of
-                the RPCSEC_GSS3_CREATE procedure. Because ctap_shared_secret
-                is a secret, after XDR encoding copy_to_auth_priv,
-                GSS_Wrap() is invoked on copy_to_auth_priv. The RPCSEC_GSS3_CREATE
-                procedure's arguments are:
-
-                <vspace blankLines="1" />
-
-                <figure>
-                  <artwork>
-   struct {
-      rpc_gss3_gss_binding    *compound_binding;
-      rpc_gss3_chan_binding   *chan_binding_mic;
-      rpc_gss3_assertion      assertions&lt;&gt;;
-      rpc_gss3_extension      extensions&lt;&gt;;
-   } rpc_gss3_create_args;
-                  </artwork>
-                </figure>
-
-                <vspace blankLines="1" />
-
-                The string "copy_to_auth" is placed in assertions[0].privs.
-                The output of GSS_Wrap() is placed in extensions[0].data. The
-                field extensions[0].critical is set to TRUE. After unwrapping,
-                verifying the seq_num, and the user principal to NFSv4 user
-                ID mapping, the destination establishes a privilege of
-                &lt;"copy_to_auth", user id, source&gt;. The successful reply
-                to RPCSEC_GSS3_CREATE has:
-
-                <vspace blankLines="1" />
-
-                <figure>
-                  <artwork>
-   struct {
-      opaque                  handle&lt;&gt;;
-      rpc_gss3_chan_binding   *chan_binding_mic;
-      rpc_gss3_assertion      granted_assertions&lt;&gt;;
-      rpc_gss3_assertion      server_assertions&lt;&gt;;
-      rpc_gss3_extension      extensions&lt;&gt;;
-   } rpc_gss3_create_res;
-                  </artwork>
-                </figure>
-
-                <vspace blankLines="1" />
-
-                The field "handle" is the RPCSEC_GSSv3 handle that the client will
-                use on COPY requests involving the source and destination server.
-                The field granted_assertions[0].privs will be equal to "copy_to_auth".
-                The server will return a GSS_Wrap() of copy_to_auth_priv.
-              </t>
-            </list>
-          </t>
-        </section>
-
-        <section title="Starting a Secure Inter-Server Copy">
-          <t>
-            When the client sends a COPY_NOTIFY request to the source server, it uses
-            the privileged "copy_from_auth" RPCSEC_GSSv3 handle. cna_destination_server
-            in COPY_NOTIFY MUST be the same as the name of the destination server
-            specified in copy_from_auth_priv. Otherwise, COPY_NOTIFY will fail with
-            NFS4ERR_ACCESS. The source server verifies that the privilege
-            &lt;"copy_from_auth", user id, destination&gt; exists, and annotates it
-            with the source filehandle, if the user principal has read
-            access to the source file, and if administrative policies give the user
-            principal and the NFS client read access to the source file (i.e., if the
-            ACCESS operation would grant read access). Otherwise, COPY_NOTIFY will fail
-            with NFS4ERR_ACCESS.
-          </t>
-
-          <t>
-            When the client sends a COPY request to the destination server, it uses
-            the privileged "copy_to_auth" RPCSEC_GSSv3 handle. ca_source_server in
-            COPY MUST be the same as the name of the source server specified in
-            copy_to_auth_priv. Otherwise, COPY will fail with NFS4ERR_ACCESS. The
-            destination server verifies that the privilege
-            &lt;"copy_to_auth", user id, source&gt; exists, and annotates it with the
-            source and destination filehandles. If the client has failed to establish
-            the "copy_to_auth" policy it will reject the request with
-            NFS4ERR_PARTNER_NO_AUTH.
-          </t>
-
-          <t>
-            If the client sends a OFFLOAD_REVOKE to the source server to rescind the
-            destination server's copy privilege, it uses the privileged
-            "copy_from_auth" RPCSEC_GSSv3 handle and the cra_destination_server
-            in OFFLOAD_REVOKE MUST be the same as the name of the destination server
-            specified in copy_from_auth_priv. The source server will then
-            delete the &lt;"copy_from_auth", user id, destination&gt; privilege
-            and fail any subsequent copy requests sent under the auspices of this
-            privilege from the destination server.
-          </t>
-        </section>
-
-        <section title="Securing ONC RPC Server-to-Server Copy Protocols">
-          <t>
-            After a destination server has a "copy_to_auth" privilege established on
-            it, and it receives a COPY request, if it knows it will use an ONC RPC
-            protocol to copy data, it will establish a "copy_confirm_auth" privilege
-            on the source server, using nfs@&lt;destination&gt; as the initiator
-            principal, and nfs@&lt;source&gt; as the target principal.
-          </t>
-
-          <t>
-            The value of the field ccap_shared_secret_mic is a GSS_VerifyMIC() of
-            the shared secret passed in the copy_to_auth privilege.  The field
-            ccap_username is the mapping of the user principal to an NFSv4 user
-            name ("user"@"domain" form), and MUST be the same as ctap_username
-            and cfap_username. The field ccap_seq_num is the seq_num of the
-            RPCSEC_GSSv3 credential used for the RPCSEC_GSS3_CREATE procedure
-            the destination will send to the source server to establish the
-            privilege.
-          </t>
-
-          <t>
-            The source server verifies the privilege, and establishes a
-            &lt;"copy_confirm_auth", user id, destination&gt; privilege. If the
-            source server fails to verify the privilege, the COPY operation will
-            be rejected with NFS4ERR_PARTNER_NO_AUTH. All subsequent ONC RPC
-            requests sent from the destination to copy data from the source to
-            the destination will use the RPCSEC_GSSv3 handle returned by the
-            source's RPCSEC_GSS3_CREATE response.
-          </t>
-
-          <t>
-            Note that the use of the "copy_confirm_auth" privilege
-            accomplishes the following:
-          </t>
-
-          <t>
-            <list style="symbols">
-              <t>
-                if a protocol like NFS is being used, with export policies,
-                export policies can be overridden in case the destination server
-                as-an-NFS-client is not authorized
-              </t>
-
-              <t>
-                manual configuration to allow a copy relationship between the source
-                and destination is not needed.
-              </t>
-            </list>
-          </t>
-
-          <t>
-            If the attempt to establish a "copy_confirm_auth" privilege fails,
-            then when the user principal sends a COPY request to destination,
-            the destination server will reject it with NFS4ERR_PARTNER_NO_AUTH.
-          </t>
-        </section>
-
-        <section title="Securing Non ONC RPC Server-to-Server Copy Protocols"
-         anchor="server-to-server-no-rpc-with-gss">
-          <t>
-            If the destination won't be using ONC RPC to copy the data, then
-            the source and destination are using an unspecified copy protocol.
-            The destination could use the shared secret and the NFSv4 user id to
-            prove to the source server that the user principal has authorized
-            the copy.
-          </t>
-
-          <t>
-            For protocols that authenticate user names with passwords
-            (e.g., HTTP <xref target="RFC2616" /> and FTP <xref target="RFC0959" />),
-            the NFSv4 user id could be used as the user name, and an ASCII
-            hexadecimal representation of the RPCSEC_GSSv3 shared secret could be
-            used as the user password or as input into non-password authentication
-            methods like CHAP <xref target="RFC1994" />.
-          </t>
-        </section>
-      </section>
-
-      <section title="Inter-Server Copy via ONC RPC but without RPCSEC_GSSv3"
-       anchor="server-to-server-with-rpc-no-gss">
-        <t>
-          ONC RPC security flavors other than RPCSEC_GSSv3 MAY be used with
-          the server-side copy offload operations described in this
-          chapter. In particular, host-based ONC RPC security flavors
-          such as AUTH_NONE and AUTH_SYS MAY be used. If a host-based
-          security flavor is used, a minimal level of protection for
-          the server-to-server copy protocol is possible.
-        </t>
-
-        <t>
-          In the absence of strong security mechanisms such as
-          RPCSEC_GSSv3, the challenge is how the source server
-          and destination server identify themselves to each other,
-          especially in the presence of multi-homed source and
-          destination servers. In a multi-homed environment, the
-          destination server might not contact the source server
-          from the same network address specified by the client
-          in the COPY_NOTIFY. This can be overcome using the
-          procedure described below.
-        </t>
-
-        <t>
-          When the client sends the source server the COPY_NOTIFY operation,
-          the source server may reply to the client with a list of target
-          addresses, names, and/or URLs and assign them to the unique quadruple:
-          &lt;random number, source fh, user ID, destination address Y&gt;.  If the destination
-          uses one of these target netlocs to contact the source server, the source
-          server will be able to uniquely identify the destination server, even
-          if the destination server does not connect from the address specified
-          by the client in COPY_NOTIFY.  The level of assurance in this identification
-          depends on the unpredictability, strength and secrecy of the random number.
-        </t>
-
-        <t>
-          For example, suppose the network topology is as shown in
-          <xref target="fig:inter-server" />.  If the source filehandle
-          is 0x12345, the source server may respond to a COPY_NOTIFY
-          for destination 203.0.113.56 with the URLs:
-        </t>
-
-        <t>
-          <list style="empty">
-            <t>nfs://203.0.113.18//_COPY/FvhH1OKbu8VrxvV1erdjvR7N/203.0.113.56/_FH/0x12345</t>
-            <t>nfs://192.0.2.18//_COPY/FvhH1OKbu8VrxvV1erdjvR7N/203.0.113.56/_FH/0x12345</t>
-          </list>
-        </t>
-
-        <t>
-          The name component after _COPY is 24 characters of base 64, more than enough
-          to encode a 128 bit random number.
-        </t>
-
-        <t>
-          The client will then send these URLs to the destination server
-          in the COPY operation. Suppose that the 192.0.2.0/24 network
-          is a high speed network and the destination server decides to
-          transfer the file over this network. If the destination
-          contacts the source server from 192.0.2.56 over this network
-          using NFSv4.1, it does the following:
-        </t>
-
-        <t>
-          <list style="hanging">
-            <t hangText="COMPOUND"> { PUTROOTFH, LOOKUP "_COPY" ;
-              LOOKUP "FvhH1OKbu8VrxvV1erdjvR7N" ;
-              LOOKUP "203.0.113.56";
-              LOOKUP "_FH" ; OPEN "0x12345" ; GETFH }
-            </t>
-          </list>
-        </t>
-
-        <t>
-          Provided that the random number is unpredictable and has been kept secret
-          by the parties involved, the source server will therefore know that these
-          NFSv4.x operations are being issued by the destination server identified
-          in the COPY_NOTIFY.  This random number technique only provides initial
-          authentication of the destination server, and cannot defend against
-          man-in-the-middle attacks after authentication or an eavesdropper that
-          observes the random number on the wire. Other secure communication
-          techniques (e.g., IPsec) are necessary to block these attacks.
-        </t>
-      </section>
-
-      <section title="Inter-Server Copy without ONC RPC and RPCSEC_GSSv3"
-         anchor="server-to-server-no-rpc-no-gss">
-        <t>
-          The same techniques as <xref target="server-to-server-with-rpc-no-gss" />,
-          using unique URLs for each destination server, can be used for other
-          protocols (e.g., HTTP <xref target="RFC2616" /> and FTP <xref target="RFC0959" />)
-          as well.
-        </t>
-      </section>
-    </section>
   </section>
 </section>
diff --git a/nfsv42_middle_errors.xml b/nfsv42_middle_errors.xml
index 9b2b6a7..8c0c897 100644
--- a/nfsv42_middle_errors.xml
+++ b/nfsv42_middle_errors.xml
@@ -87,13 +87,10 @@
       <c>NFS4ERR_NOT_SAME                  </c> <c> 10027  </c> <c> <xref target="ss:error_values:NC182" /></c>
       <c>NFS4ERR_NO_GRACE                  </c> <c> 10033  </c> <c> <xref target="ss:error_values:NC149" /> </c>
       <c>NFS4ERR_NXIO                      </c> <c> 6      </c> <c> <xref target="ss:error_values:NC51" /></c>
-      <c>NFS4ERR_OFFLOAD_DENIED            </c> <c> 10091  </c> <c> <xref target="ss:error_values:NC191" /></c>
       <c>NFS4ERR_OLD_STATEID               </c> <c> 10024  </c> <c> <xref target="ss:error_values:NC131" /> </c>
       <c>NFS4ERR_OPENMODE                  </c> <c> 10038  </c> <c> <xref target="ss:error_values:NC145" /> </c>
       <c>NFS4ERR_OP_ILLEGAL                </c> <c> 10044  </c> <c> <xref target="ss:error_values:NC113" /> </c>
       <c>NFS4ERR_OP_NOT_IN_SESSION         </c> <c> 10071  </c> <c> <xref target="ss:error_values:NC114" /> </c>
-      <c>NFS4ERR_PARTNER_NO_AUTH           </c> <c> 10089  </c> <c> <xref target="ss:error_values:NC192" /> </c>
-      <c>NFS4ERR_PARTNER_NOTSUPP           </c> <c> 10088  </c> <c> <xref target="ss:error_values:NC193" /> </c>
       <c>NFS4ERR_PERM                      </c> <c> 1      </c> <c> <xref target="ss:error_values:NC133" /> </c>
       <c>NFS4ERR_PNFS_IO_HOLE              </c> <c> 10075  </c> <c> <xref target="ss:error_values:NC157" /></c>
       <c>NFS4ERR_PNFS_NO_LAYOUT            </c> <c> 10080  </c> <c> <xref target="ss:error_values:NC158" /></c>
@@ -1173,31 +1170,6 @@
         </t>
       </section>
 
-      <section title="NFS4ERR_OFFLOAD_DENIED (Error Code 10091)" anchor="err_OFFLOAD_DENIED">
-        <t>
-          The copy offload operation is supported by
-          both the source and the destination, but the destination is not
-          allowing it for this file.  If the client sees this error, it
-          should fall back to the normal copy semantics.
-        </t>
-      </section>
-
-      <section title="NFS4ERR_PARTNER_NO_AUTH (Error Code 10089)" anchor="err_PARTNER_NO_AUTH">
-        <t>
-          The remote server does not authorize a
-          server-to-server copy offload operation.  This may be due to the
-          client's failure to send the COPY_NOTIFY operation to the remote
-          server, the remote server receiving a server-to-server copy
-          offload request after the copy lease time expired, or for some
-          other permission problem.
-        </t>
-      </section>
-
-      <section title="NFS4ERR_PARTNER_NOTSUPP (Error Code 10088)" anchor="err_PARTNER_NOTSUPP">
-        <t>
-          The remote server does not support the server-to-server copy offload protocol.
-        </t>
-      </section>
     </section>
 
     <section title="Labeled NFS Errors" anchor="errors_lnfs">
@@ -1442,12 +1414,9 @@
         NFS4ERR_MOVED,
         NFS4ERR_NOFILEHANDLE,
         NFS4ERR_NOSPC,
-        NFS4ERR_OFFLOAD_DENIED,
         NFS4ERR_OLD_STATEID,
         NFS4ERR_OPENMODE,
         NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_PARTNER_NO_AUTH,
-        NFS4ERR_PARTNER_NOTSUPP,
         NFS4ERR_PNFS_IO_HOLE,
         NFS4ERR_PNFS_NO_LAYOUT,
         NFS4ERR_REP_TOO_BIG,
@@ -1462,40 +1431,6 @@
         NFS4ERR_WRONG_TYPE
       </c>
 
-      <c>COPY_NOTIFY</c>
-      <c>
-        NFS4ERR_ACCESS,
-        NFS4ERR_ADMIN_REVOKED,
-        NFS4ERR_BADXDR,
-        NFS4ERR_BAD_STATEID,
-        NFS4ERR_DEADSESSION,
-        NFS4ERR_DELAY,
-        NFS4ERR_DELEG_REVOKED,
-        NFS4ERR_EXPIRED,
-        NFS4ERR_FHEXPIRED,
-        NFS4ERR_GRACE,
-        NFS4ERR_INVAL,
-        NFS4ERR_ISDIR,
-        NFS4ERR_IO,
-        NFS4ERR_LOCKED,
-        NFS4ERR_MOVED,
-        NFS4ERR_NOFILEHANDLE,
-        NFS4ERR_OLD_STATEID,
-        NFS4ERR_OPENMODE,
-        NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_PNFS_IO_HOLE,
-        NFS4ERR_PNFS_NO_LAYOUT,
-        NFS4ERR_REP_TOO_BIG,
-        NFS4ERR_REP_TOO_BIG_TO_CACHE,
-        NFS4ERR_REQ_TOO_BIG,
-        NFS4ERR_RETRY_UNCACHED_REP,
-        NFS4ERR_SERVERFAULT,
-        NFS4ERR_STALE,
-        NFS4ERR_SYMLINK,
-        NFS4ERR_TOO_MANY_OPS,
-        NFS4ERR_WRONG_TYPE
-      </c>
-
       <c>CREATE</c>
       <c>
         NFS4ERR_ACCESS,
@@ -2110,21 +2045,6 @@
         NFS4ERR_TOO_MANY_OPS
       </c>
 
-      <c>OFFLOAD_REVOKE</c>
-      <c>
-        NFS4ERR_ADMIN_REVOKED,
-        NFS4ERR_BADXDR,
-        NFS4ERR_COMPLETE_ALREADY,
-        NFS4ERR_DELAY,
-        NFS4ERR_GRACE,
-        NFS4ERR_INVALID,
-        NFS4ERR_MOVED,
-        NFS4ERR_NOTSUPP,
-        NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_SERVERFAULT,
-        NFS4ERR_TOO_MANY_OPS
-      </c>
-
       <c>OFFLOAD_STATUS</c>
       <c>
         NFS4ERR_ADMIN_REVOKED,
diff --git a/nfsv42_middle_new_errors.xml b/nfsv42_middle_new_errors.xml
index 957a6af..4246fe7 100644
--- a/nfsv42_middle_new_errors.xml
+++ b/nfsv42_middle_new_errors.xml
@@ -22,9 +22,6 @@
 
       <c>NFS4ERR_BADLABEL</c>         <c>10093</c> <c><xref target="err_BADLABEL" /></c>
       <c>NFS4ERR_METADATA_NOTSUPP</c> <c>10090</c> <c><xref target="err_METADATA_NOTSUPP" /></c>
-      <c>NFS4ERR_OFFLOAD_DENIED</c>   <c>10091</c> <c><xref target="err_OFFLOAD_DENIED" /></c>
-      <c>NFS4ERR_PARTNER_NO_AUTH</c>  <c>10089</c> <c><xref target="err_PARTNER_NO_AUTH" /></c>
-      <c>NFS4ERR_PARTNER_NOTSUPP</c>  <c>10088</c> <c><xref target="err_PARTNER_NOTSUPP" /></c>
       <c>NFS4ERR_UNION_NOTSUPP</c>    <c>10094</c> <c><xref target="err_UNION_NOTSUPP" /></c>
       <c>NFS4ERR_WRONG_LFS</c>        <c>10092</c> <c><xref target="err_WRONG_LFS" /></c>
     </texttable>
@@ -56,31 +53,6 @@
         </t>
       </section>
 
-      <section title="NFS4ERR_OFFLOAD_DENIED (Error Code 10091)" anchor="err_OFFLOAD_DENIED">
-        <t>
-          The copy offload operation is supported by
-          both the source and the destination, but the destination is not
-          allowing it for this file.  If the client sees this error, it
-          should fall back to the normal copy semantics.
-        </t>
-      </section>
-
-      <section title="NFS4ERR_PARTNER_NO_AUTH (Error Code 10089)" anchor="err_PARTNER_NO_AUTH">
-        <t>
-          The source server does not authorize a
-          server-to-server copy offload operation.  This may be due to the
-          client's failure to send the COPY_NOTIFY operation to the source
-          server, the source server receiving a server-to-server copy
-          offload request after the copy lease time expired, or for some
-          other permission problem.
-        </t>
-      </section>
-
-      <section title="NFS4ERR_PARTNER_NOTSUPP (Error Code 10088)" anchor="err_PARTNER_NOTSUPP">
-        <t>
-          The remote server does not support the server-to-server copy offload protocol.
-        </t>
-      </section>
     </section>
 
     <section title="Labeled NFS Errors" anchor="errors_lnfs">
@@ -141,12 +113,9 @@
         NFS4ERR_MOVED,
         NFS4ERR_NOFILEHANDLE,
         NFS4ERR_NOSPC,
-        NFS4ERR_OFFLOAD_DENIED,
         NFS4ERR_OLD_STATEID,
         NFS4ERR_OPENMODE,
         NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_PARTNER_NO_AUTH,
-        NFS4ERR_PARTNER_NOTSUPP,
         NFS4ERR_PNFS_IO_HOLE,
         NFS4ERR_PNFS_NO_LAYOUT,
         NFS4ERR_REP_TOO_BIG,
@@ -161,40 +130,6 @@
         NFS4ERR_WRONG_TYPE
       </c>
 
-      <c>COPY_NOTIFY</c>
-      <c>
-        NFS4ERR_ACCESS,
-        NFS4ERR_ADMIN_REVOKED,
-        NFS4ERR_BADXDR,
-        NFS4ERR_BAD_STATEID,
-        NFS4ERR_DEADSESSION,
-        NFS4ERR_DELAY,
-        NFS4ERR_DELEG_REVOKED,
-        NFS4ERR_EXPIRED,
-        NFS4ERR_FHEXPIRED,
-        NFS4ERR_GRACE,
-        NFS4ERR_INVAL,
-        NFS4ERR_ISDIR,
-        NFS4ERR_IO,
-        NFS4ERR_LOCKED,
-        NFS4ERR_MOVED,
-        NFS4ERR_NOFILEHANDLE,
-        NFS4ERR_OLD_STATEID,
-        NFS4ERR_OPENMODE,
-        NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_PNFS_IO_HOLE,
-        NFS4ERR_PNFS_NO_LAYOUT,
-        NFS4ERR_REP_TOO_BIG,
-        NFS4ERR_REP_TOO_BIG_TO_CACHE,
-        NFS4ERR_REQ_TOO_BIG,
-        NFS4ERR_RETRY_UNCACHED_REP,
-        NFS4ERR_SERVERFAULT,
-        NFS4ERR_STALE,
-        NFS4ERR_SYMLINK,
-        NFS4ERR_TOO_MANY_OPS,
-        NFS4ERR_WRONG_TYPE
-      </c>
-
       <c>OFFLOAD_ABORT</c>
       <c>
         NFS4ERR_ADMIN_REVOKED,
@@ -212,21 +147,6 @@
         NFS4ERR_TOO_MANY_OPS
       </c>
 
-      <c>OFFLOAD_REVOKE</c>
-      <c>
-        NFS4ERR_ADMIN_REVOKED,
-        NFS4ERR_BADXDR,
-        NFS4ERR_COMPLETE_ALREADY,
-        NFS4ERR_DELAY,
-        NFS4ERR_GRACE,
-        NFS4ERR_INVALID,
-        NFS4ERR_MOVED,
-        NFS4ERR_NOTSUPP,
-        NFS4ERR_OP_NOT_IN_SESSION,
-        NFS4ERR_SERVERFAULT,
-        NFS4ERR_TOO_MANY_OPS
-      </c>
-
       <c>OFFLOAD_STATUS</c>
       <c>
         NFS4ERR_ADMIN_REVOKED,
diff --git a/nfsv42_middle_op_cb_offload.xml b/nfsv42_middle_op_cb_offload.xml
index 348bd2c..de470b3 100644
--- a/nfsv42_middle_op_cb_offload.xml
+++ b/nfsv42_middle_op_cb_offload.xml
@@ -47,9 +47,9 @@
 
     <section toc='exclude' title='Server-side Copy'>
       <t>
-        CB_OFFLOAD is used for both intra- and inter-server asynchronous
+        CB_OFFLOAD is used for asynchronous
         copies. This operation is sent by the
-        destination server to the client in a CB_COMPOUND request.
+        server to the client in a CB_COMPOUND request.
         Upon success, the coa_resok4.wr_count presents the total
         number of bytes copied.
       </t>
diff --git a/nfsv42_middle_op_copy.xml b/nfsv42_middle_op_copy.xml
index 104a1d5..00a5b57 100644
--- a/nfsv42_middle_op_copy.xml
+++ b/nfsv42_middle_op_copy.xml
@@ -12,9 +12,7 @@
 
   <section toc='exclude' title="DESCRIPTION">
     <t>
-      The COPY operation is used for both intra-server and inter-server
-      copies. In both cases, the COPY is always sent from the client
-      to the destination server of the file copy. The COPY operation
+      The COPY operation
       requests that a file be copied from the location specified by the
       SAVED_FH value to the location specified by the combination of
       CURRENT_FH and ca_destination.
@@ -39,33 +37,10 @@
     </figure>
 
     <t>
-      If the request is for a server-to-server copy, the source-fh
-      is a filehandle from the source server and the compound
-      procedure is being executed on the destination server. In
-      this case, the source-fh is a foreign filehandle on the
-      server receiving the COPY request. If either PUTFH or
-      SAVEFH checked the validity of the filehandle, the operation
-      would likely fail and return NFS4ERR_STALE.
-    </t>
-
-    <t>
-      If a server supports the
-      server-to-server COPY feature, a PUTFH followed by a SAVEFH
-      MUST NOT return NFS4ERR_STALE for either operation. These
-      restrictions do not pose substantial difficulties for
-      servers. The CURRENT_FH and SAVED_FH may be validated in the
-      context of the operation referencing them and an NFS4ERR_STALE
-      error returned for an invalid file handle at that point.
-    </t>
-
-    <t>
-      For an intra-server copy, both the ca_src_stateid and
+      Both the ca_src_stateid and
       ca_dst_stateid MUST refer to either open or locking
       states provided earlier by the server. If either
       stateid is invalid, then the operation MUST fail.
-      If the request is for a inter-server copy, then the
-      ca_src_stateid can be ignored. If ca_dst_stateid
-      is invalid, then the operation MUST fail.
     </t>
 
     <t>
@@ -131,21 +106,6 @@
     </t>
 
     <t>
-      If the ca_source_server list is specified, then this is an inter-server
-      copy operation and the source file is on a remote server.  The client
-      is expected to have previously issued a successful COPY_NOTIFY request
-      to the remote source server. The ca_source_server list MUST be the
-      same as the COPY_NOTIFY response's cnr_source_server list. If the client
-      includes the entries from the COPY_NOTIFY response's cnr_source_server
-      list in the ca_source_server list, the source server can indicate a
-      specific copy protocol for the destination server to use by returning
-      a URL, which specifies both a protocol service and server name.
-      Server-to-server copy protocol considerations are described in
-      <xref target="server-to-server-proto" /> and
-      <xref target="server-to-server-sec" />.
-    </t>
-
-    <t>
       The ca_flags argument allows the copy operation to be
       customized in the following ways using the guarded flag
       (COPY4_GUARDED) and the metadata flag (COPY4_METADATA).
@@ -285,7 +245,7 @@
     </t>
 
     <t>
-      In the case of an inter-server copy or an intra-server copy
+      In the case of a copy
       between file systems, the attributes supported for the source
       file and destination file could be different. By definition,the
       REQUIRED attributes will be supported in all cases. If the
@@ -295,7 +255,7 @@
     </t>
 
     <t>
-      Any attribute supported by the destination server that is not
+      Any attribute supported by the destination filesystem that is not
       set on the source file SHOULD be left unset.
     </t>
 
diff --git a/nfsv42_middle_op_copy_notify.xml b/nfsv42_middle_op_copy_notify.xml
deleted file mode 100644
index dcca637..0000000
--- a/nfsv42_middle_op_copy_notify.xml
+++ /dev/null
@@ -1,89 +0,0 @@
-<!-- Copyright (C) The IETF Trust (2011-2012) -->
-<!-- Copyright (C) The Internet Society (2011-2012) -->
-
-<section anchor="op:copy_notify" title="Operation 61: COPY_NOTIFY - Notify a source server of a future copy">
-  <section toc='exclude' title="ARGUMENT">
-    <?rfc include='autogen/copy_notify_args.xml'?>
-  </section>
-
-  <section toc='exclude' title="RESULT">
-    <?rfc include='autogen/copy_notify_res.xml'?>
-  </section>
-
-  <section toc='exclude' title="DESCRIPTION">
-    <t>
-      This operation is used for an inter-server copy. A client
-      sends this operation in a COMPOUND request to the source
-      server to authorize a destination server identified by
-      cna_destination_server to read the file specified by
-      CURRENT_FH on behalf of the given user.
-    </t>
-
-    <t>
-      The cna_src_stateid MUST refer to either open or locking
-      states provided earlier by the server. If it is invalid,
-      then the operation MUST fail.
-    </t>
-
-    <t>
-      The cna_destination_server MUST be specified using the
-      netloc4 network location format. The server is not required
-      to resolve the cna_destination_server address before completing
-      this operation.
-    </t>
-
-    <t>
-      If this operation succeeds, the source server will allow the
-      cna_destination_server to copy the specified file on behalf
-      of the given user as long as both of the following
-      conditions are met:
-    </t>
-
-    <t>
-      <list style="symbols">
-        <t>
-          The destination server begins reading the source file
-          before the cnr_lease_time expires. If the cnr_lease_time
-          expires while the destination server is still reading
-          the source file, the destination server is allowed to
-          finish reading the file.
-        </t>
-
-        <t>
-          The client has not issued a COPY_REVOKE for
-          the same combination of user, filehandle,
-          and destination server.
-        </t>
-      </list>
-    </t>
-
-    <t>
-      The cnr_lease_time is chosen by the source server. A
-      cnr_lease_time of 0 (zero) indicates an infinite lease.
-      To avoid the need for synchronized clocks, copy lease times
-      are granted by the server as a time delta.
-      To renew the copy lease time the client should resend the same
-      copy notification request to the source server.
-    </t>
-
-    <t>
-      A successful response will also contain a list of netloc4
-      network location formats
-      called cnr_source_server, on which the source is willing to
-      accept connections from the destination. These might not be reachable
-      from the client and might be located on networks to which the client
-      has no connection.
-    </t>
-
-    <t>
-      If the client wishes to perform an inter-server copy, the client
-      MUST send a COPY_NOTIFY to the source server. Therefore, the source
-      server MUST support COPY_NOTIFY.
-    </t>
-
-    <t>
-      For a copy only involving one server (the source and destination
-      are on the same server), this operation is unnecessary.
-    </t>
-  </section>
-</section>
diff --git a/nfsv42_middle_op_mandlist.xml b/nfsv42_middle_op_mandlist.xml
index b102d0f..f0d5cba 100644
--- a/nfsv42_middle_op_mandlist.xml
+++ b/nfsv42_middle_op_mandlist.xml
@@ -123,8 +123,6 @@
       <c>COMMIT               </c> <c> REQ </c> <c>              </c>
       <c>COPY                 </c> <c> OPT </c> <c> COPY (REQ)   </c>
       <c>OFFLOAD_ABORT        </c> <c> OPT </c> <c> COPY (REQ)   </c>
-      <c>COPY_NOTIFY          </c> <c> OPT </c> <c> COPY (REQ)   </c>
-      <c>OFFLOAD_REVOKE       </c> <c> OPT </c> <c> COPY (REQ)   </c>
       <c>OFFLOAD_STATUS       </c> <c> OPT </c> <c> COPY (REQ)   </c>
       <c>CREATE               </c> <c> REQ </c> <c>              </c>
       <c>CREATE_SESSION       </c> <c> REQ </c> <c>              </c>
diff --git a/nfsv42_middle_op_offload_abort.xml b/nfsv42_middle_op_offload_abort.xml
index 1f017d4..f3cba4b 100644
--- a/nfsv42_middle_op_offload_abort.xml
+++ b/nfsv42_middle_op_offload_abort.xml
@@ -12,10 +12,10 @@
 
   <section toc='exclude' title="DESCRIPTION">
     <t>
-      OFFLOAD_ABORT is used for both intra- and inter-server
+      OFFLOAD_ABORT is used for
       asynchronous copies. The OFFLOAD_ABORT operation allows the client
       to cancel a server-side copy operation that it initiated. This operation
-      is sent in a COMPOUND request from the client to the destination
+      is sent in a COMPOUND request from the client to the
       server. This operation may be used to cancel a copy when
       the application that requested the copy exits before the operation
       is completed or for some other reason.
diff --git a/nfsv42_middle_op_offload_revoke.xml b/nfsv42_middle_op_offload_revoke.xml
deleted file mode 100644
index 9c66c89..0000000
--- a/nfsv42_middle_op_offload_revoke.xml
+++ /dev/null
@@ -1,51 +0,0 @@
-<!-- Copyright (C) The IETF Trust (2011-2012) -->
-<!-- Copyright (C) The Internet Society (2011-2012) -->
-
-<section anchor="op:offload_revoke" title="Operation 62: OFFLOAD_REVOKE - Revoke a destination server's copy privileges">
-  <section toc='exclude' title="ARGUMENT">
-    <?rfc include='autogen/offload_revoke_args.xml'?>
-  </section>
-
-  <section toc='exclude' title="RESULT">
-    <?rfc include='autogen/offload_revoke_res.xml'?>
-  </section>
-
-  <section toc='exclude' title="DESCRIPTION">
-    <t>
-      This operation is used for an inter-server copy. A client sends this
-      operation in a COMPOUND request to the source server to revoke the
-      authorization of a destination server identified by
-      ora_destination_server from
-      reading the file specified by CURRENT_FH on behalf of given user. If the
-      ora_destination_server has already begun copying the file, a successful
-      return from this operation indicates that further access will be prevented.
-    </t>
-
-    <t>
-      The ora_destination_server MUST be specified using the
-      netloc4 network location format. The server is not required
-      to resolve the ora_destination_server address before completing
-      this operation.
-    </t>
-
-    <t>
-      The client uses OFFLOAD_ABORT to inform the destination to stop the
-      active transfer and OFFLOAD_REVOKE to inform the source to not allow
-      any more copy requests from the destination.
-      The OFFLOAD_REVOKE operation is also useful in situations in which the
-      source server granted a very long or infinite lease on the
-      destination server's ability to read the source file and
-      all copy operations on the source file have been completed.
-    </t>
-
-    <t>
-      For a copy only involving one server (the source and destination
-      are on the same server), this operation is unnecessary.
-    </t>
-
-    <t>
-      If the server supports COPY_NOTIFY, the server is
-      REQUIRED to support the OFFLOAD_REVOKE operation.
-    </t>
-  </section>
-</section>
diff --git a/nfsv42_middle_op_offload_status.xml b/nfsv42_middle_op_offload_status.xml
index 7f5d066..5d55e59 100644
--- a/nfsv42_middle_op_offload_status.xml
+++ b/nfsv42_middle_op_offload_status.xml
@@ -12,9 +12,9 @@
 
   <section toc='exclude' title="DESCRIPTION">
     <t>
-      OFFLOAD_STATUS is used for both intra- and inter-server asynchronous
+      OFFLOAD_STATUS is used for asynchronous
       copies. The OFFLOAD_STATUS operation allows the client to poll the
-      destination server to determine the status of an asynchronous copy operation.
+      server to determine the status of an asynchronous copy operation.
     </t>
 
     <t>
-- 
1.7.9.5

---

From: "McDonald, Alex" <[email-masked]>
Date: Fri, 2 Aug 2013 17:55:06 +0000

I have comments, but to make them via a blackberry would be a Sisyphean task. Can I respond later next week?

Sent on my bb

----- Original Message -----
From: J. Bruce Fields [mailto:[email-masked]]
Sent: Friday, August 02, 2013 08:17 AM
To: Spencer Shepler <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [nfsv4] Soliciting next steps for RPCSEC_GSSv3

...

Any opinions?

--b.
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "[email-masked]'" <[email-masked]>
Date: Fri, 2 Aug 2013 14:31:51 -0400

> I have comments, but to make them via a blackberry would be a Sisyphean task. Can I respond later next week?

Of course, comments welcomed any time.--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 2 Aug 2013 16:26:22 -0400

> -          should fall back to the normal copy semantics.

OFFLOAD_DENIED I'm a little unclear about, by the way: I'm assuming it's
only useful in the server-to-server case, but I'm not sure I really
understand what it's for.

--b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Sun, 4 Aug 2013 17:08:04 +0000

> but not of any serious effort for inter-server copies.

I know our prototype supported both.

Also how trivial would it be to take the recent changes by Bryan and have a working
inter-vserver case?

Is it onerous for us to do RPCSEC_GSSv3? Sure.

Does that mean we should add another 3-4 year gap in order to get the full
set of features for SSC? No.

And as we discussed in Berlin, we do not think that removing the security
language will fly during the document review.

---

From: "Haynes, Tom" <[email-masked]>
Date: Sun, 4 Aug 2013 17:09:44 +0000

Is there anything available that describes how the Linux labeled NFS code does the labels?

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 5 Aug 2013 12:16:47 -0400

> I know our prototype supported both.

Googling around....  This one?:

	http://nfsv4bat.org/Documents/ConnectAThon/2011/ssc.pdf

Do we know what kind of progress this (or any other server-to-server
effort) is making?  Are they finishing RPCSEC_GSSv3?

> Also how trivial would it be to take the recent changes by Bryan and
> have a working inter-vserver case?

Bryan's issue needs to be resolved regardless, as it affects both inter-
and intra- server cases--Bryan's issue and this one are independent.

> set of features for SSC? No.

There will be some minimal 4.2 labeled NFS code in Linux kernel version
3.11, probably to be released in about a month.  Intra-server copy code
looks like it could be in that position soon too?  (Next year??)

So we have running code at this point.  We'd like to tell people it's
safe to use.

I definitely don't want to stand in the way of anyone *else's* running
code; but I'd like evidence that it's there.

Possible responses:

	- Come on Bruce, haven't you been paying attention?
	  Server-to-server code is here, somebody's picked up
	  RPCSEC_GSSv3, it's all on track.

	- You're being too paranoid, just go ahead!  The bits of 4.2 you
	  care about aren't going to get rewritten at this point--in the
	  unlikely event further review finds problems then we'll
	  accommodate already-deployed code somehow.

	- We tried, but we just can't release minor versions more
	  frequently than one every 7 years.  And it's terrifically
	  antisocial of you to suggest putting code into production and
	  tying the hands of reviewers before the rfc's are out.  So we
	  have to guess which things are going to be ready in 2017, and
	  RPCSEC_GSSv3 still looks like it might be one of those.  Sorry
	  you have to wait a few more years, but it's a tradeoff for not
	  making somebody else wait ten.

I'd be happy with the first reponse, reluctantly satisfied with the
second.

If it's the third, I need to go have a little cry before I come back.

> language will fly during the document review. 

Which security language are you referring to exactly?

I agree that we probably need RPCSEC_GSSv3 (or something else new) to
implement server-to-server copy (option "a" from my previous message).

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 5 Aug 2013 12:17:35 -0400

> Is there anything available that describes how the Linux labeled NFS

What exactly would you like to know?

--b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Mon, 5 Aug 2013 18:23:34 +0000

> What exactly would you like to know?

I was just wondering how it bypasses getting the subject label from RPCSEC_GSSv3?

I.e., what does it look at to infer whether the subject gets access to the object?

---

From: "Myklebust, Trond" <[email-masked]>
Date: Mon, 5 Aug 2013 18:31:12 +0000

> I.e., what does it look at to infer whether the subject gets access to the object?

That's only needed for the server-enforced model, which is not
implemented and is not necessary for the Linux use case.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: "Haynes, Tom" <[email-masked]>
Date: Mon, 5 Aug 2013 18:33:37 +0000

> effort) is making?  Are they finishing RPCSEC_GSSv3?

Inter-server copy does not depend on RPCSEC_GSSv3.

The specification of inter-server copy does depend on RPCSEC_GSSv3,
but the implementation does not.

Just like the specification of NFSv4.0 depends on Kerberos, but there
are implementations out there which work in AUTH_SYS.

> and intra- server cases--Bryan's issue and this one are independent.

My point is that if you a copy engine based solely on NFSv4.2 for intra-server,
then why should you have to wait another 7 years to get the inter-server
version?

> 	  RPCSEC_GSSv3, it's all on track.

I do not see that anyone has picked up RPCSEC_GSSv3.

> 	  accommodate already-deployed code somehow.

I think the implementation experience being gathered between the
current Labeled NFS work and SSC in Linux is either validating
the current draft or causing changes. But once those play out,
the standard will not change.

> If it's the third, I need to go have a little cry before I come back.

I think the fact that there was still NFSv4.1 implementation going on
when we started NFSv4.2 was a major hurdle.

I am very happy to see the Labeled NFS and SSC work in Linux
right now. It should have been happening the first year of the
work in NFSv4.2. :->

> Which security language are you referring to exactly?

As we introduce SSC and Labeled NFS, we need to specify strong
security language. That is currently RPCSEC_GSSv3.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 5 Aug 2013 16:28:10 -0400

On Mon, Aug 05, 2013 at 06:33:37PM +0000, Haynes, Tom wrote:
...
> I do not see that anyone has picked up RPCSEC_GSSv3.

So do you have in mind some criteria for when we *would* give up on
features that aren't making progress?

...
> security language. That is currently RPCSEC_GSSv3.

Only inter-server copy and subject labels need RPCSEC_GSSv3.

--b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Mon, 5 Aug 2013 21:10:15 +0000

If we go down the route of neutering SSC and LNFS, can we do so such that
we can get to the full capability without spinning NFSv4.3?

I.e., lets say we finish RPCSEC_GSSv3 a month after NFSv4.2 gets
canonized. Can we do inter-server copy and subject labels without
any protocol changes in NFSv4.2?

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 5 Aug 2013 17:54:04 -0400

> any protocol changes in NFSv4.2?

In the labeled NFS case, I think so.  There are no xdr changes, at
least.

In the server-side copy stuff there's a lot of protocol whose only
justification appears to be the inter-server case:

	- The COPY_NOTIFY and OFFLOAD_REVOKE ops.
	- The ca_source_server field of COPY.
	- The PARTNER_NOTSUPP, PARTNER_NO_AUTH, and OFFLOAD_DENIED
	  errors.

--b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Mon, 5 Aug 2013 23:47:20 +0000

> 	- The COPY_NOTIFY and OFFLOAD_REVOKE ops.

Even intra-server, the copy can be asynchronous, no?

And OFFLOAD_REVOKE would be used by WRITE_PLUS.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 5 Aug 2013 21:44:19 -0400

> And OFFLOAD_REVOKE would be used by WRITE_PLUS.

You're thinking of OFFLOAD_ABORT, which is used for asynchronous copy
and WRITE_PLUS.

OFFLOAD_REVOKE appears to be just a server-to-server copy thing.

--b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 7 Aug 2013 15:28:40 +0000

For each feature, we have the following states:

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

Based on the discussions we had in Berlin in Dave's talk, I'd like us to add a new
state: EXPERIMENTAL.

And then we add inter-server copy as EXPERIMENTAL.

What does this mean?

We can remove RPCSEC_GSSv3 as normative text in NFSv4.2. Actually, we remove
it altogether.

We then describe LNFS in the client case and the weak authorization for inter-server
copy.

If someone decides to finish RPCSEC_GSSv3 a month after NFSv4.2 becomes
a standard, then we can write new documents for doing LNFS with subject labels
and inter-server copy.

---

From: "Noveck, David" <[email-masked]>
Date: Wed, 7 Aug 2013 12:02:16 -0400

I think this is an idea worth pursuing.

> If someone decides to finish RPCSEC_GSSv3 a month after NFSv4.2 becomes a standard

I think you mean a proposed standard.

> then we can write new documents for doing LNFS with subject labels and inter-server copy.

Someone could.  The problem is getting volunteers.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 07, 2013 11:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

We can remove RPCSEC_GSSv3 as normative text in NFSv4.2. Actually, we remove it altogether.

We then describe LNFS in the client case and the weak authorization for inter-server copy.

If someone decides to finish RPCSEC_GSSv3 a month after NFSv4.2 becomes a standard, then we can write new documents for doing LNFS with subject labels and inter-server copy.

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 7 Aug 2013 16:36:51 +0000

I am not in favor of this.  It seems disingenuous to call is EXPERIMENTAL given that it is a result of lack of interest/effort to finish the work and doesn't really remove security concerns.  If an implementation provides support for the feature then there is no method of doing so securely, right?

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 7, 2013 8:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

---

From: "Noveck, David" <[email-masked]>
Date: Wed, 7 Aug 2013 13:00:22 -0400

> It seems disingenuous.

I think you mean it seems misleading.  It doesn't seem to me that Tom intended to deceive anyone.

> If an implementation provides support for the feature then there is no method of doing so securely, right?

There is no standard method of doing it securely, until we have RPCSEC_GSSv3.

However, two mutually trusting servers could do this securely.  The text would have to make it clear that the destination is only allowed to implement this if it can do so securely with the specified source server.  

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Spencer Shepler
Sent: Wednesday, August 07, 2013 12:37 PM
To: Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 7, 2013 8:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 7 Aug 2013 17:05:11 +0000

When it comes time for the security review, the I-D will just say, "the server has do it securely"?  Not sure I understand how that will pass a review but I am judging before the details are worked out.

Ultimately, the experimental label doesn't signify "incomplete" for me -- it says that the definition is complete but the usefulness or other feature is not well known and thus is marked as experimental.

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Spencer Shepler
Sent: Wednesday, August 07, 2013 12:37 PM
To: Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 7, 2013 8:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 7 Aug 2013 17:35:45 +0000

What you describe is fine for experimental and what I am pointing out is that in the case of server-to-server copy, by inspection, it can be found to be incomplete because it is lacking in the appropriate security definitions.  If RPCSEC_GSSv3 isn't needed to make the functionality secure, then it can be removed.  But if it is needed, then it needs to be defined.  Sure, through implementation, other issues may be found but that is not what I am seeing here.

I would like to be proven wrong about my assumptions of security in this case and that discussion is worthwhile.

Spencer

-----Original Message-----
From: Noveck, David [mailto:[email-masked]] 
Sent: Wednesday, August 7, 2013 10:28 AM
To: Spencer Shepler; Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: RE: [nfsv4] [PATCH 2/2] Remove inter-server copy language

My original concept for  "experimental" was that it should apply to ops that have a complete definition but no implementation.  An  example might be ops associated with directory delegation.  The situation is that it seems complete but until you have implementations you don't know if it truly is complete.

As I understand Tom's proposal, he wants to apply it to a feature which seems completely specified and for which there may be a limited set of special-purpose implementations.  I think this is justified by the fact that, until we have general-purpose implementations (for mutually suspicious servers), we don't know that the definition is complete.

Just to top off the confusion, I had thought of the "experimental" designation applying to ops, and Tom is proposing that it apply to only some cases of COPY.  Maybe this is a label we need to apply to *features* (in which case, we would have to be less hand-wavy about what a feature is).

-----Original Message-----
From: Spencer Shepler [mailto:[email-masked]] 
Sent: Wednesday, August 07, 2013 1:05 PM
To: Noveck, David; Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: RE: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Spencer Shepler
Sent: Wednesday, August 07, 2013 12:37 PM
To: Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 7, 2013 8:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

---

From: "Noveck, David" <[email-masked]>
Date: Wed, 7 Aug 2013 13:27:45 -0400

-----Original Message-----
From: Spencer Shepler [mailto:[email-masked]] 
Sent: Wednesday, August 07, 2013 1:05 PM
To: Noveck, David; Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: RE: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Spencer Shepler
Sent: Wednesday, August 07, 2013 12:37 PM
To: Haynes, Tom; J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

Spencer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Haynes, Tom
Sent: Wednesday, August 7, 2013 8:29 AM
To: J. Bruce Fields
Cc: [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

OPTIONAL
RECOMMENDED
REQUIRED
MANDATORY TO NOT
OBSOLESCENT

What does this mean?

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 7 Aug 2013 17:51:35 +0000

On Aug 7, 2013, at 10:35 AM, Spencer Shepler <[email-masked]>
 wrote:

> 
> What you describe is fine for experimental and what I am pointing out is that in the case of server-to-server copy, by inspection, it can be found to be incomplete because it is lacking in the appropriate security definitions.

The EXPERIMENTAL aspect is that we need to define the security.

In 3530 and 5661, we've defined security every implementer has voted with their feet by
not implementing.

I think RPCSEC_GSSv3 will ultimately be different, but I don't want to hold
everything up waiting to be proved right.

Our minorversioning rules handcuff us as to when we can make protocol changes
and our energy limits us as to how often we are willing to push out minorversions.

I'd like us to try with we know this is not secure, but it will not be marked as part
of the standard until the security is provided.

We can move to a model where we provide protocol changes when we ship
a minorversion and functionality on top of the protocol when it is ready.

>  If RPCSEC_GSSv3 isn't needed to make the functionality secure, then it can be removed.  But if it is needed, then it needs to be defined.  Sure, through implementation, other issues may be found but that is not what I am seeing here.
>

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 7 Aug 2013 18:01:17 +0000

Tom,

What do you propose be written in the Security Considerations section?  Something like "the security mechanism will be defined when there is enough implementations to warrant the work to define one"  ?

That is how I interpreted what you wrote below and I can not, in good conscience, shepherd the document forward in such a state.

How do we explain to others that it is okay to leave this part of the protocol "open" while we figure out if anyone is interested enough to define the security mechanisms that are needed?

And that is a serious question, I am not opposed to treating the work as experimental but we need to meet the general requirements of protocol work and security requirements are on the top of the list.

Spencer

-----Original Message-----
From: Haynes, Tom [mailto:[email-masked]] 
Sent: Wednesday, August 7, 2013 10:52 AM
To: Spencer Shepler
Cc: David Noveck; J. Bruce Fields; [email-masked] list
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

In 3530 and 5661, we've defined security every implementer has voted with their feet by not implementing.

I think RPCSEC_GSSv3 will ultimately be different, but I don't want to hold everything up waiting to be proved right.

Our minorversioning rules handcuff us as to when we can make protocol changes and our energy limits us as to how often we are willing to push out minorversions.

I'd like us to try with we know this is not secure, but it will not be marked as part of the standard until the security is provided.

We can move to a model where we provide protocol changes when we ship a minorversion and functionality on top of the protocol when it is ready.

---

From: "Haynes, Tom" <[email-masked]>
Date: Thu, 8 Aug 2013 00:09:32 +0000

> What do you propose be written in the Security Considerations section?  Something like "the security mechanism will be defined when there is enough implementations to warrant the work to define one…

That the plumbing for inter-server copy exists, but that it is not blessed until a secure mechanism exists.

> How do we explain to others that it is okay to leave this part of the protocol "open" while we figure out if anyone is interested enough to define the security mechanisms that are needed?

What is the difference from we define a security mechanism which no one
implements and we never define a security mechanism?

Or we define, implement, and customers do not enable?

> And that is a serious question, I am not opposed to treating the work as experimental but we need to meet the general requirements of protocol work and security requirements are on the top of the l…

Just so we are on the same page, where are those general requirements documented?

I'm not interested on an end-run on security - I am interested on an end-run to
our minor-versioning rules.

---

From: "McDonald, Alex" <[email-masked]>
Date: Thu, 8 Aug 2013 15:53:16 +0000

I promised to comment on this thread.

Tom got there before me; the use cases for inter-server are more compelling than the use cases for intra-server. 

I'm unclear why RPCSEC_GSSv3 as opposed to the current agreed RPCSEC_GSS would be required; can someone clarify for me please (or point me to any relevant messgaes/docs)? I can see some indications of why in the thread below, but it's not clear to me as to the source of some of these statements.

Additionally, what are the implications of SSC in FedFS? 

Alex McDonald
Standards & Industry Associations Group
CTO Office
NetApp
+44 7795 046686 Mobile Phone
[email-masked]
netapp.com
twitter: @alextangent

---

From: "[email-masked]'" <[email-masked]>
Date: Thu, 8 Aug 2013 12:17:35 -0400

On Thu, Aug 08, 2013 at 03:53:16PM +0000, McDonald, Alex wrote:
> Tom got there before me; the use cases for inter-server are more
> compelling than the use cases for intra-server. 

Could you explain?

I didn't see Tom say anything about use cases.

> I'm unclear why RPCSEC_GSSv3 as opposed to the current agreed RPCSEC_GSS would be required; can someone clarify for me please (or point me to any relevant messgaes/docs)? I can see some indications…

http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-19#section-3.4.1.1

	"The specification MUST provide guidance for using NFSv4.x as a
	copy protocol.
	....
	The specification MUST NOT mandate a trust relationship between
	the source and destination server."

If we accept those, then we need some way the destination server can
authenticate to the source server, and that's what RPCSEC_GSSv3 (see
details in next section) provides.

> Additionally, what are the implications of SSC in FedFS? 

Can you make that question any more specific?

--b.

---

From: "McDonald, Alex" <[email-masked]>
Date: Thu, 8 Aug 2013 17:39:29 +0000

Inline

Alex McDonald

> Sent: 08 August 2013 17:18
> To: McDonald, Alex

He didn't explicitly; but he addressed it in another reply with his observation that "I know our prototype supported both."  The prototype does so as inter-server has  more compelling use cases (for me at least).

> 
> > I'm unclear why RPCSEC_GSSv3 as opposed to the current agreed
> RPCSEC_GSS would be required; can someone clarify for me please (or point
> me to any relevant messgaes/docs)? I can see some indications of why in the
> thread below, but it's not clear to me as to the source of some of these
> statements.
> 

Thanks; I'll read further.

> Can you make that question any more specific?

I will, but since it relies on better understanding RPCSEC_GSSv3 I'll defer my question.

> 
> --b.
> 
> >
> > Alex McDonald
> > Standards & Industry Associations Group CTO Office NetApp
> > > I have comments, but to make them via a blackberry would be a
> Sisyphean task. Can I respond later next week?
> >
> > > > My copy-paste from another email thread is mildly inappropriate but I
> thought it best to continue this discussion under a separate topic to ensure
> that everyone has had a chance to comment (see thread content in question
> below).
> > > >
> > > > So the first order of business is that I am soliciting an editor
> > > > for http://tools.ietf.org/id/draft-williams-rpcsecgssv3-02.txt
> > > >
> > > > Second order of business is to decide whether to keep the dependency
> and if not, what can or should be done about it.
> > > >
> > > So I took a stab at removing references to inter-server-copy,
> > > subject labels, and RPCSEC_GSSv3 from the spec.  This eliminates a lot of
> text:
> > >
> > > Detailed changes follow.  Done fairly quickly, so I may have missed
> > > a few things.
> > >
> > > Any opinions?
> > >
> > > --b.

---

From: "Haynes, Tom" <[email-masked]>
Date: Thu, 8 Aug 2013 17:45:19 +0000

> details in next section) provides.

Right and the MUST NOT is the only concern I had with Dave's proposal yesterday:

   The security mechanism to be used for inter-server copies when there is not a trust relationship between 
   source and destination servers is left to be defined in the future.  Until the time that such a  security mechanism
   is defined and determined to be adequate, the destination server MUST  not honor requests to copy files from 
   a server with which it does not have an established trust relationship, and return the error 
   NFS4ERR_SOURCETRUST.  Similarly, if that mechanism, while defined, is not used by the client, the same applies.

We could change the text you quote, which would reflect reality. :->

I.e., a closed data center, hidden from the 'i' in IETF has established
a trust relationship.

---

From: "[email-masked]'" <[email-masked]>
Date: Thu, 8 Aug 2013 13:57:12 -0400

On Thu, Aug 08, 2013 at 01:35:15PM -0400, Noveck, David wrote:
> BTW, normally "MUST" and "MUST NOT" are typically used for the spec to lay down the law to 
> prospective implementations.    In the section referenced above, it is used for one section of the 
> spec to lay down the law to the rest of the spec.   RFC2119 is vague enough that you can't quite say
> this is wrong.  Still, I find it kind of strange.

I agree, that's weird.  They should just be garden-variety "must"'s
without the shouting.

--b.

> 
> For more details on use-of/need-for RPCSEC_GSSv3, see section 3.4.1.2.  
> 
> 
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]'
> Sent: Thursday, August 08, 2013 12:18 PM
> To: McDonald, Alex
> Cc: [email-masked]'
> If we accept those, then we need some way the destination server can authenticate to the source server, and that's what RPCSEC_GSSv3 (see details in next section) provides.
> 
> > > subject labels, and RPCSEC_GSSv3 from the spec.  This eliminates a lot of text:
> > >

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 8 Aug 2013 14:09:41 -0400

>    The security mechanism to be used for inter-server copies when
>    there is not a trust relationship between source and destination
>    servers is left to be defined in the future.  Until the time that
>    such a  security mechanism is defined and determined to be
>    adequate, the destination server MUST  not honor requests to copy
>    files from a server with which it does not have an established
>    trust relationship, and return the error NFS4ERR_SOURCETRUST.
>    Similarly, if that mechanism, while defined, is not used by the
>    client, the same applies.

OK, then we're just deciding to ditch the "must not mandate a trust
relationship" requirement above.  Doesn't cause me any problem, I guess,
but it seems like a strange reversal.

The full paragraph is:

	The specification MUST NOT mandate a trust relationship between
	the source and destination server.  The NFSv4 security model
	requires mutual authentication between a principal on an NFS
	client and a principal on an NFS server.  This model MUST
	continue with the introduction of COPY.

So we're disagreeing with that last sentence?

Also I'm now confused by your earlier statement:

	And as we discussed in Berlin, we do not think that removing the
	security language will fly during the document review. 

I thought the above was part of the security language you were referring
to.

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 8 Aug 2013 18:32:24 +0000

My comments are below.

> Sent: Wednesday, August 7, 2013 5:10 PM
> To: Spencer Shepler
> > What do you propose be written in the Security Considerations section?
> Something like "the security mechanism will be defined when there is
> enough implementations to warrant the work to define one"  ?
> 
> 
[Spencer] The definition of "blessed" is the problem.  When an RFC is published is when it is blessed.  It is implementable and deployable and thus security must be defined.  "experimental" is frown upon within the IETF and it is a RFC designation that is going away (if it has not already) (or at least this is my impression given the discussion that I have followed in other contexts).

> 
> 
> >
> > That is how I interpreted what you wrote below and I can not, in good
> conscience, shepherd the document forward in such a state.
> >
> > How do we explain to others that it is okay to leave this part of the protocol
> "open" while we figure out if anyone is interested enough to define the
> security mechanisms that are needed?
> >
> 
> 
[Spencer] If that is argument, then it seems that the definition is incorrect; it needs to be implementable.  See the following for a general discussion: http://www.apps.ietf.org/rfc/rfc3365.html

> 
> 
> > And that is a serious question, I am not opposed to treating the work as
> experimental but we need to meet the general requirements of protocol
> work and security requirements are on the top of the list.
> >
> 
[Spencer] 
http://www.apps.ietf.org/rfc/rfc3365.html
http://www.apps.ietf.org/rfc/rfc3552.html

For the best practices documents.

Spencer
> 
> I'm not interested on an end-run on security - I am interested on an end-run
> to our minor-versioning rules.
> 
> 
> > Spencer
> >
> >> What you describe is fine for experimental and what I am pointing out is
> that in the case of server-to-server copy, by inspection, it can be found to be
> incomplete because it is lacking in the appropriate security definitions.
> >
> >
> > Our minorversioning rules handcuff us as to when we can make protocol
> changes and our energy limits us as to how often we are willing to push out
> minorversions.
> >
> > I'd like us to try with we know this is not secure, but it will not be marked as
> part of the standard until the security is provided.
> >
> > We can move to a model where we provide protocol changes when we
> ship a minorversion and functionality on top of the protocol when it is ready.
> >
> >
> >
> >> If RPCSEC_GSSv3 isn't needed to make the functionality secure, then it
> can be removed.  But if it is needed, then it needs to be defined.  Sure,
> through implementation, other issues may be found but that is not what I am
> seeing here.
> >>
> >> My original concept for  "experimental" was that it should apply to ops
> that have a complete definition but no implementation.  An  example might
> be ops associated with directory delegation.  The situation is that it seems
> complete but until you have implementations you don't know if it truly is
> complete.
> >>
> >> As I understand Tom's proposal, he wants to apply it to a feature which
> seems completely specified and for which there may be a limited set of
> special-purpose implementations.  I think this is justified by the fact that,
> until we have general-purpose implementations (for mutually suspicious
> servers), we don't know that the definition is complete.
> >>
> >> Just to top off the confusion, I had thought of the "experimental"
> designation applying to ops, and Tom is proposing that it apply to only some
> cases of COPY.  Maybe this is a label we need to apply to *features* (in
> which case, we would have to be less hand-wavy about what a feature is).
> >>
> >> When it comes time for the security review, the I-D will just say, "the
> server has do it securely"?  Not sure I understand how that will pass a review
> but I am judging before the details are worked out.
> >>
> >> Ultimately, the experimental label doesn't signify "incomplete" for me -- it
> says that the definition is complete but the usefulness or other feature is not
> well known and thus is marked as experimental.
> >>
> >> Spencer
> >>
> >> However, two mutually trusting servers could do this securely.  The text
> would have to make it clear that the destination is only allowed to implement
> this if it can do so securely with the specified source server.
> >>
> >> I am not in favor of this.  It seems disingenuous to call is EXPERIMENTAL
> given that it is a result of lack of interest/effort to finish the work and doesn't
> really remove security concerns.  If an implementation provides support for
> the feature then there is no method of doing so securely, right?
> >>
> >> Spencer
> >>
> >> If someone decides to finish RPCSEC_GSSv3 a month after NFSv4.2
> becomes a standard, then we can write new documents for doing LNFS with
> subject labels and inter-server copy.
> >>
> >>

---

From: "Noveck, David" <[email-masked]>
Date: Thu, 8 Aug 2013 13:35:15 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]'
Sent: Thursday, August 08, 2013 12:18 PM
To: McDonald, Alex
Cc: [email-masked]'
Subject: Re: [nfsv4] Soliciting next steps for RPCSEC_GSSv3

Could you explain?

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 8 Aug 2013 15:15:00 -0400

> Detailed changes follow.  Done fairly quickly, so I may have missed a

For what it's worth I fixed a few mistakes in those patches; for anyone
that wants to browse through it:

	http://git.linux-nfs.org/?p=bfields/NFSv42.git;a=summary

Also noticed a few more probably less controversial things along the
way.  Tom, if you think they look OK you can pull them from

	git://linux-nfs.org/~bfields/NFSv42.git minor-edits

Also appended below.

--b.

commit 1ab61fa6bfbd6765217f0689702d8caae51fa7af
Author: J. Bruce Fields <[email-masked]>
Date:   Wed Aug 7 17:53:06 2013 -0400

    minor typo

diff --git a/nfsv42_middle_lnfs.xml b/nfsv42_middle_lnfs.xml
index eb29422..9e95835 100644
--- a/nfsv42_middle_lnfs.xml
+++ b/nfsv42_middle_lnfs.xml
@@ -384,7 +384,7 @@
           but does not enforce policies. Such a server will store and retrieve
           all object labels presented by clients, utilize the methods
           described in <xref target='ss:macss:lnfs:changes' /> to allow
-          the clients to detect changing labels,, but will not restrict access
+          the clients to detect changing labels, but will not restrict access
           via the subject label. Instead, it will expect the clients to enforce
           all such access locally.
         </t>

commit 05522a85e8e49ade42ceddf7453465cf52ae7f35
Author: J. Bruce Fields <[email-masked]>
Date:   Wed Aug 7 17:13:42 2013 -0400

    Remove redundant pnfs-considerations introduction
    
    This first sentence doesn't really tell us anything the section title
    doesn't already tell us.

diff --git a/nfsv42_middle_lnfs.xml b/nfsv42_middle_lnfs.xml
index 2ebe96e..eb29422 100644
--- a/nfsv42_middle_lnfs.xml
+++ b/nfsv42_middle_lnfs.xml
@@ -254,12 +254,6 @@
   </section>
 
@@ -269,7 +263,6 @@

commit 1261561c1ec980e4dec83d1e7a3b6b6ea639b155
Author: J. Bruce Fields <[email-masked]>
Date:   Wed Aug 7 17:08:01 2013 -0400

    Servers don't always get subject labels
    
    Maybe we could change "As the server is always" to "if the server is
    always"?
    
    Also "communicate... to the client" looks like a relic of the time when
    we had a dedicated label-change callback.
    
    Let's just ditch this paragraph--it doesn't look particularly helpful to
    me anyway.

diff --git a/nfsv42_middle_lnfs.xml b/nfsv42_middle_lnfs.xml
index f667021..2ebe96e 100644
--- a/nfsv42_middle_lnfs.xml
+++ b/nfsv42_middle_lnfs.xml
@@ -231,15 +231,6 @@
       </t>
 
       <t>

---

From: "Mike Eisler" <[email-masked]>
Date: Thu, 8 Aug 2013 13:41:20 -0700

> but not of any serious effort for inter-server copies.

That has never been an impediment to moving an I-D to proposed standard
before.

> RPCSEC_GSSv3

Cutting inter-server copy because we don't have RPCSEC_GSSv3 strikes me as
indefinitely postponing inter-server copy.

As one of the original authors of the COPY stuff, we decided to specify a
dependency on RPCSEC_GSSv3, because it looked like, at the time, that it
was happening, and it was a most elegant way to satisfy the security
requirement.

But RPCSEC_GSSv3 isn't happening.

So, I suggest removing references to the copy-related RPCSEC_GSSv3
privileges from the NFSv4.2 spec. The way to make inter-vserver copy
secure is implied by:

3.4.1.3. Inter-Server Copy via ONC RPC but without RPCSEC_GSSv3

"For example, suppose the network topology is as shown in Figure 3.
   If the source filehandle is 0x12345, the source server may respond to
   a COPY_NOTIFY for destination 203.0.113.56 with the URLs:

      nfs://203.0.113.18//_COPY/FvhH1OKbu8VrxvV1erdjvR7N/203.0.113.56/
      _FH/0x12345

      nfs://192.0.2.18//_COPY/FvhH1OKbu8VrxvV1erdjvR7N/203.0.113.56/_FH/
      0x12345

   The name component after _COPY is 24 characters of base 64, more than
   enough to encode a 128 bit random number."

So, servers that want to enforce secure inter-server copy simply reject
COPY_NOTIFY requests that do not use RPCSEC_GSS(v1 or v2) privacy. When
privacy is used, the URL in the COPY_NOTIFY reply is encrypted.

The client SHOULD then send COPY to the destination with RPCSEC_GSS v1 or
v2 privacy so that the shared secret is encrypted.

---

From: "Mike Eisler" <[email-masked]>
Date: Thu, 8 Aug 2013 14:08:15 -0700

Not as elegant.

On Thu, August 8, 2013 1:59 pm, Noveck, David wrote:
>> it was a most elegant way to satisfy the security requirement.
>
> So now, inelegance is to be preferred to non-existence.
>
>
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Mike Eisler
> Sent: Thursday, August 08, 2013 4:41 PM
> To: [email-masked]
>> I'm aware of at least one implementation effort for intra-server
>> copies, but not of any serious effort for inter-server copies.
>

-- 
Mike Eisler, Senior Technical Director, NetApp, 719 599 9026,
http://blogs.netapp.com/eislers_nfs_blog/

---

From: "Noveck, David" <[email-masked]>
Date: Thu, 8 Aug 2013 16:59:52 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mike Eisler
Sent: Thursday, August 08, 2013 4:41 PM
To: [email-masked]
Subject: Re: [nfsv4] [PATCH 2/2] Remove inter-server copy language

> copies, but not of any serious effort for inter-server copies.

That has never been an impediment to moving an I-D to proposed standard before.

> Also, this allows us to remove references to the (so far unfinished)

Cutting inter-server copy because we don't have RPCSEC_GSSv3 strikes me as indefinitely postponing inter-server copy.

As one of the original authors of the COPY stuff, we decided to specify a dependency on RPCSEC_GSSv3, because it looked like, at the time, that it was happening, and it was a most elegant way to satisfy the security requirement.

So, I suggest removing references to the copy-related RPCSEC_GSSv3 privileges from the NFSv4.2 spec. The way to make inter-vserver copy secure is implied by:

So, servers that want to enforce secure inter-server copy simply reject COPY_NOTIFY requests that do not use RPCSEC_GSS(v1 or v2) privacy. When privacy is used, the URL in the COPY_NOTIFY reply is encrypted.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 9 Aug 2013 12:02:35 -0400

> v2 privacy so that the shared secret is encrypted.

OK, if that's good enough, great.

A very quick attempt follows.

--b.

commit 9b786365d6432462338c7b5425ba2bac22beae32
Author: J. Bruce Fields <[email-masked]>
Date:   Fri Aug 9 11:39:57 2013 -0400

    Remove dependency of server-to-server copy on RPCSEC_GSSv3
    
    RPCSEC_GSSv3 progress seems stalled, while implementation
    of other features is progressing.
    
    It appears that the random-cookie-in-a-URL approach could provide
    sufficient security on its own, so recommend that instead.

diff --git a/nfsv42_middle_copy.xml b/nfsv42_middle_copy.xml
index 13e9174..1a5e958 100644
--- a/nfsv42_middle_copy.xml
+++ b/nfsv42_middle_copy.xml
@@ -577,14 +577,12 @@
 
     <t>
       NFSv4 clients and servers supporting the inter-server copy
-      operations described in this chapter are REQUIRED to implement
-      <xref target="rpcsecgssv3" />, including the RPCSEC_GSSv3
-      privileges copy_from_auth and copy_to_auth. If the server-to-server
-      copy protocol is ONC RPC based, the servers are also REQUIRED to
-      implement the RPCSEC_GSSv3 privilege copy_confirm_auth. These
-      requirements to implement are not requirements to use. NFSv4 clients
-      and servers are RECOMMENDED to use <xref target="rpcsecgssv3" />
-      to secure server-side copy operations.
+      operations described in this chapter are REQUIRED to implement the
+      mechanism described in <xref target="server-to-server-with-rpc">,
+      and to support rejecting COPY_NOTIFY requests that do not use
+      RPCSEC_GSS with privacy.  This requirement to implement is not
+      a requirement to use; for example, a server may depending on
+      configuration also allow COPY_NOTIFY requests that use only AUTH_SYS.
     </t>
 
     <section title="Inter-Server Copy Security" anchor="server-to-server-sec">
@@ -596,7 +594,7 @@
         <t>
           <list style="symbols">
             <t>
-              The specification MUST NOT mandate an inter-server copy
+              The specification must not mandate an inter-server copy
               protocol. There are many ways to copy data. Some will be
               more optimal than others depending on the identities of
               the source server and destination server. For example
@@ -614,14 +612,14 @@
             </t>
 
             <t>
-              The specification MUST provide guidance for using NFSv4.x as
+              The specification must provide guidance for using NFSv4.x as
               a copy protocol. For those source and destination servers
               willing to use NFSv4.x there are specific security considerations
               that this specification can and does address.
             </t>
 
             <t>
-              The specification MUST NOT mandate pre-configuration between the
+              The specification must not mandate pre-configuration between the
               source and destination server. Requiring that the source and
               destination first have a "copying relationship" increases the
               administrative burden. However the specification MUST NOT
@@ -629,7 +627,7 @@
             </t>
 
             <t>
-              The specification MUST NOT mandate a trust relationship between
+              The specification must not mandate a trust relationship between
               the source and destination server. The NFSv4 security model
               requires mutual authentication between a principal on an NFS
               client and a principal on an NFS server. This model MUST continue
@@ -639,364 +637,11 @@
         </t>
       </section>
 
+      <section title="Inter-Server Copy via ONC RPC"
+       anchor="server-to-server-with-rpc">
+          In the absence of a strong security mechanism designed for the
+	  purpose, the challenge is how the source server
           and destination server identify themselves to each other,
           especially in the presence of multi-homed source and
           destination servers. In a multi-homed environment, the
@@ -1066,6 +711,14 @@
           observes the random number on the wire. Other secure communication
           techniques (e.g., IPsec) are necessary to block these attacks.
         </t>
+
+	<t>
+	  Servers SHOULD reject COPY_NOTIFY requests that do not use
+	  RPCSEC_GSS with privacy, thus ensuring the URL in the
+	  COPY_NOTIFY reply is encrypted.  For the same reason, clients
+	  SHOULD send COPY requests to the destination using RPCSEC_GSS
+	  with privacy.
+	</t>
       </section>
 
       <section title="Inter-Server Copy without ONC RPC and RPCSEC_GSSv3"

---

From: Andy Adamson <[email-masked]>
Date: Fri, 9 Aug 2013 15:29:12 -0400

> http://tools.ietf.org/id/draft-williams-rpcsecgssv3-02.txt

I will give it a go.

I found the git hub for RPCSEC_GSSv3 which I suppose has a list of
issues. Is there any other repository of past reviews?

How much interest is there in retaining the channel binding portion of
RPCSEC_GSSv3?
At first glance, it looks to replicate the RPCSEC_GSSv2 channel
bindings, and I would consider removing it.

-->Andy
.
> Second order of business is to decide whether to keep the dependency and if
> not, what can or should be done about it.
>
>
>
> I am soliciting discussion on those choices (and if there are more).
>
>
>
> Opinions?
>
>
>
> Spencer
>
>
>
>
>
>
>
> On Fri, May 10, 2013 at 08:37:28PM -0400, David Quigley wrote:
>
>> On 05/09/2013 15:56, J. Bruce Fields wrote:
>
>> >On Thu, May 09, 2013 at 07:30:59PM +0000, Spencer Shepler wrote:
>
>> >>
>
>> >>I will, again, take the opportunity to note that there is a
>
>> >>normative reference in NFSv4.2 on this:
>
>> >>
>
>> >>[5]   Haynes, T. and N. Williams, "Remote Procedure Call (RPC)
>
>> >>         Security Version 3", draft-williams-rpcsecgssv3 (work in
>
>> >>         progress), 2011.
>
>> >>
>
>> >>If there is a desire to see NFSv4.2 move ahead to a published RFC,
>
>> >>then either this dependency needs to be removed or it needs to move
>
>> >>forward with NFSv4.2.
>
>> >
>
>> >As far as I know, the rpcsecgssv3 draft isn't ready, and there's no
>
>> >one working on implementation.  If that's accurate, I think we should
>
>> >remove the dependency.
>
>> >
>
>> >I'd be delighted to be corrected on either point.
>
>> >
>
>> >--b.
>
>> >
>
>>
>
>> That no one is working on it is only partly true. The guys in
>
>> singapore who were helping me with rebasing the LNFS patches for Linux
>
>> before Steve Dickson took over were tasked with figuring out the
>
>> RPCSEC_GSSv3 patches that I gave them. If anyone wants to spend time
>
>> on those patches I'm glad to hand them over and have them work on
>
>> them.
>
>
>
> I can't volunteer for the development effort, but I'd like to take a quick
> look, thanks.
>
>
>
>> They are against an older kernel and Linux userspace library and
>
>> deamon but the ideas are probably still sound.
>
>>
>
>> For the initial implementation we have for Linux RPCSEC_GSSv3 isn't
>
>> necessary. However I'm not sure what changes need to be made to the
>
>> NFSv4.2 to say that process label transport will be handled later in a
>
>> future RFC.
>
>
>
> I think we just say process labels are future work.  I don't think removing
> references to RPCSEC_GSSv3 would be a big job.
>
>
>
> Server-to-server looks harder; I guess we'd either:
>
>
>

---

From: Spencer Shepler <[email-masked]>
Date: Fri, 9 Aug 2013 19:39:46 +0000

Thanks for offering to help, Andy.

As you have pointed to one question on content, the first step would be to enumerate (with the WG's help) the list of items to be completed for RPCSEC_GSSv3; once we agree on that, then you have clear direction and won't waste editing time where unneeded.

So, please provide input to Andy as he builds his own list of todos.

Spencer

> From: Andy Adamson [mailto:[email-masked]]
> Sent: Friday, August 9, 2013 12:29 PM
> To: Spencer Shepler
> Cc: [email-masked]
> > thought it best to continue this discussion under a separate topic to
> > ensure that everyone has had a chance to comment (see thread content
> > in question below).
> >
> >
> >
> > As my original email noted, the NFSv4.2 I-D has a normative reference
> > on the
> > RPCSEC_GSSv3 I-D.
> >
> >
> >
> I found the git hub for RPCSEC_GSSv3 which I suppose has a list of issues. Is
> there any other repository of past reviews?
> 
> At first glance, it looks to replicate the RPCSEC_GSSv2 channel bindings, and I
> would consider removing it.
> 
> -->Andy
> .
> >
> >
> >
> > I can't volunteer for the development effort, but I'd like to take a
> > quick look, thanks.
> >
> >
> >
> > I think we just say process labels are future work.  I don't think
> > removing references to RPCSEC_GSSv3 would be a big job.
> >
> >
> >

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 9 Aug 2013 19:49:20 +0000

> issues. Is there any other repository of past reviews?

They would be on the mailing list.

> bindings, and I would consider removing it.

I think it was replacing it.

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 9 Aug 2013 19:58:48 +0000

Pulled these in…

---

From: David Quigley <[email-masked]>
Date: Sat, 10 Aug 2013 00:30:32 -0400

On 08/09/2013 15:29, Andy Adamson wrote:
>> ensure
>> The topic is completing the NFSv4.2 work and by association the 
>> RPCSEC_GSSv3
>> I-D.
>> 
>> 
>> 

Nico seemed pretty adamant about the channel binding portion. He 
believes its a significant cleanup of the spec in terms of channel 
bindings and even though it looks like the channel bindings from 
RPCSEC_GSSv2 the last time I heard him mention them he said it was our 
chance to do channel binding properly so I assume he has made 
improvements over the v2 spec.

>> and if
>> quick
>> removing
>>                 Specifically, "The specification MUST provide guidance 
>> for
>> using
>> 
>>                 b) Specify only intra-server copy, leave inter-server 
>> for
>> 4.3.
>> 
>> 
>>

---

From: Nico Williams <[email-masked]>
Date: Sat, 10 Aug 2013 16:29:45 -0500

On Fri, Aug 9, 2013 at 11:30 PM, David Quigley <[email-masked]> wrote:
> Nico seemed pretty adamant about the channel binding portion. He believes
> its a significant cleanup of the spec in terms of channel bindings and even
> though it looks like the channel bindings from RPCSEC_GSSv2 the last time I
> heard him mention them he said it was our chance to do channel binding
> properly so I assume he has made improvements over the v2 spec.

The I-D lays that out (see quote below).  In particular v2 uses a hash
function, which it has to negotiate (ugh), while v3 does not because
it turns out that channel bindings data can be assumed to be small
(all the TLS ones are, and if anyone implements RFC5660 then we can
trivially define small IPsec CB data as well).

A second reason to prefer the v3 approach is that it's notionally
simpler since it really just builds on v1, plus it's extensible and
all that.

2.1.2.2.  Channel binding

   RPCSEC_GSSv3 provides a different way to do channel binding than
   RPCSEC_GSSv2.  Specifically:

   a.  RPCSEC_GSSv3 builds on RPCSEC_GSSv1 by reusing existing,
       established context handles rather than providing a different RPC
       security flavor for establishing context handles,

   b.  channel bindings data are not hashed because the community now
       agrees that it is the secure channel's responsibility to produce
       channel bindings data of manageable size.

   (a) is useful in keeping RPCSEC_GSSv3 simple in general, not just for
   channel binding. (b) is useful in keeping RPCSEC_GSSv3 simple
   specifically for channel binding.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Mon, 12 Aug 2013 18:04:05 -0400

> simpler since it really just builds on v1, plus it's extensible and

Right, but has anyone actually used channel bindings, or indicated any
intention to?

If not then the simplest thing by far would be just to leave them alone
and deprecate them entirely, if that's possible....

--b.

---

From: Nico Williams <[email-masked]>
Date: Mon, 12 Aug 2013 17:53:26 -0500

> Right, but has anyone actually used channel bindings, or indicated any

Not to IPsec -- no one has bothered implementing RFC5660.  But people
do run NFS over SSHv2, and channel bindings for SSHv2 are trivial to
define: it's the SSHv2 session ID, so all you need to get NFS over SSH
w/ CB to work is for the ssh client/server to export the session ID,
and that's easy enough.

> and deprecate them entirely, if that's possible....

For me CB is secondary functionality for RPCSEC_GSSv3, but I don't
want to go do a v4 later on when we do need it.

Let's do a thought experiment we've done before: can we get any of
these features without a new RPCSEC_GSS version?  Yes, we can:

a) let the mechanism take care of compound authentication (e.g., using
FAST in the Kerberos case, see RFC6113);

b) use generic name attributes for client assertions (i.e., use
RFC6680 GSS_Set_name_attribute() on the client side and
GSS_Get_name_attribute() on the server side);

c) use a new NFSv4 op for CB with one argument (the client's CB data),
and a single result (the server's CB data).

If no one wants RPCSEC_GSSv3 now (or ever) then I propose that each of
these features be tackled individually as they come up, roughly as
outlined above...

...which perhaps is superior to new RPCSEC_GSS anyways because it's
all more generic.

Nico
--

---

From: Andy Adamson <[email-masked]>
Date: Fri, 23 Aug 2013 17:17:40 -0400

> these features be tackled individually as they come up, roughly as

I said that I would give RPCSEC_GSSv3 a go, driven by it's use in
NFSv4.2. So, I'm reviewing  draft-ietf-nfsv4-rpcsec-gssv3-05 WRT
server to server copy and labeled-nfs needs. Given an RPCSEC_GSSv3
draft that satisfies those two features and is ready for review, it
should not be much more work to add CB or even client attributes -
especially if I get help from those interested.

To that end, I started with understanding down to the individual
protocol field level how draft-ietf-nfsv4-minorversion2-19 uses
draft-ietf-nfsv4-rpcsec-gssv3-05. I have summarized it in the
attachment, expanding the NFSv4.2 draft "Figure 4: A synchronous
inter-server copy" to include all of the RPCSEC_GSSv3 and
RPCSEC_GSS_v1 messages and details.

Nico (and anyone else) - could you please take a look and see if I got it right?

Thanks

-->Andy

---

From: "faibish, sorin" <[email-masked]>
Date: Fri, 23 Aug 2013 17:19:28 -0400

Why? Who will own the draft?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy Adamson
Sent: Friday, August 23, 2013 5:18 PM
To: Nico Williams
Cc: J. Bruce Fields; NFSv4
Subject: Re: [nfsv4] Soliciting next steps for RPCSEC_GSSv3

On Mon, Aug 12, 2013 at 6:53 PM, Nico Williams <[email-masked]> wrote:
>> Right, but has anyone actually used channel bindings, or indicated 
>> any intention to?
>
>> If not then the simplest thing by far would be just to leave them 
>> alone and deprecate them entirely, if that's possible....
>

I said that I would give RPCSEC_GSSv3 a go, driven by it's use in NFSv4.2. So, I'm reviewing  draft-ietf-nfsv4-rpcsec-gssv3-05 WRT server to server copy and labeled-nfs needs. Given an RPCSEC_GSSv3 draft that satisfies those two features and is ready for review, it should not be much more work to add CB or even client attributes - especially if I get help from those interested.

To that end, I started with understanding down to the individual protocol field level how draft-ietf-nfsv4-minorversion2-19 uses draft-ietf-nfsv4-rpcsec-gssv3-05. I have summarized it in the attachment, expanding the NFSv4.2 draft "Figure 4: A synchronous inter-server copy" to include all of the RPCSEC_GSSv3 and
RPCSEC_GSS_v1 messages and details.

Thanks

-->Andy

---

From: Andy Adamson <[email-masked]>
Date: Fri, 23 Aug 2013 17:27:27 -0400

> Why? Who will own the draft?

I will, if it is decided that the solution in
draft-ietf-nfsv4-minorversion2-20 is either not as good - or will take
as long/longer.

-->Andy

---

From: Nico Williams <[email-masked]>
Date: Fri, 23 Aug 2013 17:04:41 -0500

> Nico (and anyone else) - could you please take a look and see if I got it right?

Sure, see below.

> ******************************************************************
> Detailed inter-server copy example using NFSv4.1 and RPCSEC_GSS_V3
> ******************************************************************
> 
> RPCSEC_GSSv3 background. Using draft-ietf-nfsv4-minorversion2-19.txt and
> draft-ietf-nfsv4-rpcsec-gssv3-05 
> 
> Creating an RPCSEC_GSS3_CREATE message with Privilege Assertions piggy-backed
> on RPCSEC_GSS_VERS_1 contexts: (Creating an RPCSEC_GSS_DESTROY message
> is similar). 
> 
> 1) RPC message call_body struct: rpcvers = 2, prog and vers is for NFSv4.x
>    and the proc is set to the COMPOUND = 1. 
> 
>    RPCSEC_GSSv3 control messages are the same as RPCSEC_GSSv1 data
>    messages, but with a gss_proc value that indicates a control message
>    is contained in the data payload.

No, they are the same as v1 control messages, but with the union
rpc_gss_cred_t version set to 3 and the value being of type
rpc_gss_cred_vers_3_t instead of rpc_gss_cred_vers_1_t.

> 	??? An RPCSEC_GSS_V1 data message - doesn't it need
> 	    an NFSv4 operation. Perhaps solo SEQUENCE op?

The application's RPCs are not relevant in any RPCSEC_GSSv* control
message.

> 2) Within the RPC message protocol's cred structure, flavor is set to
>    RPCSEC_GSS (6).  The opaque data of the cred structure (the body
>    field) constituting the credential encodes the rpc_gss_cred_t:

A complete RPCSEC_GSSv3 CREATE control message from the client looks like:

   XID
   msg_type (CALL)
   call_body
    rpcvers (2)
    prog (NFS)
    vers (NFS version)
    proc (NULLPROC)
    opaque_auth cred
     flavor (GSS -- 6)
     opaque<>
    opaque_auth verf
     flavor (GSS -- 6)
     opaque<>
      GSS MIC token per-RPCSEC_GSSv1 (IIRC)
    call data
     rpc_gss_cred_t
      rgc_version (3)
       rpc_gss_cred_vers_3_t
        gss_proc (6)
        seq_num (whatever)
        rpc_gss_service_t (1, 2, or 3)
        handle (RPCSEC_GSS handle number)
        handle_version                    <---- dunno what this is
        opaque<>
         GSS Wrap token or plain data, per rpc_gss_service_t
          rpc_gss3_create_args
           *compound_binding
           *chan_binding_mic
           assertions<>
           extensions<>

Now, each of those four fields (compound_binding, chan_binding_mic,
assertions, and extensions) are defined in the draft.

If you only want to implement one, or two of them, you can.  You could
implement none also, but there'd be no point :)

> 3) Verifier field in the RPC message header: Just like any other RPCSEC_GSS_V1
>    Data message, constructed using the gss_context from the
>    handle in rpc_gss_cred_vers_3_t.

Correct.

> 4) Request data:
> 
>    struct rpc_gss_integ_data {
>        opaque databody_integ<>; < the RPCSEC_GSS3_CREATE data >
>        opaque checksum<>;
>    };

Yes.

> Creating RPCSEC_GSS3_CREATE privilege assertion:
> 
> These go into the RPCSEC_GSS_V1 Data payload, opaque databody_integ<> for 
> rpc_gss_svc_integrity.

Well, v1 control payload, but sure.

>    struct rpc_gss3_assertion {
>        bool                    critical;
>        rpc_gss3_assertion_u    assertion;
>    };
> 
>    union rpc_gss3_assertion_u
>        switch (rpc_gss3_assertion_type atype = PRIVILEGES;
>     
>        rpc_gss3_privs privs = <the Privilege to transmit> 
>     
>    where typedef opaque rpc_gss3_privs<>

I'm not sure what privileges have to do with server-to-server copy.

>      Client                Source            Destination
>         +                    +                    +
>         |                    |                    |
>    C1   |- copy_from_auth -->| S1                 |
>         |<------------------/| Returns GSS3 H1    |
>         |                    |                    |
>         |                    |                    |
>    C2   |--- copy_to_auth ----------------------->| D1
>         |<---------------------------------------/| Returns GSS3 H2
>         |                    |                    |
>    C3   |--- OPEN        --->|                    | Returns os1
>         |<------------------/|                    |
>         |                    |                    |
>    C4   |--- COPY_NOTIFY --->| Uses GSS3 H1       |
>         |<------------------/|                    |
>         |                    |                    |
>    C5   |--- OPEN ------------------------------->| Returns os2
>         |<---------------------------------------/|
>         |                    |                    |
>    C6   |--- COPY ------------------------------->| Uses GSS3 H2
>         |                    |                    |
>         |                 S2 |<-copy_confirm_auth-| D2 
>         |    Returns GSS3 H3 |\------------------>|
>         |                    |                    |
>         |     Uses GSS3 H3   |<----- READ --------| D3
>         |                    |\------------------>|
>         |                    |                    |
>         |                    |        .           | Multiple reads may
>         |                    |        .           | be necessary
>         |                    |        .           |
>         |                    |                    |
>         |                    |                    |
>  C6-REP |<---------------------------------------/| Destination replies
>         |                    |                    | to COPY
>         |                    |                    |
>         |                    |<--destroy H3-------| D4
>         |                    |\------------------>|
>         |                    |                    |
>    C7   |--- CLOSE ------------------------------>| Release open state
>         |<---------------------------------------/|
>         |                    |                    |
>         |<----------destroy H2--------------------| D5
>         |/--------------------------------------->|
>         |                    |                    |
>    C8   |--- CLOSE       --->|                    | Release open state
>         |<------------------/|                    |
>         |                    |                    |
>         |<--destroy H1 ------| S3                 |
>         |/------------------>|                    |
> 
>                 Figure 4: A synchronous inter-server copy.

Sure, I guess.  The client authenticates to the source and destination
and tells the destination that the source principal is allowed to write
to the given object.  So far so good.  What's RPCSEC_GSSv3 got to do
with any of this?  I would think you just need an OP that the client can
use to ask the source what principal name it will be using, then another
OP by which to inform the destination of the one-time (danger, here be
dragons?) authorization.

I stop here because I'm at a loss as to v3's relevance to
server-to-server copy.  Maybe I knew once but if so, I've forgotten.
What was its relevance?

Nico
--

---

From: Andy Adamson <[email-masked]>
Date: Fri, 23 Aug 2013 18:56:04 -0400

> OP by which to inform the destination of the one-time (danger, here be

Hi Nico

Thanks for the review.

> server-to-server copy.  Maybe I knew once but if so, I've forgotten.

Privileges: for exactly the "danger, here be dragons" you use above:
one-time authorization. Because you create and the destroy the
RPCSEC_GSSv3 privilege handles.

-->Andy

---

From: Nico Williams <[email-masked]>
Date: Fri, 23 Aug 2013 18:01:06 -0500

> RPCSEC_GSSv3 privilege handles.

So you want to grant one principal, one-time (difficult), a narrow bit
of access.

RPCSEC_GSSv3 allows clients to assert specific privs for themselves,
but that's not what the source server needs to do anyways (the
destination will figure it out all by itself and what the client said
about the source).  And because RPCSEC_GSSv3 is about describing the
*subject*, not some third party, it's really not relevant.

What I think you want is just a representation at the NFSv4 layer of
{principal <source> is allowed to write <N> bytes at <offset> of
<FH>}.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Fri, 23 Aug 2013 18:30:13 -0500

So, I think RPCSEC_GSSv3 is not needed for server-to-server copy.  I
don't see how it could be useful there.

I think the only uses for RPCSEC_GSSv3 are:

 - labeled security, for clients to narrow a GSS credential's label range

 - AUTH_SYS replacement, for clients to be able to better describe
process credentials, particularly when either the client "owns" the
share (and can assert anything it likes) or when the process'
credentials are narrower than those described by the GSS credential
(e.g., the process dropped supplementary groups)

 - compound authentication, for protecting against the cache poisoning attack

 - channel binding without the need to negotite a hash function as in v2

All four uses could be addressed using RFC6680 GSS name attributes,
but at the price of having many more RPCSEC_GSS context handles than
is necessary.

Nico
--

---

From: Andy Adamson <[email-masked]>
Date: Fri, 23 Aug 2013 20:17:12 -0400

> So you want to grant one principal, one-time (difficult), a narrow bit

We want to enable the user on the client to grant the destination
server the ability to act as a client on the users behalf, and
exercise the granted privilege until the privilege is revoked - the
privilege to READ a specific file.

> about the source).

The source server needs to know that the client (the destination
server for the COPY) has the privilege to do the copy, granted by the
user who has the file OPEN on the source server. So AFAICS, as you put
it above, it allows the "client to assert specific privs for
themselves".

> And because RPCSEC_GSSv3 is about describing the

Well, the RPCSEC_GSSv3 compound authentication is all about
identifying  some third party (the inner context handle). That
combined with a privilege seems to me to do what NFSv4.2 SSC requires.

> {principal <source> is allowed to write <N> bytes at <offset> of

Yes, but that is not all. We want to have a mechanism to limit the
ability of the destination server to copy only one FH which I believe
this does.

I also think that revoking the RPCSEC_GSS [privilege + compound auth]
also stops the ability of the COPY to be re-done etc.

>From draft-ietf-nfsv4-minorversion2-19: section 3.4.1.2

   An approach that sends delegated credentials of the client's user
   principal to the destination server is not used for the following
   reasons.  If the client's user delegated its credentials, the
   destination would authenticate as the user principal.  If the
   destination were using the NFSv4 protocol to perform the copy, then
   the source server would authenticate the destination server as the
   user principal, and the file copy would securely proceed.  However,
   this approach would allow the destination server to copy other files.
   The user principal would have to trust the destination server to not
   do so.  This is counter to the requirements, and therefore is not
   considered.  Instead an approach using RPCSEC_GSSv3 [rpcsecgssv3]
   privileges is proposed.

-->Andy

---

From: Nico Williams <[email-masked]>
Date: Fri, 23 Aug 2013 20:16:49 -0500

> combined with a privilege seems to me to do what NFSv4.2 SSC requires.

Oh.  I hadn't understood that you wanted to combine a user and a
server initiator credential.  The goal for me had been to combine a
client host and a client user credential.  I guess this alternative
can work too, but... the server that will be acting as a client needs
to be in possession of the security context, OR that server has to
round-trip all its messages to the other server via the real client.
Seems tricky.  I'll think about it.

Nico
--

---

From: Andy Adamson <[email-masked]>
Date: Tue, 27 Aug 2013 12:43:43 -0400

On Fri, Aug 23, 2013 at 9:16 PM, Nico Williams <[email-masked]> wrote:
> to be in possession of the security context,

I believe here you are referring to the RPCSEC_GSS_v1 GSS context of
the principal who initiated the COPY.  Said principal must OPEN the
file to be copied on the source and destination server, and NFSv4.2
must insist that this OPEN use RPC_AUTH_GSS, which ensures that both
the source and destination server have the GSS security context.

-->Andy

> OR that server has to