Subject: [nfsv4] nfs objects discussion meeting minutes

From: Benny Halevy <[email-masked]>
Date: Fri, 09 Nov 2012 05:45:10 -0800

We had a good discussion about my proposal for the new nfs-objects layout
at the IETF-85 WG meeting in Atlanta, GA.

Since there is a lot of interest in this layout we held an informal
meeting of interested parties at the hotel venue to continue the discussion.

Participants:
Benny Halevy, Tonian
Tom Haynes, Netapp
Pranoop Erasani, netapp
Mike Eisler, netapp
Boaz Harrosh, Panasas
Matt Benjamin, The Linux Box
Adam Emerson, The Linux Box

Pranoop: do files layout need extension?
Matt: need flexible stripes
Pranoop: You can do that by giving each file its own deviceid but that's not scalable.
Boaz: problematic with notify deviceid

Control protocol:
Benny: Need to specify requirements;
orthogonality, define a protocol that complies with the requirements
Pranoop: forcing users and vendors to implement T10 OSD is a no go
Mike Eisler: How do you protect against malicious clients with NFSv3
Tom: The NFSv3 method seems like a kludge.
Benny: with Kerberized NFSv3 the MDS can forge creds that are hard to break.
Mike: you need to change the software on the server to be able to revoke outstanding creds.
Benny: document the caveats for v3-based security, allow tight security with NFSv4 / NFSv4.1
Mike: With 4.0 you get global stateid + DS based auth
Is 4.0 interesting?
Benny: yes for legacy servers
Tom, Boaz: using 4.0 stateid can work. CLOSEing on the MDS will revoke the stateid
This is worth further investigation
Look into using v4.0 as the base version for robust security model
using v4.0 stateid as the basic "capability".

Matt: Tech requirements for pNFS/Ceph:
files layout + per-file flex striping
use sparse striping pattern (apparently, need to validate that requirement).
Boaz: striped pattern with obj layout is similar to mirrors, just no need to write to all mirrors.
mechanism for extending the obj layout for 4.1 DS: FH, stateid, uid==-1, gid==-1 (full auth)
Mike: need separate requirements I-D

Tom: We need to state the requirements for any new layout protocol
Benny: This exists in 4.1, we need to agree on their interpretation.
Informational RFC possible. Tom volunteers to own this.

Implementation:
Tom: having too many options in the layout complicates testing
Mike: historically, the nfs client has always been the most complex part of the implementation

To summarize, I see the following technical and business requirements for:
1. Adding flexible, per-file striping patterns to the files-layout, a-la RFC5664
   - consider also borrowing RFC5663 client assisted COW method
2. Accordingly, simplify the device info structure
3. Supporting legacy DSs
4. MDS/DS mix-and-match (well specified / standardized back-end control protocol)

WG items:
1. Clarify requirements for new layout types
2. Specify requirements for files layout back-end control protocols
3. Specify a standard MDS/DS control protocol that is optional to implement.

Have I missed anything?

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 9 Nov 2012 13:30:26 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Benny Halevy
> Sent: Friday, November 09, 2012 8:45 AM
> To: nfsv4 mailing list
> Cc: Erasani, Pranoop; Haynes, Tom; Eisler, Michael
> Subject: [nfsv4] nfs objects discussion meeting minutes
> 
> Since there is a lot of interest in this layout we held an informal meeting of
> interested parties at the hotel venue to continue the discussion.
> 
> Participants:
> Benny Halevy, Tonian
> Tom Haynes, Netapp
> Pranoop Erasani, netapp
> Mike Eisler, netapp
> Boaz Harrosh, Panasas
> Pranoop: forcing users and vendors to implement T10 OSD is a no go Mike
> Eisler: How do you protect against malicious clients with NFSv3

Why bother with NFSv3? If you really want to reuse an existing NFSv3 server as a DS, and you can't reflash it to use NFSv4, then another option is to put a $10 computer in front of it to act as a NFSv4 gateway. Trying to get NFSv3 to do safe multi-pathed writes (i.e. safe fallback to write-through-mds) is something that most people gave up trying to do 20 years ago.
As Mike said, any scheme that involves revoking GSS sessions will require modifications to most/all existing servers. Manipulating ACLs as you described in your first draft runs into the problem that there is no RPC standard for changing ACLs on NFSv3...

> Mike: With 4.0 you get global stateid + DS based auth Is 4.0 interesting?
> Tom, Boaz: using 4.0 stateid can work. CLOSEing on the MDS will revoke the
> stateid This is worth further investigation Look into using v4.0 as the base
> version for robust security model using v4.0 stateid as the basic "capability".

Note that a 4.1 stateid will work too as long as the DS doesn't require EXCHGID4_FLAG_BIND_PRINC_STATEID on the open stateid. In order to allow the MDS to set up and own the lease on the DS, you would need to have it pass the resulting short clientid as part of the layout device info. The NFS client would then use that clientid to set up its own session, and would be able to use the stateids that are associated with that lease.

>    - consider also borrowing RFC5663 client assisted COW method 2.
> Accordingly, simplify the device info structure 3. Supporting legacy DSs 4.
> MDS/DS mix-and-match (well specified / standardized back-end control
> protocol)
> 
> WG items:
> 1. Clarify requirements for new layout types 2. Specify requirements for files
> layout back-end control protocols 3. Specify a standard MDS/DS control
> protocol that is optional to implement.
> 
> Have I missed anything?
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4