Subject: Getting to proposed spec, again (Chapter 1).

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 16 Jul 2002 12:35:03 -0700

This message is broken up macause of majordomo posting limits.
This is the first chapter.

I've been reviewing the issues list, seeing what we need to
do to get the spec resubmitted by early August.  There are
tons of things that are on the list that are still not marked
resolved.  Lots of stuff on the list is of the it-would-be-
nice-if variety.  Those things are just not going to get 
resolved in time, and we have to recognize that and focus on
the things that actually need to get resolved.

The following is my categorization of the 205 issues on the 
issues list.  I'd like people to look at it and see where they
agree and disagree.  If something is important to you, then
let's get it on the table, and get someone actively addressing
it (and that may be you), rather than having it just sit on the
issues list.

Things that are marked resolved.  I haven't checked the spec to
verify the resolution, but I don't expect these to be troublesome.

     1, 3-15, 17-37, 43, 46, 48, 49, 55, 56, 66, 70, 71, 75, 84,
     88, 90, 102, 122-124, 126-128, 130, 134, 135, 137, 146, 149-152,
     155-157, 161-164, 171, 179, 180, 182, 183, 185, 190, 196,
     200, 201, 205 

Trivial stuff.  It is clear what extra sentence or two needs to
be put in so let's just do it.

     2, 53, 67, 68, 69, 91, 103, 106, 117, 153, 154, 173, 176

Should be no-change or at least any change is not important enough 
to hold things up for.  This includes cases where something is dup'ed 
or otherwise merged into another issue.  This is my judgement and I
expect different opinions, which we can hash out but I'd like people
to try to be flexible to help us get the spec out in a reasonable
time.

     16, 45, 52, 54, 58, 60, 62, 64, 73, 79, 80, 82, 85, 87, 89, 94,  
     95, 105, 107, 108, 110, 111, 118, 119, 121, 125, 131, 132, 138,  
     140, 141, 143, 145, 147, 147A, 148, 159, 160, 168-170, 172, 175, 
     184, 188, 189, 195, 197, 199, 202-204

Why aren't these resolved?  What more is to be done? (i.e. I think these
are resolved from recent changes).

     38, 41, 44, 77, 86, 109, 167, 174

Group needs to discuss this to decide what if anything we need to
do about these.

     42, 47, 59, 63, 99, 115, 120, 133, 136, 165, 177, 

I think something needs to be done here, to get us to a reasonable
spec for a new Proposed Standard.  A good number of these in areas 
that I've written so it makes sense for Spencer and me to divide 
these up.

     50, 51, 57, 61, 72, 81, 83, 92, 93, 96, 97, 98, 101, 104, 112,  
     113, 114, 129, 142, 178, 181, 186, 187, 191, 193, 194  
 
What's the issue here?  There wasn't enough in the description
for me to understand it.  Anybody who understands the issue should 
please provide some detail:

     58, 74, 76, 100, 139, 144, 166, 192 

----------------------------

So here's some more detailed comments on some of the issues.

Issue 16: Should a value be added to the fh_expire_type attribute to allow 
the server to signify that for replicated filesystems the filehandles are 
the same between servers. 

     Isn't that what persistent means?  If they are persistent except that
     they expire on migration, we have a value for that.  Looks like
     that's done.

Issue 38: The current state model does not allow for proper implementation 
for a Posix client in the face of fork(). Other issues will be documented 
here as the resolution is more fully defined. 

     It says this is partially resolved by the new locking model but says 
     there are issues remaining.  What are they?  If they exist, let's
     get them documented.  If not, let's declare victory.

Issue 41: Clarify the special stateid's of 0 and all 1's and clean up 
first paragraph of 8.9. 

      Wasn't this addressed by my rewrite of this area?

Issue 42: Section 14.2.10 (LOCK). Clarify that LOCK requests are subject 
to permission checks. Also need to clarify association between nfs4_lock_type 
and the permissions that would have to be specified in the corresponding 
OPEN call. For example, WRITE_LT requires OPEN4_SHARE_ACCESS_BOTH, 
not just OPEN4_SHARE_ACCESS_WRITE. 

      If we go down this road, there's probably tons of semantics.  The
      group should discuss this and decide whether we want to go down this
      road.

Issue 44: Clarify semantics of user@domain usage and mappings and implicit
association with RPC credentials.

     What needs to be done here?  Has this been addressed (as well as we
     are going to address it anyway) by the work I did in this area.

Issue 45: Clarify the client's behavior when a filesystem is replicated 
and provides persistent filehandles (section 6.4) 

     What's to clarify?  They are persistent.  They can't change so they
     can't change in going from server to server, so they have to be same.
     This is hard but not impossible to do.  I don't see that any change
     is required.

Issue 47: Dangling state references at the server 
 
     The group should decide how important this is.  My opinion is that the
     issues of dealing with possible client and server implementation errors
     is something that we don't want to take on, and can't take on if we are
     going to get a spec out in timely fashion.

Issue 50: Clarify the behavior of a RENAME of a file that is open and has 
an outstanding delegation. Need to describe the problems fully and describe 
potential solutions - server recalls delegation at the point a RENAME is 
attempted. The server can choose to add this type of delegation recall when 
a parent directory is renamed along with support for the target file itself. 

     I think this is reasonable to do. 

Issue 51: Value of initial sequence id needs to be described and clarified so 
that unnecessary restrictions are placed upon the value by server implementations. 

     Assume that should be "no unnecessary restrictions" in which case this
     should be done.  I believe the answer is that the initial sequence id
     is unconstrained.  Right?

Issue 52: When the FH4_NOEXPIRE_WITH_OPEN bit should it apply to the server 
returning NFS4ERR_STALE. In other words, should it indicate that the server
will not allow a REMOVE of an OPEN file along with not allowing expiration. 

     I don't see any reason to make this change.  Expire-type deals with 
     expiration and this isn't expiration.  Further, even if a server normally,
     prevented remove on open files, it would be likely to allow local removes
     (e.g. if requested by administrator), or unexport to cause STALE, so the
     guarantee couldn't be reliable.

Issue 53: For write-only attributes, need to clarify behavior when these are 
specified for VERIFY/NVERIFY operations.

     The client's reach has exceeded its grasp, or what's EINVAL for?
     Also the rdattr_error (from Issue 119) should be dealt with here.

     So add the following sentence to the last paragraph of the 
     IMPLEMENTATION sections for NVERIFY/VERIFY:

     When the attribute rdattr_error or any write-only attribute (e.g. 
     time_modify_set) is specified, the error NFS4ERR_INVAL is returned 
     to the client.  If both of these conditions apply, the server is free
     to return either error.

     Also add INVAL of the error list for NVERIFY (it is in VERIFY).

Issue 54: Section on I18N needs to cleaned up for consistency in usage of 
terms (charset, language, script).      

     It doesn't need to.  It would be nice to, if we had the time.  I suspect
     we don't.

Issue 57: OPEN upgrade and share conflicts. Need to specify how the server 
is to apply share checks on an OPEN_UPGRADE. Suggestion is to have the share 
checks done first before applying OPEN_UPGRADE and will a conflicting share
mode fail an OPEN_UPGRADE? 

     I don't understand the suggestion.  I would resolve this by stating that
     an OPEN is to check for any conflicting opens, with the exclusion of the
     one being upgraded (i.e. the one for the same file and lockowner) which the
     client can check locally.

     I can come up with a detailed proposal for spec changes.

Issue 59: File locking clarifications

     My choices:

     All one's means upto 2**64-1 (or 2**32-1).  Any other case of overflow
     or zero length --> EINVAL.

     You're not required to support 64 bits.  A server may truncate requests
     that space the end of the 32-bit range.  However, he has to return an
     error (not sure which one yet), if the range includes the last byte
     in the 32-bit range but not all of the range beyond it up to 2**64-1.

     If the server says he can do it, he does and client just sends requests.
     If not, up to client to simulate this monstrosity but no guarantees.

     The group should discuss this stuff and then someone can come up with a 
     detailed spec proposal.

Issue 60: Recovery from "out-of-sequence" conditions. 

     I say no.  Unless already someone has a detailed proposal, there's no
     chance of addressing this in time.  Even if they did, it would be kind
     of risky. 

Issue 61: Retransmission of CLOSE (how long to hold state) 

     I'll come up with something based on my comments in the issue and the
     fact that the reply cache will help here.

Issue 62: Security Considerations section should have a description that 
the stateid is not cryptographically secure so should not be used for access
checking.

     It clearly isn't cryptographically secure so why bother.

Issue 63: Add description of RPC authentication and the use of the principal 
for various file operations: OPEN, READ, WRITE. Should principal and its 
access be check for all of these operations? 

     I'm inclined to leave the spec alone here.  Not sure what needs to be 
     added.  I think the groups needs to discuss this.

Issue 65: What choice should the implementor make when building the list of
acceptable security mechanisms for the pseudo filesystem nodes? Should it be 
union of those specified for the real filesystems? If an administrative model
exists to specify the mechanisms, is this the best choice? 

     Whatever choice matches his degree of paranoia.  We don't have to discuss
     every implementation choice.  This could be discussed in an implementation
     RFC.

Issue 67: For LOCKT, need explanation of how READ_LT/READW_LT should be handled
in the same fashion (same for WRITExxx)

     So let's add the relevant sentence or two. 

Issue 68: Named attribute clarification: OPEN needs to mention creation of a 
named attribute file.

     This is basically just a paragraph saying that if the current file handle
     is the attribute directory, OPEN will create or open a named attribute 
     file. Let's just do it.

Issue 64: SECINFO needs to include a interpretation of what security mechanism 
is required for the operation. In other words, is it the current filehandle 
or the component that the server uses to determine if the RPC mechanism in use 
is appropriate?

     I don't understand.  How could it be the component?  If it were the 
     component, you would have to know in advance what security mechanism
     you could use and if you knew that, why would you need to ask? 

Should SECINFO follow the same search/access control for finding 
the component as is done for LOOKUP?

     Yes.  It looks things up and there is no separate LOOKUP-for-SECINFO
     ACCESS right. 

Issue 69: Clarify the "named_attr" attribute to signify that it is TRUE for a 
non-empty named attribute directory

     So let's just add that sentence.

Issue 72: OPEN needs clarification of permission/ACLs be checked against 
the requested access. ACCESS also needs to be referenced.

     I guess this should be done. 

Issue 73: Clarify what credentials should be used for subsequent READ/WRITE 
requests as they relate to the initial OPEN.  

     I don't think there is any clarification required.  Here imagine that
     in most cases the same credentials would be required but there's no
     reason I can see to require that.  Shouldn't we leave things as they
     are?

Issue 74: Based on comments in http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2001/0048.html 
add appropriate clarification comments on the incrementing of seqid on error. 

     I'm confused.  The URL leads to a discussion of reiserfs and READDIR 
     cookies???

Issue 76: For user/group identification, add discussion of implementation 
choices for domain name equivalence. 

     I'm not sure exactly what is being requested here.

Issue 77: Clarification in Section 8.1.4. It says:

     I'm pretty sure this was addressed by my new version of this section.

Issue 79: Clarify upgrade/downgrade paths for file LOCKing 

     Besides what is being done for Issue 57, not sure what else is
     there is to do.

Issue 80: What are the effects of OPEN(upgrade) with respect to share 
reservations?

     Sounds like a dup of 79 or 57.

Issue 81: Clarification of the reuse of persistent filehandles by the 
server (after file/directory removal) needs to be added.

     Someone needs to come up with a paragraph that says don't reuse
     filehandles for a very long time, or you'll be sorry.

Issue 82: Add a discussion of a server's implementation of a duplicate request cache and expectation of how it should be used.   

     No chance of this getting done in time.  It would be nice but there's
     no way.

Issue 83: Clarify what happens when OPEN returns a delegation and the client
does not do a OPEN_CONFIRM. For example, conflicting OPEN is received, does 
the server recall the delegation? Or client returns the delegation before 
OPEN_CONFIRM, what error is returned, if any. What happens if the client 
specified the delegation in an OPEN with claim type DLG_CUR? Does this 
confirm the delegation, or is it invalid? 

     I need to address this. 

Issue 85: Clarify that SETCLIENTID/SETCLIENTID_CONFIRM sequence with same 
verifier is a no-op (no release of state).

     I think this should be merged into issue 114 (along with issue 175).

Issue 86: For delegation recall, the client needs to use separate programs 
or port numbers to differentiate between servers callbacks since filehandles 
and stateids may collide between servers 

     This is obviated by the resolution of Issue 205.

Issue 87: Clarify that the root filehandle and public filehandle are 
directories; the root filehandle is the actual root of the server's 
namespace and of all pseudo filesystems

     The spec talks about finding the filesystems by walking down
     the directory tree (or something like that).  That the root is
     directory is a fair implication.  I don't see making every tiny
     detail explicit.  Let's let this go.

Issue 89: When the client obtains the value for fs_locations, the request 
should be made with either integrity or privacy so that the client is not 
redirected surreptitiously.

     And when you read a file?  Based on the environment, implementors
     will decide.  The spec defines the protocol.  Let's leave it at
     that without getting into all this.  Even if we had the time. 

Issue 91: Clarify that the server is supposed to free state for a restarted/
rebooted client when SETCLIENTID_CONFIRM is received. The document is 
inconsistent, sections 8.1.1 and 14.2.34 say that the server should wait 
for SETCLIENTID_CONFIRM; Section 14.2.33 says the server can free state 
upon receipt of SETCLIENTID.

     14.2.33 is wrong so let's fix it.

Issue 92: Clarify statements in section 8.5.1 where it says that in a 
``secure environment'', only the locks that are ``associated with the 
authenticated requester'' are freed when a stale nfs_client_id verifier is 
received. Additional discussion of the security considerations is needed.

     I agree this that this needs to be clarified.  I think this referring
     to the principal check and CLID_INUSE but as written it seems to say
     that a subset of the locks for the client are going to be freed, which
     is not the case.

Issue 93: Section 8.5.3 doesn't give the whole story about partitions and 
recovery. At the very least, it should include a pointer to the discussion 
of "unvalidated" locks in Section 8.7. 

     8.5.3 discusses this from the server point of view while the discussion
     in 8.7 about "unvalidated" locks discusses the client's handling.  A
     pointer makes sense.

---

From: [email-masked]
Date: Thu, 18 Jul 2002 11:52:32 -0700

This is in your "what is this issue?" list.

The issues list shows this as

    OPEN upgrade and OPEN_DOWNGRADE paths and the ability of NT and
    Unix to interoperate appropriate with the way the state
    information is currently designed.

Is this Sergey's 1 Oct 2001 mail
(http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2001/0578.html)
?

mike

---

From: [email-masked]
Date: Thu, 18 Jul 2002 11:52:58 -0700

This is in your "what's the issue?" list.

I think the URL in the Issues list is wrong.  It should apparently be
0038.html, not 0048.html.

mike

---

From: [email-masked]
Date: Thu, 18 Jul 2002 11:53:35 -0700

I believe this has to do with a situation we've run into inside Sun,
where users can have equivalent logins in different DNS domains (e.g.,
[email-masked] is the same person as [email-masked], and
in a couple months will be the same as [email-masked]).  The
Solaris v4 implementors tossed around a couple approaches to make this
easier to manage, and I think the issue is to get some of these ideas
onto paper for the benefit of other implementors.

mike