Subject: RE: V4 complexity (was re: Proposed charter addition)

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 13 Dec 2002 09:13:22 -0800

I don't think much of this fits the "remove features when going 
to draft" model, as I understand it.

The issue of leaving leaving lots of elements of semantics 
unspecified might somehow be coerced into that model with the
"feature" being the ability of the server to provide different
behaviors while staying within the spec.  But that would be 
a stretch.  The spec takes the position that certain semantic
matters are outside the scope of the protocol, making a
restriction look (at least at first glace) more like a
straight change rather than a feature removal.  Besides, I
don't think that this is truly something that will prevent 
interoperable clients and servers.  It is more a question
of user confusion and I hope that we can reduce that problem
by documenting our implementation choices better (and in a
more common format).

In the filehandle area, it might be possible to get rid of
filehandles that can disappear even when a file is open.
I'm at least hoping that nobody will implement that.

As far as the state stuff goes, I don't think there is
anything that can be removed.  I think something would need
to be added (a common mechanism for exactly-once semantics) 
to provide the simplification.  And I don't think the protocol 
is too complicated to allow good inter-operable implementa-
tions.  I would prefer it be simpler and I'm sorry that it isn't 
simpler and I worry about implementing possible future changes
for minor versions in the current framework.  But in a minor
version, we could decide to solve the basic problem.

-----Original Message-----
From: Talpey, Thomas [mailto:[email-masked]]
Sent: Thursday, December 12, 2002 4:26 PM
To: [email-masked]
Subject: Re: V4 complexity (was re: Proposed charter addition)

One of the things about a Proposed standard is that it can
contain speculative protocol features, which are in theory
weeded out after seeing what actually gets implemented.
How many of the items you mention below fall into this
potential category?

In other words, it's our collective call whether the protocol
stays the way it was written. If it's too complex, the proof
is in the interoperable implementations.

Tom.

At 11:59 AM 12/12/2002, Noveck, Dave wrote:
>Neil Brown wrote: 
>> On Wednesday December 11, [email-masked] wrote:
>> > 
>> > I can't say whether you're the only administrator, but I think
>> > almost everybody on this list is an NFS user.  Will everybody who
>> > doesn't use NFS, please sound off?  Also, any other administrators. 
>> > 
>> 
>> Well, seeing you asked.  I'm an administrator (and user and
>> developer..)
>>
>> As an administrator I want my protocol to 'just work'.   I should be
>> able to configure all sorts of interesting things, but I shouldn't
>> have to in most cases.
>> I should be able to expect that different implementations will
>> inter-operate seamlessly, and the each implementation will be
>> bug-free.   (No great surprises here).
>
>One protential issue is that the protocol, in order to accommodate
>many possible implementations leaves the semantics unspecified, as
>not part of the protocol.  For example, a NFS-v4 server may reject
>rename of an open file or may allow it.  It's hard to fault this
>position, since such behavior is, after all, not part of the protocol,
>but it does pose interoperability issues.  There are a large number of
>possible implementation choices and so substituting one fully-compliant
>v4 server with another may result in totally different behavior,
>causing the client application to fail.
>
>In practice, at least initially, V4 servers will implement the NFS/UNIX
>semantics we know and love/hate/are-ambivalent-about.  But when there
>are differences, the protocol's interoperability story will suffer.
>I'd like us to do something to forestall this.  One idea is to list
>all the possiblities we can think of, so a server vendor can provide
>his choices in a coherent way, and the client can see if there are
>differences to worry about.  This might be something for that never-
>quite-started implementation RFC.  
>
>> How does that relate to complexity of the protocol and the possible
>> consideration of RDDP??
>
>> Well, as an developer I know that correctness and inter-operability of
>> implementations is closely linked with complexity, but more with the
>> complexity of the conceptual model than with the complexity of the
>> protocol. 
>
>> A typical programmer will have a much better chance of writing correct
>> code if they can understand the protocol - if they can hold a
>> model of the protocol completely in their head.
>> This may allow that the protocol has lots of features and be complex
>> in that sense, but it requires that the features in the protocol have
>> a clean, elegant, uniform model.
>
>If one of the goals was to accommodate windows semantics, than a 
>clean, elegant, uniform model, would not be acceptable.  Sorry.  
>Couldn't resist.
>
>> My assessment of NFSv4 as it stands is that the complexity of the
>> protocol is reasonable, but that the complexity of the model is
>> terrible.  It is terribly in-elegant.  Particularly in the state
>> model and the filehandle model, but also in other areas.  As such I
>> don't have a great deal of confidence that implementors will get it
>> right.  But I don't think there is any hope of major improvements.  I
>> just have to trust that we have a bunch of exceptional programmers.
>
>Yes, but but "exceptional" has come to have such a range of conno-
>tations :-)
>
>I'm not sure what you mean about the filehandle model.  Are you 
>referring to volatile filehandles?  If so, I'm in agreement.  I'm
>definitely in the "Peristent filehandles -- accept no substitutes"
>camp.
>
>As to the state stuff, I don't think it is the state *model* that
>is complex.  The state model is that there are a set of clients
>which identify themself, get an id.  Then, you have individual 
>owners (i.e. processes) which open files, and get an id to 
>identify that open file.  Multiple owners (processes) can get
>locks on the same open file and the processes each get an id
>that identifies the state of their particular locks on the
>file.  Open-upgrade/downgrade is less than elegant but overall,
>the model is pretty clean.
>
>The complexity comes, not in the model, but in the protocol
>specifics.  Most of that is in some form of replay protection.
>Sequence numbers, per-object last request info, the OLD_STATEID
>stuff, some object recycling and confirmation complexities all
>derive from the fact that you are doing stuff that is stateful and
>the old NFS-v2/v3 at-least-once model of things is inadequate.
>That is the one problem that is solved (at least I hope it is 
>solved) in a half-dozen (at least) different places in various
>different ways.  The problem that NFS-v4 faced was that NFS
>implementations had a solution for replays, that did not work
>well enough (i.e. that you could specify it in the protocol and
>show it worked) to adopt for the state stuff, but did not work
>poorly enough to be gotten rid of (and replaced with something
>else).  Not biting the bullet and taking on this issue is the
>biggest mistake we made, and we can't blame it on the charter
>(at least I don't think we can).
>
>I've come to this conclusion after some experience of another
>approach.  The DAFS locking model is very similar to that of V4 
>(actually it is currently RFC3010-based), but it has exactly-
>once semantics.  DAFS and V4 share the wafl code that handles 
>the locking model, and we have tried to separate out the code
>that is v4-specific.  The result is that I have a very
>visible indication (via an if) where a major section of 
>code can simply be avoided since replay protection is somebody
>else's job (the overall protocol by a separate mechanism
>rather than something for the locking code to do).  This has
>given me a severe case of exactly-once-semantics-envy.   
>
>One thing that we need to do to deal with v4 complexity is more 
>sophisticated interoperability tests, including looking at forcing
>pathologoies.  At the last bakeathon, we first tried the connectathon
>tests while rebooting the server.  Let me propose soemthing else
>for the upcoming connectathon.  Let's try the connectathon tests
>while randomly dropping replies.  To avoid not testing very much,
>let's make sure that our NFS reply caches do not cache the state
>management stuff (we'll need to finesse open-create somehow) so
>that the code within the v4 state management that is supposed to
>deal with cases in which the reply cache doesn't, has something 
>substantive to do.  I expect this to be a productive exercise
>(in terms of bugs found), but if not, the resulting mental
>serenity is nothing to sneeze at. 
>
>
>
>  
>
>
>
>