Subject: RE: special stateids (was "LOOKUP vs OPEN in NFSV4")

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 16 Aug 2001 06:52:19 -0700

Mike Kupfer wrote:
> [Dave Noveck wrote:]
> > This leaves,
> ...
> >      all-ones-stateid on write -- about which the spec is unclear
> >      (or I'm confused, or both).
> > 
> > As not accounted for.
> 
> Section 8.1.4 says this case does not bypass record locking checks.
>
> > Reading 8.1.4 closely, the first paragraph says that zero-stateids
> > are subject to conflicting locks and I assume that includes both 
> > share reservations and byte-range locks (I assume the latter only 
> > if they are mandatory but the spec doesn't say).  Also, in the 
> > penultimate sentence of the first paragraph, it says an error is
> > returned if there is a conflict with an "explicit lock".  What's
> > an implicit lock?  
>
> I believe "implicit lock" refers to the server's wrapping a READ or
> WRITE in a lock to insure atomicity.  See the very last sentence of
> Section 8.1.4.

OK.  
>
> > To top it off the last sentence of the paragraph
> > reads "This allows 'mandatory locking' to be implemented."
> > Any guess on what the referrent for "this" is in that sentence?
> 
> I assumed it refers to the lock checking that happens when an all-0
> stateid is used.

Ok, Maybe.  It just seems odd.

The server is implementing mandatory locking (for all files or for
particular files, such as those with some mode bit set).  I guess
you could say that the fact that it doesn't completely ignore 
mandatory locking on such IO requests "allows mandatory locking
to be implemented" but that just seems an odd way to describe the
fact that this form of IO, like others, obeys the constraints of
mandatory locking.

> > The next paragraph talks about stateid of all ones.  It says that
> > READ operations bypass *record* locking checks on the server (my
> > emphasis).  So do they bypass share reservations locks?  The 
> > implication is that they don't.  But why?  Generally, the poeple
> > who complain about Windows locks stopping them from accessing a
> > file are complaining about share reservations rather than record
> > (i.e. byte-range) locks.
> 
> I dug through my archives, including some 1998 Sun-internal mail that
> led up to David Robinson's locking proposal, which is the basis for
> the current scheme.
> 
> It looks like the idea was that Windows clients would use an all-0
> stateid, which would mean "I don't have any locks, I want to do I/O,
> please honor record locks and share reservations", since Windows apps
> expect mandatory locking semantics.  

But that expectation means that they want (and expect) that their locks
will prevent others, including UNIX others, from doing IO with inconsistent
locks held.  The exclusion is *mandatory*.  The thing about mandatory
locking is it applies even when you do something incosistent, even
if you don't politely ask for it to.

As I understand the spec now, if we leave aside special stateid's
(and non-V4 clients and local access...), share locks are inherently
mandatory while it is up to the server to decide whether byte-range 
locks have mandatory effect (and the server could do it for some files
based on mode bits or whatever).  Is that everybody else's understanding?

Except for this all-ones/all-zeros stuff, there is no way for *clients*
to choose their locking semantics.  Not that I think there should be.

> Unix clients would use an all-1
> stateid, which would mean "I don't have any locks, I want to do I/O,
> please don't honor any locks or share reservations", since Unix apps
> expect advisory locking semantics. 

It would be possible to implement advisory lock semantics by allowing 
byte-range locks to be created that don't have this property of excluding 
inconsistent IO's, but the protocol doesn't allow any way for the client 
to specify that on a per-lock basis.

But providing a way a for one client to ignore another client's mandatory
locks isn't advisory locking.  It's something different and kind of
scary. 

> We then modified the all-1 stateid
> so that reads could bypass locks, but writes couldn't.

Allowing mandatory locking to be bypassed on reads is harmless
(since the requestor knows he might get inconsistent data and
implicitly assents to that).  On writes, it's anything but
harmless.

The issue with the lock bypass on reads is that all servers may
not be able to provide it.  I'm pretty sure NT-based servers
would have problems with this.

> And at some point during the life of the spec we tried to tighten up
> the locking-related language, to clearly distinguish share
> reservations from record locking.  In a few places I suspect we
> (accidentally) changed the meaning from "shares and record locks" to
> "record locks only".
> 
> So, to answer Dave's question, I think that the current text in
> Section 8.1.4 should be changed.  A READ with an all-1 stateid should
> bypass both record locks and share reservations, and a WRITE with an
> all-1 stateid is equivalent to a WRITE with an all-0 stateid.

"should bypass" as opposed to "must bypass"!

That's OK, but my preference would be to avoid two different ways
of doing the same thing.  I would rather allow:

    Read with all-0: Normal READ with no locks or opens.  Subject
    (of course) to mandatory locking.

    Write with all-0: Normal WRITE with no locks or opens.  Subject
    (of course) to mandatory locking.

    Read with all-1: Special READ which bypasses locking checks, and
    thus an exception to mandatory locking, if the server supports the
    exception.

And make invalid:

    Write with all-1: Invalid since a WRITE which bypasses locking checks
    is too horrible to contemplate (for long).

> > Finally (for those who haven't hit delete yet), there is the 
> > issue of special stateid's and the grace period.
> > 
> >      "During the grace period, the server must reject READ and
> >       WRITE operations".
> > 
> > It doesn't seem to make an exception for special stateid's.
> 
> I suspect the logic here is that the server has to recover
> delegations.  So rejecting all I/Os, independent of the stateid, is
> the safest course.

Right.  I think it would help if we added the following text at
the end of the fifth paragraph of section 8.5.2.

   READs with a stateid of all-ones, must also return NFS4ERR_GRACE,
   even though such READs bypass locking checks, unless the server
   can determine that no reclaims of delegations can possibly occur.

---

From: Mike Kupfer <[email-masked]>
Date: Mon, 20 Aug 2001 15:27:51 -0700

>>>>> "DN" == Noveck, Dave <[email-masked]> writes:

    DN> The server is implementing mandatory locking (for all files or
    DN> for particular files, such as those with some mode bit set).
    DN> I guess you could say that the fact that it doesn't completely
    DN> ignore mandatory locking on such IO requests "allows mandatory
    DN> locking to be implemented" but that just seems an odd way to
    DN> describe the fact that this form of IO, like others, obeys the
    DN> constraints of mandatory locking.

I think the point was to contrast with the Unix advisory locking
model, where the client first obtains locks that cover the I/O region
and then does the I/O.

    DN> As I understand the spec now, if we leave aside special
    DN> stateid's (and non-V4 clients and local access...), share
    DN> locks are inherently mandatory while it is up to the server to
    DN> decide whether byte-range locks have mandatory effect (and the
    DN> server could do it for some files based on mode bits or
    DN> whatever).  Is that everybody else's understanding?

That's my understanding.

    DN> But providing a way a for one client to ignore another
    DN> client's mandatory locks isn't advisory locking.  It's
    DN> something different and kind of scary.

What would you say then is the difference between the current spec and
advisory locking?

    >> Section 8.1.4 should be changed.  A READ with an all-1 stateid
    >> should bypass both record locks and share reservations, and a
    >> WRITE with an all-1 stateid is equivalent to a WRITE with an
    >> all-0 stateid.

    DN> "should bypass" as opposed to "must bypass"!

Right.

Though it just occurred to me that extending the special all-1
stateid to share reservations may not make much of a difference, since
share reservations are primarily enforced on OPEN, not READ/WRITE, and
OPEN doesn't take any magic stateids.

    DN> Generally, the poeple who complain about Windows locks
    DN> stopping them from accessing a file are complaining about
    DN> share reservations rather than record (i.e. byte-range)
    DN> locks.

Well, that, combined with my observation above about OPEN, implies
that the all-1's stateid will be of limited usefulness.

    DN> my preference would be to avoid two different
    DN> ways of doing the same thing.  I would rather allow:

    DN>     Read with all-0: Normal READ with no locks or opens.
    DN>     Subject (of course) to mandatory locking.

    DN>     Write with all-0: Normal WRITE with no locks or opens.
    DN>     Subject (of course) to mandatory locking.

    DN>     Read with all-1: Special READ which bypasses locking
    DN>     checks, and thus an exception to mandatory locking, if the
    DN>     server supports the exception.

    DN> And make invalid:

    DN>     Write with all-1: Invalid since a WRITE which bypasses
    DN>     locking checks is too horrible to contemplate (for long).

That's okay with me.

I'm debating with myself whether to propose any significant changes in
this area (e.g., removal of the magic stateids, or changing OPEN so
that servers may ignore deny=read under some circumstances).  But none
of the changes I've thought of is compellingly superior to the current
spec, so I'm inclined to leave things pretty much as they are.

I will suggest that servers be allowed to reject deny=read OPENs if
the requestor only has read access to the file in the first place.

Also, as long as I'm thinking about share reservations, did the WG
ever discuss the Windows semantics where having a file open keeps the
file from being renamed or deleted?  My reading of RFC 3010 is that v4
share reservations do not keep the file from being renamed or
deleted.  Is that everyone's understanding?

    DN> I think it would help if we added the following text
    DN> at the end of the fifth paragraph of section 8.5.2.

    DN>    READs with a stateid of all-ones, must also return
    DN>    NFS4ERR_GRACE, even though such READs bypass locking
    DN>    checks, unless the server can determine that no reclaims of
    DN>    delegations can possibly occur.
   
Fifth paragraph or fourth paragraph?

I wouldn't bother mentioning the locking bypass here.  I'd just say
something like

    Note that a stateid of all-ones does not provide any special
    exceptions for READ requests.  The server must reject such
    requests with NFS4ERR_GRACE unless it can determine that no
    reclaims of conflicting delegations can possibly occur.

mike