Subject: RE: Issue 99 (negative LOOKUP caching)

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 18 Jul 2002 13:58:55 -0700

I do have a problem with prohibiting negative name caching.  It's
a useful technique in many situations, which has been used in
previous versions of NFS without problems.  If there is some
reason to prohibit it, then OK.  But what is it?

My memory on this point is weak but I recall that the argument
was made that in a protocol which had OPEN's and required you
to do OPEN's to the server, as opposed to V3 which you could
just start reading if the LOOKUP was cached, it followed that
all OPEN attempts should go to the server rather than being
short-circuited on account of a negative LOOKUP cache.
I'm not sure I fully buy that argument but that's what I
understood people were agreeing to.

I don't see where this general prohibition of negative name
caching comes from and why a client should be prohibited from 
using it to resolve a stat call, for instance.

-----Original Message-----
From: Mike Kupfer [mailto:[email-masked]]
Sent: Thursday, July 18, 2002 2:52 PM
To: Noveck, Dave
Cc: [email-masked]'
Subject: Issue 99 (negative LOOKUP caching)

>>>>> "Dave" == Dave Noveck <Noveck> writes:

    Dave> Issue 99: Section 9.7 says not to do negative LOOKUP
    Dave> caching. But if we're doing READDIR caching, shouldn't it be
    Dave> okay to have a negative LOOKUP cache entry, as long as it
    Dave> expires when the READDIR cache expires?

    Dave>      I think the intention was for OPEN's.  In the case of
    Dave>      positive name caching, it is clear you can't use this
    Dave>      for OPEN's, but for non-existence you might be tempted
    Dave>      to avoid the OPEN attempt and I don't think that would
    Dave>      be right.  Anyone else have opinions about this?

I don't have a problem with prohibiting negative caching, but I think
the spec is ambiguous and could be misinterpreted.

The start of Section 9.7 says

   The results of LOOKUP and READDIR operations may be cached to avoid
   the cost of subsequent LOOKUP operations.

That could be taken to mean that the READDIR cache can be used to
avoid LOOKUP calls that the client "knows" will fail.

I suppose that this sentence, combined with the first bullet item,
implies that if the entry is not in the READDIR cache, the client
should issue a LOOKUP anyway.  But I'd rather have the spec say that
explicitly, rather than forcing the client implementor to notice the
implication.

mike

---

From: [email-masked]
Date: Fri, 19 Jul 2002 16:22:02 -0700

>>>>> "Dave" == Dave Noveck <Noveck> writes:

    Dave> I do have a problem with prohibiting negative name caching.
    Dave> It's a useful technique in many situations, which has been
    Dave> used in previous versions of NFS without problems.  If there
    Dave> is some reason to prohibit it, then OK.  But what is it?

Beats me.  I don't have an opinion one way or the other about negative
name caching. :-)

mike

---

From: Eric Werme USG <[email-masked]>
Date: Mon, 22 Jul 2002 10:00:19 -0400

I do!

Our fine, high quality(?) build process uses very long search paths for
executable and header files.  Most of these refer to NFS files.  When
compiling nfs_common.c, a rather small file as there isn't much shared
between client and server, I see:

Negative name caching on:

Client nfs V3: (341 out of 341 calls)
       null    getattr    setattr     lookup     access   readlink       read
     0   0%   316  92%     0   0%    23   6%     0   0%     0   0%     0   0%
      write     create      mkdir    symlink      mknod     remove      rmdir
     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%
     rename       link    readdir   readdir+     fsstat     fsinfo   pathconf
     0   0%     0   0%     2   0%     0   0%     0   0%     0   0%     0   0%
     commit
     0   0%

Negative name caching off:

Client nfs V3: (363 out of 363 calls)
       null    getattr    setattr     lookup     access   readlink       read
     0   0%   263  72%     0   0%    98  26%     0   0%     0   0%     0   0%
      write     create      mkdir    symlink      mknod     remove      rmdir
     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%
     rename       link    readdir   readdir+     fsstat     fsinfo   pathconf
     0   0%     0   0%     2   0%     0   0%     0   0%     0   0%     0   0%
     commit
     0   0%

caching saves 75% of the lookups of files that aren't there and never will
be there.  It looks like the increase in GETATTRs is because I let the
attribute cache time out.  If I compile twice in quick succession, the
attribute cache suppresses all the lookups:

Negative name caching on, attribute cache in good shape:

Client nfs V3: (265 out of 265 calls)
       null    getattr    setattr     lookup     access   readlink       read
     0   0%   263  99%     0   0%     0   0%     0   0%     0   0%     0   0%
      write     create      mkdir    symlink      mknod     remove      rmdir
     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%     0   0%
     rename       link    readdir   readdir+     fsstat     fsinfo   pathconf
     0   0%     0   0%     2   0%     0   0%     0   0%     0   0%     0   0%
     commit
     0   0%

I presume if I disable close-to-open consistency most of the getattrs
would disappear too.

The point of all this is that without a negative name cache, there are
situations where the client will issue lookups that failed dozens of
time in the last few seconds that any human would say should never have
been issued.

Given our interest in making NFS V4 work well in high latency environments,
the implemention notes should recommend that clients have a negative name
cache.

Also, note that our kernel builds are a good example of why keeping
delegations after close(2) is a good idea.  If the build process had
read delegations for all the executables and header files, all those
getattrs could go away and close-to-open consistency would be maintained
across multiple invocations of compilers, reads of header files, etc.

Caveat: If someone wanted to check access times on the files, the client
could either force the getattr or release the delegation.  Problem is,
Unix clients can't tell if the user is interested in that....

	-Ric Werme