Subject: idea - new cache control operation

From: "Wittle, Mark" <[email-masked]>
Date: Thu, 15 Mar 2001 20:12:00 -0800

SUMMARY
    Provides the server with cache management hints.

#define	CACHE_HINT_NORMAL 	0
#define	CACHE_HINT_RANDOM		1
#define	CACHE_HINT_SEQUENTIAL	2
#define	CACHE_HINT_WILLNEED	3
#define	CACHE_HINT_DONTNEED	4

ARGUMENTS
struct CacheHint_Args
    {
    filehandle_type	filehandle;
    uint64			offset;
    uint32			length;
    uint32			hint;		/* From defines above */
    };

RESULTS
    None.

DESCRIPTION
The Server Caching Hint mechanism is similar to the POSIX
madvise(3) functionality.  In the NFS context, it provides
the server with cache management hints that indicate the
client's intended usage for specific file offset ranges. 
The operations provide for SEQUENTIAL, RANDOM, WILLNEED,
and DONTNEED flags, along with a NORMAL flag that indicates
that previous hints from the client should be discarded.
The server is free to make use of these hints in a manner
that will optimize its caching and I/O performance.  The
hints provided by the client are:

CACHE_HINT_NORMAL		No special treatment is
				indicated for the file region.

CACHE_HINT_RANDOM		Client expects future file
				accesses to this region to be
				random.

CACHE_HINT_SEQUENTIAL	Client expects future file
				accesses to this region to be
				sequential.

CACHE_HINT_WILLNEED	Client indicates with high
				degree of certainty that a file
				region will be needed in the
				future.

CACHE_HINT_DONTNEED	Client indicates with high
				degree of certainty that a file
				region will no longer be needed.

IMPLEMENTATION

The protocol does not dictate what actions a server must take
upon reception of cache hints or even if the server is
required to take any actions at all.  The protocol does not
dictate how long a server must make use of a hint issued by
the client.  It is viewed as "good-citizenship" for the client
to turn-off any hints it may have issued by eventually issuing
a CACHE_HINT_NORMAL. This is, however, not required.

RATIONALE

Giving the server this type of hint information can prove
incredibly useful for the server's buffer cache management
and predictive algorithms that support read-ahead from its
stable store.  For example, specifying RANDOM means the
filer might as well turn off its read ahead. WILLNEED lets
the client control read-ahead to the disk subsystem.
The fact that they are hints allows the server to benefit
if it can, without any additional resource commitment.   The
protocol does not dictate what actions a server must take
upon reception of cache hints or even if the server is
required to take any actions at all.

---

From: "Matthew J. Zito" <[email-masked]>
Date: Fri, 16 Mar 2001 18:07:14 -0500

>required to take any actions at all.

This sounds very good, except that server-side implementations would 
need to guard against "bad-citizen" clients that would claim behavior 
other than that expected.  For example, a client could request 
read-ahead behavior on a very large byte range on a file, in the 
hopes that it would be able to randomly access within that range more 
quickly by drawing it all into cache.

Related to this, what about cache priority functionality?  On heavily 
loaded servers, it would be very nice to be able to say that a 
readdir() call by a non-time-sensitive application could get lower 
priority, both in scheduling and in cache, than the highly critical 
database process on a different server that has to complete a write 
in a very limited timeframe.  The priority could be assigned upon the 
open() call.  The server would be under no obligation to honor the 
priority at all.

Thanks,
Matt

-- 
Matthew J. Zito
Systems Engineer
Register.com, Inc., 11th Floor, 575 8th Avenue, New York, NY 10018
Ph: 212-798-9205
PGP Key Fingerprint: 4E AC E1 0B BE DD 7D BC  D2 06 B2 B0 BF 55 68 99

---

From: Brent <[email-masked]>
Date: Fri, 16 Mar 2001 17:15:48 -0800

> open() call.  The server would be under no obligation to honor the

Sounds like a feature asking to be abused.  I think QoS is best
handled outside of the protocol, e.g. "give priority to 
these clients" or "give priority to these shared filesystems".

A MIB feature ?

	Brent

---

From: "Matthew J. Zito" <[email-masked]>
Date: Fri, 16 Mar 2001 21:53:13 -0500

>these clients" or "give priority to these shared filesystems".

The potential for abuse can be mitigated a great deal by inserting in 
the MIB an option to disable priority scheduling for certain clients. 
That way, clients or applications that abuse any priority scheduling 
available can be disabled, and thereby learn their lesson - or at 
least be removed as a problem.

The trouble with putting the whole feature in the MIB is the loss of 
granularity - oftentimes, its file-level operations that you want to 
be prioritized.  Putting the prioritizing at the protocol level lets 
us fine-tune the scheduling.

MIB-level priority is, of course, better than nothing, but I'm 
interested in whether anyone in this group can think of a more 
functional way to implement this.

Thanks,
Matt