Subject: Re: [reiserfs-list] Directory cookie issues

From: Eric Werme USG <[email-masked]>
Date: Mon, 5 Feb 2001 10:05:39 -0500 (EST)

Xuan Baldauf wrote:
>Eric Werme USG wrote:
>> Xuan Baldauf wrote:
>> > Eric Werme USG wrote:
>>
>> > > It would seem to me the cookie would have to be the _next_ filename to read.
...
>> > > Note this presupposes deleting files doesn't change the position of the
>> > > "next file" when directory reading resumes.  I imagine that could be a
>> > > problem with some tree-structured directory designs.
>>
>> > I don't think so. See following direntries:
...

>That is why I say that this is still legal, but why should we miss
>something we are able to hit? This is just about taste, nothing
>crucial. :-)

We are talking about something that will be used for at least the next
decade between OSes that are and OSes to be and with server FSes that
are and FSes to be.  And we are talking about a protocol with an
extremely high cost to change.  IETF procedures, hoops, etc should
not be taken lightly.  While they may seem onerous, they are the
best policies developed to date that keep interoperability and
congestion control in the forefront.

You are suggesting that a directory cookie no longer refer to a file
but to a string that is a hint to the physical file system.  That's a
bit of a conceptual change to NFS that affects more than just one line
in the spec.  It would be fine if there were no client impact to the
definition of what a cookie is.  Perhaps the solution is to further
"opaquify" the definition.  Instead of your (or was it Hans?)
suggestion of "variable length opaque data referring to the last file
returned and mine of "... the next file to be returned", perhaps we
should consider "variable length opaque data that the server's
physical file system can use to continue reading a directory."

That way, UFS and FAT can continue to return the directory offset of
the next file, SGI's XFS can return a tree node whatever-it-is-they-return,
Reiserfs can use search key and start with next filename after match or
mismatch.

>[tree walk description deleted.] This method
>is independent of the sort order used, there just has to be one sort
>order.

Some file systems don't have a sort order in their directories.  NFS
has to to work with them.

	-Ric Werme

---

From: Xuan Baldauf <[email-masked]>
Date: Mon, 05 Feb 2001 18:21:51 +0100

Eric Werme USG wrote:

> but to a string that is a hint to the physical file system.

A directory cookie never referred to a file, AFAIK. (Did it?) If you imagine a listing
of all files of one directory, a cookie is a pointer within this list. Unfortunately,
the cookie apparently was not built with a directory listing which can change over time
in mind. For static directory listings, the current cookie seems to be adequate.

>  That's a

That's the way we meant it. For the NFS client side, the cookie is just variable length
data which the client has to give the server if it expects to continue successfully the
readdir. For the reiserfs specific internal server side, the value of the cookie data
can be the last filename which was returned, or the next one, or something else we do
not yet know.

> Reiserfs can use search key and start with next filename after match or

Yes, this is the spirit of the protocol change we want to achieve.

> has to to work with them.

That's right. At least for our reiserfs-specific implementation of the usage of the
cookie, we use the sort order feature. Other filesystems may find other solutions.
AFAIK FAT (not VFAT) has a fixed length for every directory entry, so FAT might want to
use a fixed offset. (As long as FAT is not repacking their directory data structures.)
You can generally divide filesystems into two types regarding this problem. The
filesystems which never repack their directory data, just overwrite, invalidate and
append data chunks (so that a direntry, once written, never changes its position), and
filesystems which do repack their directory data (insert data, delete data), so that
directory entries may change their positions. ReiserFS belongs to the latter ones.
Maybe there should be a query to all maintainers of filesystems of the latter type,
what their solution to this problem could be. Maybe the variable length cookie
suffices, maybe not. Does the Phase T(h)ree filesystem belong to the latter type?

>         -Ric Werme

Xuân. :-)

P.S. The mail to you directly seems to be undeliverable:

   ----- The following addresses had transient non-fatal errors -----
<[email-masked]>

   ----- Transcript of session follows -----
<[email-masked]>... Deferred: Connection timed out with zmamail01.zma.compaq.com.
Warning: message still undelivered after 4 hours
Will keep trying until message is 5 days old

---

From: Hans Reiser <[email-masked]>
Date: Mon, 05 Feb 2001 21:49:24 +0300

> Reiserfs can use search key and start with next filename after match or

We agree on this, variable length cookies that happen to be able to be at least
the size of a maximum filename would be great.