Subject: Re: Directory cookies (a brief summary)

From: Jim Rees <[email-masked]>
Date: Wed, 07 Feb 2001 09:25:57 -0500

Sure would be nice if we could require the client to fetch and cache whole
directories.  That would solve the cookie problem and speed things up
considerably.  But I don't think that's going to happen.

---

From: "Corbett, Peter" <[email-masked]>
Date: Wed, 7 Feb 2001 10:37:40 -0800

Here is Mike's proposed structure, again:

const NFS4_NAMES_ARE_COOKIES 0xffffffffffffffff; /* 2^64 -1 */

union cookie_switch4 switch (nfs_cookie4 cookie) {
 case NFS4_NAMES_ARE_COOKIES:
         component4     name;
 default:
         void;
};

struct READDIR4args {
        /* CURRENT_FH: directory */
        verifier4       cookieverf;
        count4          dircount;
        count4          maxcount;
        cookieswitch4   locator;
        bitmap4         attr_request;
};

In the server's response message, you would not want to use this
cookie_switch4 structure, as it again includes the component4 utf8string,
which we don't want to have to put into the response packet twice in each
entry4.  So, I think the intention was to replace the nfs_cookie4 in the
request with the cookie_switch4 structure, but in each entry4 of the
response, to just include the nfs_cookie4 as it is, and use it as an
indicator that the component4 name portion of the entry4 is to be used as
the cookie, without repeating the component4 name in each entry4 structure.

However, that's an extra 8 bytes per entry4, if the cookie is the component
name, so another alternative would be, for the result structures:

struct entry4 {
	nfs_cookie4	cookie;
	component4	name;
	fattr4	attrs;
	entry4	*nextentry;
};

struct entry_no_cookie4 {
	component4	name;
	fattr4	attrs;
	entry_no_cookie4 *nextentry;
};

union entry_switch4 switch ( bool names_are_cookies ) {
  case TRUE:
	struct entry_no_cookie4	entry1;
  default:
	struct entry4		entry; 
};

struct dirlist4 {
	entry_switch4	*entries;
	bool 			eof;
};

with the requirement that the client MUST copy the last returned component4
name field to the discriminated union name field of the cookie_switch4 in
its next READDIR request, if the names_are_cookies bool is true.

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]]
Sent: Wednesday, February 07, 2001 12:07 PM
To: [email-masked]
Cc: [email-masked]; [email-masked];
[email-masked]
Subject: Re: Directory cookies (a brief summary)

> A lot of discussion about directory cookies.  We have a proposal on
> the table to change the nfs_cookie4 definition from an uint64 to
> an opaque <>.

I don't think there's much support for that idea; it would double the
length of readdir results in the reiserfs case, and even worse, with
redundant data. Your summary neglects to mention my counter proposal to
define a special cookie value that tells the client that the component
name is the real cookie, and changes the readdir arguments to uses the
name or the integer cookie based on a discriminated union that uses the
special cookie value as the discriminator.

> The main issue is with the NFS client's ability to interact
> appropriately with the APIs on the client.  There has been a
> suggestion that the client may be able to implement a mechanism that
> would provide appropriate behavior for the telldir()/lseek()/seekdir()
> APIs but this claim seems a little weak at the moment.

As has been explained, the issue with reiserfs exists on whether the
application is using the API on a local reiserfs directory, or an NFS
mounted reiserfs directory.

> So at this point, I don't see a consensus as to changing the directory
> cookie unless we have some implementation experience with an
> appropriate solution for the client.

I agree we need implementation experience, and suggest that in absence
of a better one, my proposal be prototyped before modifying the
protocol.

	-mre

---

From: Mike Eisler <[email-masked]>
Date: Wed, 7 Feb 2001 10:58:05 -0800 (PST)

>  case NFS4_NAMES_ARE_COOKIES:

BTW, I just realized that XDR doesn't allow discriminators to be 64
bits, so the above would have to be played with, or the XDR
language extended. It probably wouldn't take more than few minutes
to modify rpcgen to do the right thing, and such an extension does not
break backward compatibility.

> the cookie, without repeating the component4 name in each entry4 structure.

Yes, that's the intent.

> its next READDIR request, if the names_are_cookies bool is true.

The above data structure replaces an 8 byte cookie with a 4 byte
boolean, resulting in a net savings of 4 bytes per entry if
names_are_cookies is true, because the 4 byte discriminaot needs to be
encoded. If names_are_cookies is false, then it ADDS 4 more bytes.

Modulo the issue with the 64 bit discriminator in the arguments, I
think my proposal is still the best (or perhaps the least evil) of the
now three proposed.

	-mre

---

From: "Corbett, Peter" <[email-masked]>
Date: Wed, 7 Feb 2001 11:20:03 -0800

Actually, i'm not really advocating the structure i presented below, as it
is probably too big a change to the protocol.  I was just trying to make the
point that it is possible to replace a discriminator per entry with a
discriminator per response, which should add up to quite a saving, albeit
only in the reiserfs case.  The structure I sent was not defined properly.
What I should have sent is the following:

struct entry4 {
	nfs_cookie4	cookie;
	component4	name;
	fattr4	attrs;
	entry4	*nextentry;
};

struct dirlist4 {
	union entries_switch4 switch (bool names_are_cookies) {
		case TRUE:
			entry_no_cookie4	*entries_nc;
		case	FALSE:
			entry4		*entries;
	};
	bool			eof;
};

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]]
Sent: Wednesday, February 07, 2001 1:58 PM
To: Corbett, Peter
Cc: 'Mike Eisler'; [email-masked]; [email-masked];
[email-masked]; [email-masked]
Subject: RE: Directory cookies (a brief summary)

Yes, that's the intent.

	-mre

---

From: Mike Eisler <[email-masked]>
Date: Wed, 7 Feb 2001 16:57:32 -0800 (PST)

>From: Neil Brown <[email-masked]>

>I"m pretty much in agreement... but thinking about the issues of
>implementing the client side of name-based-cookies, it doesn't seem to
>be fundamentally different from support 64bit cookies on a system with
>a 32bit offset type (off_t).

It's a bit different in that with reisefs there's no POSIX API for
doing a seekdir with true reiserfs cookies, whereas with LP64 POSIX,
the API deals with 64 bit cookies (assuming the filesystem is
conventional) just fine. The hack I suggest tells the NFSv4 client
unambiguously that its ok to make a half hearted attempt (or no attempt
at all) to emulate random seekdir.
 
>Trond has told us about his attempts to manage 64bit cookies in
>Linux.  Is anyone else implementing an NFSv4, or even NFSv3, client in
>a 32bit API?  How do other people handle 64bit cookies?  Would there
>be a significant difference using variable length cookies?

As far as I know, most NFSv3 client implementations, when doing
a readdir on behalf of an ILP32 process, return an error if the
upper 32 bits of the cookie are non-zero.

	-mre

---

From: Mike Eisler <[email-masked]>
Date: Sat, 10 Feb 2001 08:06:22 -0800 (PST)

> 
> I have been skiing for the last week and came back to this pile of

> My question is why pass filenames as cookies when you could pass
> fileids and translate them back to names on the server?

Hard links make this hard to do.

	-mre