Subject: RE: still more CLID_INUSE

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 19 Jul 2002 08:52:05 -0700

Peter Astrand wrote:
> On Thu, 18 Jul 2002, Spencer Shepler wrote:
> > > > > How does the Solaris or Linux client construct the clientid?
> > > > 
> > > > The Solaris client uses a combination of:
> > > > 	hostname of client
> > > > 	a string identifying the client as the Solaris NFSv4 kernel
> > > > 	the ip address of the server
> > > 
> > > This nfs_client_id4.id looks independent of the IP adress to me. And your
> > > server returns CLID_INUSE if the client IP changes? Then, how do you
> > > change IP of the client, without getting CLID_INUSE?
> > 
> > It doesn't.  The client will receive CLID_INUSE and in the current
> > implementation will error out of the operation (I will be opening a
> > bug on the Solaris code to have it wait a server lease period and
> > retry the operation).  

> If your Solaris server releases the clientid after a lease period, then
> this will work fine.  But according to the current spec, servers are not
> required to do so. 

In fact the spec says (section 8.1.2):

   It should be clear that the server must be very hesitant to release a
   clientid since the resulting work on the client to recover from such
   an event will be the same burden as if the server had failed and
   restarted.  Typically a server would not release a clientid unless
   there had been no activity from that client for many minutes. 

So, while a server that released the clientid after expiration of the
lease period would not be non-conformant, it would not be behaving
in the expected manner either.

I think there is a confusion here with the stuff in section 8.5.3,
which discusses what happens when a lease expires.  The server
has the option of freeing all locks (note that is not locking-related
information in general, which could include the clientid, but the
locks, and thus the associated stateids themselves).  The server 
also has the option of keeping the locks around pending their revocation
in the event that a conflicting lock request appears.  But, again, nothing
is said about the clientid.

There is an idea around (See Issue 131) that you have a lease when
you have a clientid.  You don't.  You only have a lease when you have
one or more locks.  Section 1.2 says:

   Lease     An interval of time defined by the server for which the
             client is irrevocably granted a lock.  At the end of a
             lease period the lock may be revoked if the lease has not
             been extended.  The lock must be revoked if a conflicting
             lock has been granted after the lease interval.

I think this confusion has arisen because of the change to RENEW in which
it now takes clientid rather than a stateid.  The reason for the change
was so that the client did not have a complicated search to find a stateid
to use, but the effect has been to convince people that leases exist
when you have a clientid, which might be valid design, but isn't the
one we have in NFS-v4.

> Servers can keep the clientid forever, which means the
> client would retry and get CLID_INUSE forever.  Or at least until some
> administrator manually clears the clientid, if this is possible with the
> current server.

Since the server is free to release the clientid when there are no locks
(it's a policy matter and spec recommends that the time be a long one),
I would think it would be sensible for the server not to keep around a 
clientid that caused a CLID_INUSE for a very long time.  It should
(I didn't say "SHOULD" but I wouldn't object to that) accelerate the
recylcing of the clientid in those circumstances.

Similarly, since the server is free to release locks upon lease 
expiration, I would say that even if it normally does not do so, it
would be reasonable to stop keeping locks around, if the lease
expriration followed a CLID_INUSE conflict (and also get rid of
them if the conflict happened while they are being kept (revocably)
on sufference).  I would leave these things as implementation 
choices which the spec allows.

> > For the case of a client's use of DHCP and the switching of IP
> > addresses between reboots, I don't think this will occur that often
> > that the client can't handle the CLID_INUSE error as I have described.
> 
> This requires that servers actually releases the clientid.  Even then, it
> means that booting the client will take an extra lease period every time
> the IP has changed.

If there are no locks, the server can get rid of the clientid without
waiting for the lease time.

> I'm still not convinced that the current way of doing SETCLIENTID is
> optimal.  But, since we have much to do, I will stop complaining about it,
> at least until 4.0 is finished :-)

---

From: [email-masked]
Date: Fri, 19 Jul 2002 12:30:24 -0500

>>Peter wrote:

I would agree that SETCLIENTID has some holes that should addressed. I
would prefer to see this get addressed in 4.0 instead of being deffered.
Given this is the foundation upon which NFS V4 state mangement is built, it
seems like it needs to be complete the first time out of the chute.

Independent of when it is handled, shouldn't an item be added to the NFS
issues list for this? Here is my attempt to capture what I think are the
main issues. I hoped I have included the original problem Rick reported.

  SETCLIENTID does not handle the case of two clients
  independently generating the same nfs_client_id4.id value. While the
  spec encourages implementations to create unique values, the potential
  exists in the absence of an RFC dictated algorithm. Unless servers
  further qualify the client's signature with additional SETCLIENTID
  information such as the RPC authentication and/or client network
  information (callback info, reverse binding), the potential for
  indefinite CLID_INUSE errors exists.

  The RPC authentication is probably not a good choice as
  a distinguishing client property. Some environments will use the
  same credential across all the clients to avoid the creation of
  per client principals and the associated administrative overhead. In
  fact I would guess that AUTH_SYS (UID=0) will be the predominant choice.

  If the server chooses to use client network information as a
distinguishing
  feature, then support for multiple callback paths to a client with
  more than one network address is prohibited. This would preclude NFS
  implementations from supporting a desirable availability characteristic.

  If a client does receive repeated CLID_INUSE errors because of
  accidental id collision, the specification does not provide guidance on
  how a client might react. It seems reasonable, that a client could make
  a decision to generate a new id value and retry the SETCLIENTID operation
  after some delay period.

  Ideally, there is probably an opportunity to modify the SETCLIENTID
  "handshaking" mechanism to handle the coincidental occurrence of
   duplicate nfs_client_id4.id values given it is possible.

Carl

Carl Burnett
AIX Kernel Architecture - Distributed File Systems
(512) 838-8498, TL 678-8498
(please reply to [email-masked])

|---------+---------------------------------->
|         |           Peter Astrand          |
|         |           <[email-masked].  |
|         |           se>                    |
|         |           Sent by: owner-nfsv4-  |
|         |           [email-masked] |
|         |                                  |
|         |                                  |
|         |           07/19/2002 03:47 AM    |
|         |                                  |
|---------+---------------------------------->
  >---------------------------------------------------------------------------------------------------------------|
  |                                                                                                               |
  |       To:       [email-masked]                                                                  |
  |       cc:                                                                                                     |
  |       Subject:  Re: still more CLID_INUSE                                                                     |
  |                                                                                                               |
  |                                                                                                               |
  >---------------------------------------------------------------------------------------------------------------|

.
.
.
.

--
/Peter Åstrand <[email-masked]>