Subject: RE: [nfsv4] New pNFS ID available

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 1 Aug 2005 13:48:54 -0400

Marc Eshel wrote:
> It will take a long time to use the 32 bit name space but it might be
a 
> problem in the future where the server is up for a long time and just 
> replacing or recovering nodes. 

I'm having trouble imaginging a situation in which the total set of 
nodes that we ever put in layouts is over 4 billion, but the actual
set of nodes that we access at any one time is much much smaller
(i.e. less than one billion, say).  In other words, to me, running
out of the 32-bit space is an indication, not that we need to conserve
devid'sby recycling them but that the 32-bit space is, at least 
potentially, too small.

Given that analysis, I would say that if we think we are going to run
out of the 32-bit space, let's just make this 64-bit.  After all, the
motivation for devid's as a shorthand was primarily that the full thing
that the shorhand represents was very large (order of 1K) in the blocks
case.  With a 64-bit devid, we still have something 99.2% smaller
than that, rather than something 99.6% smaller.

> Another less important reason is an 
> implementation one. It would be nice to keep the numbers in a low
range so 
> they can be used as an index to the device list table.

That's on the server.  On the client, he can't depend on a restricted
range so he has to code for the full range anyway.  On the server, the
advantage of the restricted range is fairly small since I would imagine
that most pieces of the server would use an internal identifier anyway
(e.g.. the address of the data structure) and you fetch the devid to 
put it in the layout before sending it.  On the client, you probably
take the layout you receive and convert the devid's to a structure
address.  So you will have to use a hash for this but you do it only
when you recieve the layout, but you will have to do a hash anyway. 

> At the minimum we 
> can say that when we recall all layouts associated with an FSID we
expect 
> the client to refresh the device list.

This is a possibility but I think that the simplicity benefit of saying 
that that a devid identifies something that never changes (during the
life
of the server) is significant and I'd like to maintain that.

-----Original Message-----
From: Goodson, Garth 
Sent: Friday, July 22, 2005 9:34 PM
To: Marc Eshel
Cc: [email-masked]
Subject: Re: [nfsv4] New pNFS ID available

Marc Eshel wrote:
> Garth Goodson <[email-masked]> wrote on 07/22/2005 05:33:06
PM:
> 
> 
>>Marc Eshel wrote:
>>
>>>> If data are reorganized from a storage
>>>> device with a given device ID to a different storage device (i.e., 
> 
> if
> 
>>>> the mapping between storage device and data changes), the layout
>>>> describing the data MUST be recalled rather than assigning the new
>>>> storage device to the old device ID.
>>>
>>>
>>>If the device ID changes we can recall all layout that include the 
> 
> changed 
> 
>>>id but we also need to update the device list that the client cached.

> 
> How 
> 
>>>do we do it? can we add a call to invalidate the device list on the 
>>>client?
>>>Marc. 
>>
>>The goal is that device IDs aren't reused between metadata server 
>>reboots.  With a 32 bit device ID name space, do you think this will 
>>really be an issue.  I would prefer not adding more complexity to
handle 
> 
> 
>>the invalidation of device IDs unless it is absolutely necessary.
>>
>>The client may have old device IDs cached, but it can invalidate it's 
>>cache every so often.
>>
>>-Garth
> 
> 
> Is there a reason why you did not cc the list?
> 
> It will take a long time to use the 32 bit name space but it might be
a 
> problem in the future where the server is up for a long time and just 
> replacing or recovering nodes. Another less important reason is an 
> implementation one. It would be nice to keep the numbers in a low
range so 
> they can be used as an index to the device list table. At the minimum
we 
> can say that when we recall all layouts associated with an FSID we
expect 
> the client to refresh the device list.
> Marc. 

Sorry, forgot to hit reply to all.  I think this something we can 
discuss more.  Your proposal is reasonable; let's see what others think.

-Garth

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Marc Eshel <[email-masked]>
Date: Mon, 1 Aug 2005 12:24:26 -0700

I still think that is reasonable to allow the server to invalidate the 
device list cache on the client so it can reorganize it device name space 
and not force it to use new ids just to avoid this SIMPLE call to the 
client. Like I said it can even be implied by the existing recall all 
layouts for a given fsid.
Marc.