Subject: Re: [nfsv4] detailed NOTIFY_DEVICEID4_CHANGE semantics?

From: Christoph Hellwig <[email-masked]>
Date: Mon, 1 Sep 2014 20:22:31 +0200

On Mon, Sep 01, 2014 at 09:51:56AM +0300, Benny Halevy wrote:
> The new device info applies to the current layout so I see no reason to wait
> with a GETDEVICEINFO until the client gets new layouts as the server is
> retirin the old mapping in the background.

At least for the !immediate case I think we very much require the
new device Id to device mapping to be a strict superset of the old one.

Talking in block layout terms this is what I allow for now (and plan
to write down as an informative errata for RFC5663 in nicer words):

 - extending a SLICE volume to a larger size
 - implicitly extending a SIMPLE volume (there is no size in it, so the
   client has to rely on the storage device defined size)
 - extending the last member in a CONCAT volume
 - adding new members to the end of a CONCAT volume
 - extending all members of a STRIPE volume by the same amount

all other changes would change mappings for existing blocks and easily
cause data corruption, so in that cases a NOTIFY_DEVICEID4_DELETE
should be issued instead to make sure all layouts are recalled first,
and the clients stop using the deviceid.

> To prevent needless I/O errors
> caused by using a stale mapping I'd suggest the following instead:
> 
>   The client may finish any outstanding I/Os that reference the previously provided
>   device-ID-to-device-address mapping and SHOULD use GETDEVICEINFO to obtain the
>   updated mapping before issuing new I/Os that reference the device ID via either
>   outstanding layouts or layouts acquired after receiving the deviceid change
>   notification.

This wording is fine with me, and actually matches what I have done for the
client, but it seems stricter than nessecary in general.

Any idea how to make sense of the ndc_immediate = TRUE case?

---

From: Benny Halevy <[email-masked]>
Date: Tue, 02 Sep 2014 16:22:06 +0300

> Any idea how to make sense of the ndc_immediate = TRUE case?

Not really.  Other than trying to abort in-flight I/Os I don't see
how the client would want to act any different

Benny

---

From: Christoph Hellwig <[email-masked]>
Date: Wed, 3 Sep 2014 17:50:57 +0200

> client, but it seems stricter than nessecary in general.

While preparing an errara I went back to my original language.  The whole
point of the change is to be less disruptive than delete, so requriting
clients to invalidate their layouts seems harmful, and against the spirit
of the original language.  But I'm still open to be convinced otherwise.

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
---end quoted text---

---

From: Benny Halevy <[email-masked]>
Date: Wed, 03 Sep 2014 19:14:54 +0300

> of the original language.  But I'm still open to be convinced otherwise.

Why do you equate invalidating the deviceid mapping to invalidating layouts?
The client can keep the layouts and refresh the deviceid mapping.
In particular this is useful for updating only the addressing / networking
information, e.g. change of target ip address or LUN # in the blocks case.

Benny

---

From: Christoph Hellwig <[email-masked]>
Date: Wed, 3 Sep 2014 18:21:21 +0200

> information, e.g. change of target ip address or LUN # in the blocks case.

This is the sort of feedback I've een looking for.  target IP change makes
sense in the object and probably files case.  For the block case where
devices are addressed by content (signature) this case simply doesn't exist
at the pNFS level. (Although most OSes will create horrible problems for you in
the storage stack if you change the LUN # number..)

So fom the spec point of view allowing device-to-device-id-mappings to
change on a live layouts sounds fine to me, although not nessecarily easy to
implement in practice.