Subject: [nfsv4] proposed RFC5661 errata: deviceid lifetimes

From: Christoph Hellwig <[email-masked]>
Date: Wed, 3 Sep 2014 18:09:05 +0200

Type: Technical

Section 12.2.10 says:

   A device ID lives as long as there is a layout referring to the
   device ID.  If there are no layouts referring to the device ID, the
   server is free to delete the device ID any time.

It should say:

   A device ID is established by referencing it in the result of a
   GETDEVICELIST or LAYOUTGET operation and can be deleted by the server
   as soon as there are no layouts referring to the device ID.

   If both the client and server support notifications for device ID
   mappings, the server SHOULD send CB_NOTIFY_DEVICEID notifications for
   device ID deletions or changes to the device-ID-to-device-address
   mappings irrespective to the client as soon as the client has used the
   device I once, even if no layouts currently refer to it.

Notes:

   The lifetime rules in RFC5661 are contradictory - both GETDEVICELIST
   and CB_NOTIFY_DEVICEID (NOTIFY4_DEVICEID_DELETE) operations imply
   that device IDs are valid even without layouts referring to them.
   Common implementations rely on this fact by caching not referenced
   device IDs to avoid the huge setup costs, and thus require notifications
   to be sent for that case.

---

From: Tom Haynes <[email-masked]>
Date: Wed, 3 Sep 2014 12:16:26 -0400

>   device I once, even if no layouts currently refer to it.

^^ what does "device I once” mean?

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Christoph Hellwig <[email-masked]>
Date: Wed, 3 Sep 2014 18:17:39 +0200

> ^^ what does "device I once” mean?

Sorry, this should read:

device _ID_ once.

---

From: Trond Myklebust <[email-masked]>
Date: Wed, 3 Sep 2014 13:02:47 -0400

Hi Christoph,

>    device I once, even if no layouts currently refer to it.

Does the above paragraph imply that the client no longer needs to
subscribe to device notifications through the GETDEVICEINFO operation?
It seems to talk only about "support" and not about whether or not the
client actually issued a requested for notifications.

>    to be sent for that case.

GETDEVICELIST has always been a wart because it is completely divorced
from any layout operations. IIRC, it was added in order to allow
blocks clients to try to discover the devices at boot time. The
notification semantics were added to GETDEVICEINFO, which is supposed
to be called after a layout has been granted to the client.

In retrospect, we should probably have also added the ability to
request notifications through GETDEVICELIST, as it is pretty much a
useless operation without them.

---

From: Christoph Hellwig <[email-masked]>
Date: Wed, 3 Sep 2014 19:22:44 +0200

> client actually issued a requested for notifications.

Good catch.  The opt in at GETDEVICEINFO is not very nice from
the coherency perspective, but I think we can live with it and work around
that issue.  While I would like the notifications to be sent unconditionally
that seems like a too large change for an errata update.

> useless operation without them.

True.  So would have to have to keep track of and outstanding device ID that
was returned from GETDEVICELIST but not actually used in a LAYOUT and not
perform operations that would lead to a NOTIFY4_DEVICEID_CHANGE callback.
Eww.

I've actually removed GETDEVICELIST support from my block server for other
reasons and would be happy to just deprecated it in the strongest possible
way the errata and minor versioning (for 4.2) conventions allow.

---

From: David Noveck <[email-masked]>
Date: Thu, 4 Sep 2014 13:53:58 -0400

>  device I once, even if no layouts currently refer to it.

I had trouble understanding the paragraph above given where 'irrespective'
was
placed.

Would the following be consistent with your intention;?

If both the client and server support notifications for device ID
mappings, the server SHOULD send CB_NOTIFY_DEVICEID notifications for
device ID deletions or changes to the device-ID-to-device-address
mappings to any client which has used the device-ID in question
at least once, irrespective of whether the client has any layouts
currently referring to it.

---

From: Christoph Hellwig <[email-masked]>
Date: Thu, 4 Sep 2014 20:16:41 +0200

> currently referring to it.

That is consistent with my intention, but as pointed out by Trond
the client actually needs to request the notifications and not just
support them, so it should be something like the following:

If the client requested notifications for device ID mappings, the server
SHOULD send CB_NOTIFY_DEVICEID notifications for device ID deletions or
changes to the device-ID-to-device-address mappings to any client which
has used the device-ID in question at least once, irrespective of whether
the client has any layouts currently referring to it.

This still doesn't address the problem with a device-id that has been
returned from GETDEVICELIST but never had an outstanding layout on it.
The answer to that increasingly seems to be "don't implement GETDEVICELIST",
but I need to find a good wording suitable for an RFC errata for that.

---

From: Tom Haynes <[email-masked]>
Date: Fri, 5 Sep 2014 15:38:28 -0400

> way the errata and minor versioning (for 4.2) conventions allow.

^^

So, while we wait on RPCSEC_GSSv3, is there some text you want to propose for NFSv4.2?

---

From: Christoph Hellwig <[email-masked]>
Date: Mon, 8 Sep 2014 01:37:46 +0200

Type: Technical

Section 12.2.10 says:

It should say:

   Given that GETDEVICELIST does not support requesting notifications a
   server that implements GETDEVICELIST MUST not not advertise support
   for NOTIFY_DEVICEID4_CHANGE notification in GETDEVICEINFO, and client
   using GETDEVICELIST must not rely on NOTIFY_DEVICEID4_CHANGE or
   NOTIFY_DEVICEID4_DELETE notifications to work reliably.

Notes: