Subject: RE: [nfsv4] Re: Seperation of generic pNFS and layout encoding/de coding

From: [email-masked]
Date: Wed, 23 Nov 2005 12:16:15 -0500

> As far as blocks go, I think there is a single address type regardless
> of FC, IP, etc, (using volume names).

Yes, and I'm generally opposed to putting in actual device addresses
for the block/volume world because:

(1) It's an opportunity to get device multipathing spectacularly wrong.
There are a number of subtleties in device multipathing that have been
worked out in the SCSI I/O stacks and other areas - it is best not to
dive back into this tarpit.

(2) Addresses can be inconsistent/incomplete - two storage initiators
looking at the same LUN at the same target may see different LUs (storage
volumes); this is considered a "feature" in block/volume storage.  Using
volume content matching avoids problems caused by the pNFS server
having an incorrect view of storage configuration.

I could live with device addresses as a "hint" of some sort, but there
will be explicit words in the block draft that any such address "hint"
MUST be verified, and MUST NOT exclude the possible use of other
addresses by the client for multipathing.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of Garth Goodson
> Sent: Wednesday, November 23, 2005 11:41 AM
> To: Halevy, Benny
> Cc: William A.(Andy) Adamson; Noveck, Dave; Blackberry Admin; 
> 'Garth Goodson'; Welch, Brent; Black, David; [email-masked]
> Subject: Re: [nfsv4] Re: Seperation of generic pNFS and 
> layout encoding/de coding
> 
> Some of this issue has been brought up before (i.e., whether we want a 
> separate dev addr type or just use the layout type).  As far as blocks 
> go, I think there is a single address type regardless of FC, IP, etc, 
> (using volume names).  And I believe you guys have defined a single dev 
> addr structure for objects.  I don't mind changing it to be  switched on 
> a dev addr type, but currently I don't see a compelling example.
> 
> The r_netid follows the same definition as it does in the NFSv4 rfc.  I 
> believe it just encodes the IP address.
> 
> -Garth
> 
> Halevy, Benny wrote:
> >>I'd vote for adding a pnfs_devaddrtype4 field that could hold, for
example,
> >>IP, FC, or IB.
> > 
> > 
> > Err, or is this captured in pnfs_netaddr4.r_netid?
> > 
> > looking at the draft again, I also saw this comment:
> >    [NOTE: other device addresses will be added as the respective
> >    specifications mature.  It has been suggested that a separate
> >    device_type enumeration is used as a switch to the pnfs_deviceaddr4
> >    structure (e.g., if multiple types of addresses exist for the same
> >    layout type).  Until such a time as a real case is made and the
> >    respective layout types have matured, the device address structure
> >    will be left as is.]
> > 
> > so how is pnfs_netaddr4 used?  can we give an example or two to
> > make it clearer?
> > 
> > Benny
> > 
> > 
> >>-----Original Message-----
> >>From: [email-masked] 
> >>[mailto:[email-masked]]On Behalf Of
> >>Halevy, Benny
> >>Sent: Wednesday, November 23, 2005 2:56 PM
> >>To: 'Garth Goodson'; William A.(Andy) Adamson
> >>Cc: Welch, Brent; [email-masked]; Noveck, Dave
> >>Subject: RE: [nfsv4] Re: Seperation of generic pNFS and layout
> >>encoding/de coding
> >>
> >>
> >>On November 23, 2005 1:30 AM, Garth Goodson wrote:
> >>
> >>>Here are my proposed changes based on Andy's suggestion.  Some 
> >>>differences include: Keeping a separate pnfs_deviceaddr4 structure as
it 
> >>>is used in both GETDEVICELIST and GETDEVICEINFO; Merging the layout 
> >>>structure with the opaque portion; Keeping separate layout_update and
> >>>layout_hint structures (for cleanliness).
> >>>
> >>>If there are no disagreements I will send to Spencer for inclusion.
> >>>
> >>>-Garth
> >>>
> >>>Proposed changes:
> >>>-----------------
> >>>
> >>>6.3  pnfs_deviceaddr4
> >>>
> >>>      struct pnfs_netaddr4 {
> >>>               string           r_netid<>;   /* network ID */
> >>>               string           r_addr<>;    /* universal address */
> >>>      };
> >>>
> >>>      struct pnfs_deviceaddr4 {
> >>>               pnfs_layouttype4 type;
> >>>               opaque           device_addr<>;
> >>>      };
> >>>
> >>>    The device address is used to set up a communication channel with
the
> >>>    storage device.  Different layout types will require different
types
> >>>    of structures to define how they communicate with storage devices.
> >>
> >>>    The opaque device_addr field must be interpreted based on the
> >>>    specified layout type.
> >>
> >>Does this mean we'll need different layout types for blocks/objects
> >>over IP, FC, or IB?  Although the interconnect media might be different
> >>I'm not sure that the layout itself will be any different with respect
> >>to mapping bytes in the file to bytes on storage once you know how
> >>to name the device and the logical unit inside it.
> >>
> >>I'd vote for adding a pnfs_devaddrtype4 field that could 
> >>hold, for example, IP, FC, or IB.
> >>
> >>Benny
> >>
> >>_______________________________________________
> >>nfsv4 mailing list
> >>[email-masked]
> >>https://www1.ietf.org/mailman/listinfo/nfsv4
> >>
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nfsv4
> 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Brent Welch <[email-masked]>
Date: Wed, 23 Nov 2005 10:02:03 -0800

Yeah, we are converging toward this in the object draft, in which
we ultimately identify the OSD by a value of one of the standard
attributes on the root object (e.g., the serial number or system ID).
Any addressing information will be a hint that the client can use
to help guide their device discovery process.   That way you can
support an OSD that is reachable by both iSCSI and FC.

>>>[email-masked] said:
 > 

--
Brent Welch
Software Architect, Panasas Inc
Accelerating Time to Results(tm) with Clustered Storage

www.panasas.com
[email-masked]