Subject: Re: [storm] I-D Action: draft-ietf-storm-rdmap-ext-01.txt

From: <[email-masked]>
Date: Mon, 12 Sep 2011 14:11:57 -0400

In looking at the changes in this draft from the -00 version, I noticed something that needs attention, although it was already present in the -00 version:

The term "naturally aligned" needs to be defined for buffer addresses, because failure to use a "naturally aligned" buffer address causes a "Catastrophic Error" at the recipient.  For this reason, there needs to be a tight specification of what the sender needs to do to avoid that error.  Specifying a boundary that is always "naturally aligned" (e.g., 64-bit) should suffice (and is probably superior to providing a way to figure this out on-the-wire).

Thanks,
--David

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Monday, July 11, 2011 4:17 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [storm] I-D Action: draft-ietf-storm-rdmap-ext-01.txt
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work
> item of the STORage Maintenance Working Group of the IETF.
> 
> 	Title           : RDMA Protocol Extensions
> 	Author(s)       : Hemal Shah
>                           Felix Marti
>                           Wael Noureddine
>                           Asgeir Eiriksson
>                           Robert Sharp
> 	Filename        : draft-ietf-storm-rdmap-ext-01.txt
> 	Pages           : 30
> 	Date            : 2011-07-11
> 
>    This document specifies extensions to the IETF Remote Direct Memory
>    Access Protocol (RDMAP [RFC5040]). RDMAP provides read and write
>    services directly to applications and enables data to be transferred
>    directly into Upper Layer Protocol (ULP) Buffers without
>    intermediate data copies. The extensions specified in this document
>    provide the following capabilities and/or improvements: Atomic
>    Operations and Immediate Data.
> 
> 
> 
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-storm-rdmap-ext-01.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> This Internet-Draft can be retrieved at:
> ftp://ftp.ietf.org/internet-drafts/draft-ietf-storm-rdmap-ext-01.txt
> _______________________________________________
> storm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/storm

---

From: Tom Talpey <[email-masked]>
Date: Tue, 13 Sep 2011 12:21:46 +0000

Actually, this question goes deeper - alignment requirements such as this are actually on the *application*. Are we requiring that they align their atomics? Is 64-bits always enough to ensure correctness? Also remember the peer performs no processing, it simply issues an operation to an application-provided value (the atomic's RDMA triplet).

Normally such a requirement would be specified in the Verbs. Since local behavior is not in scope here, perhaps the document should make no requirement at all, and simply give some advice. The mechanism to avoid it is necessarily at the upper layer.

Tom.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, September 12, 2011 2:12 PM
To: [email-masked]
Subject: Re: [storm] I-D Action: draft-ietf-storm-rdmap-ext-01.txt

Thanks,
--David

> A New Internet-Draft is available from the on-line Internet-Drafts 
> directories. This draft is a work item of the STORage Maintenance Working Group of the IETF.
>

---

From: <[email-masked]>
Date: Tue, 13 Sep 2011 10:49:33 -0400

Restating my concern:

If an alignment violation causes an error in this protocol, then this protocol must specify how to avoid causing an alignment violation.

Thanks,
--David

> From: Tom Talpey [mailto:[email-masked]]
> Sent: Tuesday, September 13, 2011 8:22 AM
> To: Black, David; [email-masked]
> Subject: RE: [storm] I-D Action: draft-ietf-storm-rdmap-ext-01.txt
> 
> Actually, this question goes deeper - alignment requirements such as this are actually on the
> *application*. Are we requiring that they align their atomics? Is 64-bits always enough to ensure
> correctness? Also remember the peer performs no processing, it simply issues an operation to an
> application-provided value (the atomic's RDMA triplet).
> 
> Normally such a requirement would be specified in the Verbs. Since local behavior is not in scope
> here, perhaps the document should make no requirement at all, and simply give some advice. The
> mechanism to avoid it is necessarily at the upper layer.
> 
> Tom.
> 
> In looking at the changes in this draft from the -00 version, I noticed something that needs
> attention, although it was already present in the -00 version:
> 
> The term "naturally aligned" needs to be defined for buffer addresses, because failure to use a
> "naturally aligned" buffer address causes a "Catastrophic Error" at the recipient.  For this reason,
> there needs to be a tight specification of what the sender needs to do to avoid that error.
> Specifying a boundary that is always "naturally aligned" (e.g., 64-bit) should suffice (and is
> probably superior to providing a way to figure this out on-the-wire).
> 
> Thanks,
> --David
>

---

From: Tom Talpey <[email-masked]>
Date: Tue, 13 Sep 2011 15:21:02 +0000

Ok, agreed the document needs to make such a statement, and "natural alignment" is not sufficient especially when considered end-to-end.

Tom.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, September 13, 2011 10:50 AM
To: Tom Talpey; [email-masked]
Subject: RE: [storm] I-D Action: draft-ietf-storm-rdmap-ext-01.txt

Restating my concern:

Thanks,
--David

> Actually, this question goes deeper - alignment requirements such as 
> this are actually on the *application*. Are we requiring that they 
> align their atomics? Is 64-bits always enough to ensure correctness? 
> Also remember the peer performs no processing, it simply issues an operation to an application-provided value (the atomic's RDMA triplet).
> 
> Normally such a requirement would be specified in the Verbs. Since 
> local behavior is not in scope here, perhaps the document should make 
> no requirement at all, and simply give some advice. The mechanism to avoid it is necessarily at the upper layer.
> 
> Tom.
> 
> In looking at the changes in this draft from the -00 version, I 
> noticed something that needs attention, although it was already present in the -00 version:
> 
> The term "naturally aligned" needs to be defined for buffer addresses, 
> because failure to use a "naturally aligned" buffer address causes a 
> "Catastrophic Error" at the recipient.  For this reason, there needs to be a tight specification of what the sender needs to do to avoid that error.
> Specifying a boundary that is always "naturally aligned" (e.g., 
> 64-bit) should suffice (and is probably superior to providing a way to figure this out on-the-wire).
> 
> Thanks,
> --David
>