Subject: [rddp] Looking for some insight

From: "Barry Reinhold\(home\)" <[email-masked]>
Date: Mon, 21 Mar 2005 15:47:19 -0500

While looking at the design of some conformance tests a couple of issues
have come up:

Item 5 in clause 10.2.1 states:

     " 5.  An RNIC MUST provide a mechanism for the ULP to establish and 

          revoke read, write, or read and write access to the ULP Buffer 

          referenced by an STag. "

I am unclear as to the intent of this statement. Is it stating that the RNIC
must provide the ULP with the ability to revoke read and write access
independent of the revoking of the association between an Stag and a buffer?

In particular, if I have Stag 1 which was allocated with write access, must
the RNIC provide an interface that allows the ULP to change the access
granted through Stag 1 to just "read" ? (i.e. a mechanism in addition to
revoking Stag access and allocating a new Stag with read access)

Is memory associated to a Stag TO pair, or just a Stag? A buffer appears to
be defined by both the Stag and the TO, so is it possible for the RNIC
interface to return (STAG=100, TO =0), associating a buffer to a (STAG, TO)
and then return (STAG=100, TO= 500) for another buffer? (Assuming this did
not violate any length constraints)

Item 6 in clause 10.2.1 states:

      6.  An RNIC MUST ensure that the network interface can no longer 

          modify an advertised buffer after the ULP revokes remote 

          access rights for an STag.

The actual wording makes no assertion about what happens to READ access via
the network interface to a buffer once the ULP revokes remote access rights.
I am assuming the intent of this is really type of access. (I think this is
a editorial issue and "modify" should be replaced by a more general terms)\

Item 7 in clause 10.2.1 states:

     7.  An RNIC MUST ensure that a Remote Peer is not able to 

          invalidate an STag enabled for remote access, if the STag is 

          shared on multiple streams.

What is the expected behavior of the RNIC when its remote peer attempts to
invalidate a Stag that is valid within the context of two streams? The draft
does not indicate what the actual behavior is - I am assuming that the RNIC
will terminate the stream. Is this correct?

Item 8 in clause 10.2.1 states:

      8.  An RNIC MUST choose the value of STags in a way difficult to 

          predict. It is RECOMMENDED to sparsely populate them over the 

          full range available.  

Is the "full range" a reference to the bits available in the fields used to
describe the Stag (i.e. the Stags should range over 32 bits from 0 to 2^32
-1) or is it some vendor specific range?

Item 11 in clause 10.2.1 states:

      11. An RNIC implementation SHOULD provide a mechanism to cap the 

          number of outstanding RDMA Read Requests.

Does "cap" imply that the ULP API must provide the ability to define and
hence "cap" the number of outstanding READ requests that the RNIC will
accept from the ULP, or does CAP imply that, whatever the limit is (lets say
16) then the RNIC will ensure that there are never more than 16 READ REQUEST
as compared to READ REPLIES (as seen on the wire)?

 

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Caitlin Bestler" <[email-masked]>
Date: Mon, 21 Mar 2005 15:18:33 -0800

________________________________

	From: [email-masked] [mailto:[email-masked]] On
Behalf Of Barry Reinhold(home)
	Sent: Monday, March 21, 2005 12:47 PM
	To: RDDP
	Subject: [rddp] Looking for some insight

	While looking at the design of some conformance tests a couple
of issues have come up:

	     " 5.  An RNIC MUST provide a mechanism for the ULP to
establish and 

	          revoke read, write, or read and write access to the
ULP Buffer 

	          referenced by an STag. "

	I am unclear as to the intent of this statement. Is it stating
that the RNIC must provide the ULP with the ability to revoke read and
write access independent of the revoking of the association between an
Stag and a buffer?  

	In particular, if I have Stag 1 which was allocated with write
access, must the RNIC provide an interface that allows the ULP to change
the access granted through Stag 1 to just "read" ? (i.e. a mechanism in
addition to revoking Stag access and allocating a new Stag with read
access)

I believe the intended syntax if "for the ULP to establish and revoke"
<read,write or read and write access>
 
As was briefly touched upon during the security discussions as to
whether IPSEC was required, one
characteristic of virtually all current local interfaces is that STags
never change from meaning A to
meaning B. You go from meaning A to invalid and *then* to meaning B.

	Is memory associated to a Stag TO pair, or just a Stag? A buffer
appears to be defined by both the Stag and the TO, so is it possible for
the RNIC interface to return (STAG=100, TO =0), associating a buffer to
a (STAG, TO) and then return (STAG=100, TO= 500) for another buffer?
(Assuming this did not violate any length constraints) 

Yes, an application can create a memory region or window enabled for
remote access and then advertise
multiple buffers within it. Each time it advertises a buffer it is
guaranteeing to its peer that it has enabled
access to all bytes within advertised buffers. It is not guaranteeing
that it enabled no other access, but
if its peer is totally ULP compliant this fact would never be relevant.
[Hint: that's a good reason why you
generally want to use one window/one buffer. But that's advice to
implementers, not something that a
wire validator can rely upon.]
 
As far as the wire protocol goes, the STag is deliberately confusing, I
mean flexible.
 
The STag defines a context in which the TO is interpreted. If the STag
state has not changed, a given TO
translates to the same byte in target memory consistently.
 
The ULP is allowed to advertise a range of Target Offsets as a "buffer",
and each of <n> bytes within a range
reference a distinct byte in target memory. Presumably they are also
somehow ordered in target memory.
 
The ULP is guarnateed the ability to create a valid range within an
STag. It would be legal, although highly
unlikely, for a local interface to enable multiple valid ranges on a
single STag. The only rationale I have seen
for such a scenario would be to do a ring buffer within an STag -- in
which case the enabled section (which 
is conceptually one range) might look like two ranges to hardware that
did not understand that it was a 
ring buffer.

	      6.  An RNIC MUST ensure that the network interface can no
longer 

	          modify an advertised buffer after the ULP revokes
remote 

	The actual wording makes no assertion about what happens to READ
access via the network interface to a buffer once the ULP revokes remote
access rights. I am assuming the intent of this is really type of
access. (I think this is a editorial issue and "modify" should be
replaced by a more general terms)\ 

I would agree, it falls under the general principle that the RDMA
implemenation
makes *no* memory access that has not been authorized. 

	          invalidate an STag enabled for remote access, if the
STag is 

	What is the expected behavior of the RNIC when its remote peer
attempts to invalidate a Stag that is valid within the context of two
streams? The draft does not indicate what the actual behavior is - I am
assuming that the RNIC will terminate the stream. Is this correct? 

My reading was that this qualifies as a remote access violation. 

	      8.  An RNIC MUST choose the value of STags in a way
difficult to 

	          predict. It is RECOMMENDED to sparsely populate them
over the 

	          full range available.  

	Is the "full range" a reference to the bits available in the
fields used to describe the Stag (i.e. the Stags should range over 32
bits from 0 to 2^32 -1) or is it some vendor specific range?

	      11. An RNIC implementation SHOULD provide a mechanism to
cap the 

	Does "cap" imply that the ULP API must provide the ability to
define and hence "cap" the number of outstanding READ requests that the
RNIC will accept from the ULP, or does CAP imply that, whatever the
limit is (lets say 16) then the RNIC will ensure that there are never
more than 16 READ REQUEST as compared to READ REPLIES (as seen on the
wire)?