Subject: RE: [rddp] draft-carrier-rddp-rnic-interop-00.txt

From: "Jim Pinkerton" <[email-masked]>
Date: Mon, 29 Nov 2004 12:00:10 -0800

Commenting to 5), the reason why both CreateQP and ModifyQP is needed is
because some protocols (notably ISER) don't want to negotiate RDMA
capability unless they know that resources are available for RDMA.  So I
would expect this class of protocols to first do a CreateQP before
exchanging keys. At this time, you don't know what the version number
will be negotiated to (assuming the interoperability protocol suggested
in the draft), thus only after reception of a REQ/REP frame do you set
the version number, which must be done using a ModifyQP.

On 4), possibly the wording could be improved. It really comes down to
RDMAC versions which wish to interoperate with IETF versions. Those that
don't try, won't :-)

Mike, with the exceptions of the concerns on changes to MPA/DDP/RDMAP to
give the right level of "escape hatch", does the approach appear sound?
Note that the ambiguity in some of them has to work the other way - make
it more ambiguous - i.e. if you are able to talk with version 0, allow
you to do so (current text prohibits it).

Jim

________________________________

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Michael Krause
Sent: Monday, November 29, 2004 11:11 AM
To: [email-masked]
Subject: RE: [rddp] draft-carrier-rddp-rnic-interop-00.txt

At 06:40 PM 11/23/2004, Jim Pinkerton wrote:

Feedback is appreciated on this draft. This was a result of an informal
group working over beers after the RDDP work group meeting gave us
direction to try to solve interoperability as an informative appendix to
the MPA specification. 

John and I tried to be very specific about the exact recommended text
changes for the various drafts, to hopefully expedite the review process
with something concrete to shoot at.

Initial comments.

Mike

(1) It is not clear to me where there is ambiguity within the
specifications.  The RDMAP specification (draft-ietf-rddp-rdmap-02.txt)
section 6.1 states that the RDMA version MUST be 01b.  The table in
section 6.8 states that a remote operation error is generated for a
RDMAP version error.  The RDMAC RDMAP specification contains the same
language but states the version MUST be 00b.  As such, it clear what the
version must be and the error that should be returned if there is a
version mismatch which contradicts the assertions made in section 3 of
the interop draft.

(2) The DDP specification (draft-ietf-rddp-ddp-03.txt) section 4.1
states the DV bits must be set to 1.  Section 7.2 states that an invalid
version must be returned upon detecting a mismatch and section 7.1
states that upon error detection, DDP  must stop placing segments and
return an error to the ULP.  In this case, it would be to surface an
error to RDMA.  The RDMAC DDP specification contains the same language
but states the version MUST be 00b.  Again, this contradicts the
assertions made in section 3 of the interop draft.

(3) I agree that there are three cases of implementations that can
exist.  However, it seems that the specifications are relatively clear
on what the versions must be for each set of specifications.  It is also
clear what the expected behavior should be as well, i.e. the generation
/ surfacing of an error.  As such, a RDMAC implementation would require
the version to be 0 to be compliant while the IETF implementations would
require the version to be 1.  The only way to achieve interoperability
is to set the appropriate version number prior to a connection
transition to RDMA / DDP mode.  

(4) A RDMAC-only implementation would not negotiate MPA since this is
not part of the RDMAC specifications.  The draft implies that any RDMAC
compliant implementation would be augmented to provide MPA negotiation.
I don't see how this can be required - at best, it is an option to
implement and may or may not be present on a given endnode.  If this is
not implemented, the two sides will simply fail the RDMA stream
establishment.  In general, the tables showing the negotiation at the
MPA level look reasonable.

(5) It is not clear that new verbs extensions are required for
interoperability.  The verbs specification section 9.3 already states
how a version mismatch (DDP or RDMA) should be surfaced.  There is value
in being able to configure a RNIC (on a per session basis) as to what
version to support.  This should be done once and cause all layers to
use a consistent version for a given QP.  The value should be set at QP
creation and then updated upon session establishment if the RNIC can
support both RDMAC and IETF specifications.  I also see value in having
the query what the current version is.  I don't see why a ModifyQP is
required as one knows up front what is desired and the implementation if
permissive, would negotiate accordingly.  Therefore all that is needed
is to understand the result.

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Michael Krause <[email-masked]>
Date: Mon, 29 Nov 2004 13:54:14 -0800

At 12:00 PM 11/29/2004, Jim Pinkerton wrote:
>Content-class: urn:content-classes:message
>Content-Type: multipart/alternative;
>         boundary="----_=_NextPart_001_01C4D64E.08822EE0"
>
>
>because some protocols (notably ISER) don’t want to negotiate RDMA 
>exchanging keys. At this time, you don’t know what the version number will 
>be negotiated to (assuming the interoperability protocol suggested in the 
>draft), thus only after reception of a REQ/REP frame do you set the 
>version number, which must be done using a ModifyQP.

Agreed.

>don’t try, won’t J

Understood.

>give the right level of “escape hatch”, does the approach appear sound? 
>Note that the ambiguity in some of them has to work the other way – make 
>it more ambiguous – i.e. if you are able to talk with version 0, allow you 
>to do so (current text prohibits it).

Yes.  I agree with the general approach.  This is a good first draft so 
perhaps my comments are being perhaps a bit picky to start.

Mike

>----------
>section 6.8 states that a remote operation error is generated for a RDMAP 
>version error.  The RDMAC RDMAP specification contains the same language 
>but states the version MUST be 00b.  As such, it clear what the version 
>must be and the error that should be returned if there is a version 
>mismatch which contradicts the assertions made in section 3 of the interop 
>draft.
>
>(2) The DDP specification (draft-ietf-rddp-ddp-03.txt) section 4.1 states 
>the DV bits must be set to 1.  Section 7.2 states that an invalid version 
>must be returned upon detecting a mismatch and section 7.1 states that 
>upon error detection, DDP  must stop placing segments and return an error 
>to the ULP.  In this case, it would be to surface an error to RDMA.  The 
>RDMAC DDP specification contains the same language but states the version 
>MUST be 00b.  Again, this contradicts the assertions made in section 3 of 
>the interop draft.
>
>exist.  However, it seems that the specifications are relatively clear on 
>what the versions must be for each set of specifications.  It is also 
>clear what the expected behavior should be as well, i.e. the generation / 
>surfacing of an error.  As such, a RDMAC implementation would require the 
>version to be 0 to be compliant while the IETF implementations would 
>require the version to be 1.  The only way to achieve interoperability is 
>to set the appropriate version number prior to a connection transition to 
>RDMA / DDP mode.
>
>(4) A RDMAC-only implementation would not negotiate MPA since this is not 
>part of the RDMAC specifications.  The draft implies that any RDMAC 
>compliant implementation would be augmented to provide MPA negotiation.  I 
>don't see how this can be required - at best, it is an option to implement 
>and may or may not be present on a given endnode.  If this is not 
>implemented, the two sides will simply fail the RDMA stream 
>establishment.  In general, the tables showing the negotiation at the MPA 
>level look reasonable.
>
>interoperability.  The verbs specification section 9.3 already states how 
>a version mismatch (DDP or RDMA) should be surfaced.  There is value in 
>being able to configure a RNIC (on a per session basis) as to what version 
>to support.  This should be done once and cause all layers to use a 
>consistent version for a given QP.  The value should be set at QP creation 
>and then updated upon session establishment if the RNIC can support both 
>RDMAC and IETF specifications.  I also see value in having the query what 
>the current version is.  I don't see why a ModifyQP is required as one 
>knows up front what is desired and the implementation if permissive, would 
>negotiate accordingly.  Therefore all that is needed is to understand the 
>result.