Subject: [rddp] (no subject)

From: "Barry Reinhold" <[email-masked]>
Date: Tue, 30 Nov 2004 08:36:01 -0500

I would like to know if there is agreement on the following:

Given a Marker supporting Data Source that sends:

++++++++++++++++++++++++++++
| TCP SEG #1               |
++++++++++++++++++++++++++++
Contains FPDU A (write STag 1, loc 100 with 100 A's)

++++++++++++++++++++++++++++
| TCP SEG #2               |
++++++++++++++++++++++++++++
Contains FPDU B (write STag 1, loc 200 with 100 B's)

++++++++++++++++++++++++++++
| TCP SEG #3               |
++++++++++++++++++++++++++++
Contains FPDU C (write STag 1, loc 300 with 100 C's), and FPDU D (Read
STag 1, 100 bytes from loc 300)

A marker falls in FPDU D.

During transmission TCP SEG #2 is lost, such that TCP SEG #3 arrives out
of order at the Data Sink. The iWARP receiver recovers FPDU sync by
using the marker in FPDU D. TCP SEG #2 then arrives.

Is it the case that for a "conformant" Data Sink the contents in the
RDMA READ RESPONSE SHALL be "C"s?

This is interesting in that there is no gap in the TCP sequence space as
delivered by TCP to MPA, but there is a gap in terms of what MPA has
"processed".

Since clause 5.4 states that:

At the Data Sink, DDP MUST Deliver a DDP Message if and only if all 
   of the following are true: 

     * the last DDP Segment of the DDP Message had its Last flag set,  

     * all of the DDP Segments of the DDP Message have been Placed, 

     * all preceding DDP Messages have been Placed, and 

     * each preceding DDP Message has been Delivered to the ULP.

I assume that we can expect "C"s in the READ RESPONSE.

This is scheduled to become a conformance test, so I would like to
ensure there is agreement.

Barry Reinhold
Lamprey Networks
[email-masked]
(603) 868-8411

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Caitlin Bestler" <[email-masked]>
Date: Tue, 30 Nov 2004 06:17:29 -0800

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Barry Reinhold
> Sent: Tuesday, November 30, 2004 5:36 AM
> To: RDDP
> Cc: [email-masked]
> Subject: [rddp] (no subject)
> 
> 
> Contains FPDU C (write STag 1, loc 300 with 100 C's), and 
> FPDU D (Read STag 1, 100 bytes from loc 300)
> 
> During transmission TCP SEG #2 is lost, such that TCP SEG #3 
> arrives out of order at the Data Sink. The iWARP receiver 
> recovers FPDU sync by using the marker in FPDU D. TCP SEG #2 
> then arrives.
> 

Recovering the marker for FPDU D allows DDP to process FPDU D.
That allows the RDMA Read Request to be *placed*, presumably
Into an internal IRD queue. But it cannot be acted upon by
The RDMAP layer until it is deliverable, as described in 
The section you cited (5.4).  Therefore all prior writes,
including to location 300, will have been placed before
FPDU D is "delivered" to the RDMAP layer. And until it 
is delivered it is not an RDMA Read Request, ad the DDP
layer it is just an untagged message to queue 1.

> Is it the case that for a "conformant" Data Sink the contents 
> in the RDMA READ RESPONSE SHALL be "C"s?
> 
> This is interesting in that there is no gap in the TCP 
> sequence space as delivered by TCP to MPA, but there is a gap 
> in terms of what MPA has "processed".
> 
> This is scheduled to become a conformance test, so I would 
> like to ensure there is agreement.
>  
>

---

From: "Culley, Paul" <[email-masked]>
Date: Tue, 30 Nov 2004 08:59:37 -0600

The case you describe must result in reading "Cs" from location 300.  It
would not matter what order FPDU B and C arrived in, they always place
in different locations.  And the RDMA Read always takes place after both
placements have occurred (when delivered).

If your test case was meant to place FPDU C in the same location as FPDU
B (location 200) then you can get unexpected results.

This is an interesting case, one that was explicitly described in the
RDMA spec.  I quote from section 7.5 "ordering and completions":

     3.  RDMA Messages that use Tagged and Untagged Buffers MAY be 
          Placed in any order.  If an application uses overlapping 
          buffers (points different Messages or portions of a single 
          Message at the same buffer), then it is possible that the last

          incoming write to the Data Sink buffer will not be the last 
          outgoing data sent from the Data Source. 

So if both FPDU B and C placed at location 200, and FPDU B arrived later
than FPDU C, then you would read back "Bs" from 200, even though you (as
a sender) thought you sent the "Cs" later, and they should have been
read back.

Paul R. Culley
HP Fellow
281-514-5543

> Sent: Tuesday, November 30, 2004 7:36 AM
> To: RDDP