Subject: RE: [rddp] One vs Two CRCs

From: "John Hufferd" <[email-masked]>
Date: Fri, 22 Nov 2002 17:55:49 -0800

Yes, you will find that all the new protocols in the IETF will be faced
with IPsec.  This was not a storage issue, it is an IETF issue.  Storage
did not seek this our, it was inflicted on Storage (for right or wrong
reasons).  And it had nothing to do with something special about storage.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

"Uri Elzur" <[email-masked]> on 11/21/2002 11:30:31 PM

To:    John Hufferd/San Jose/IBM@IBMUS
cc:    "Somesh Gupta" <[email-masked]>, [email-masked]
Subject:    RE: [rddp] One vs Two CRCs

John

To me RNIC is a solution for Storage as well as for other types of
applications. Assuming iSCSI on RNIC (which is not even dealt with yet)
will have the same requirements as iSCSI, you are right. However, does it
mean ALL RNICs MUST implement it?

Uri

-----Original Message-----
From: John Hufferd [mailto:[email-masked]]
Sent: Thursday, November 21, 2002 5:19 PM
To: Uri Elzur
Cc: Somesh Gupta; [email-masked]
Subject: RE: [rddp] One vs Two CRCs

Uri,
You were going good until you fell into the area of Security.  That will be
a completely different thing.  I strongly suspect that since iSCSI was
required to have IPsec as a MUST implement, that will be our requirement
here.  Now to be fair, you said, below,  "Use of the feature", so
technically you were not at odds to the IETF direction, but I think that
the intent of your message was to exclude it from the MUST implement
requirement.  And that will probably will not pass go.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

"Uri Elzur" <[email-masked]>@ietf.org on 11/21/2002 03:51:29 PM

Sent by:    [email-masked]

To:    "Somesh Gupta" <[email-masked]>, [email-masked]
cc:
Subject:    RE: [rddp] One vs Two CRCs

There has not been any argument as for the benefit of the additional CRC.
The only argument I hear is implementation. "After listening to many NIC
vendors" in the same meeting, one must reach the conclusion that many other
vendors feel it is easier to implement one CRC.

One CRC is computed the same exact way as an Ethernet CRC - do you mean to
say the Ethernet CRC is harder to do then the 2 CRC scheme?

A separate CRC forces checking the header CRC first (limit the
implementation). IFT forces the implementation to find the beginning of the
message (offset x into the Ethernet frame, varies from frame to frame!) and
compute CRC for few header bytes
then compute Data CRC either as a split compute on the ingess or in one
time on the egress.

IPsec is fully supported by this architecture. Use of it has to do with
security risks and there is no reason to mandate it when the user risk
model is not mandating it.

Uri

-----Original Message-----
From: Somesh Gupta [mailto:[email-masked]]
Sent: Thursday, November 21, 2002 3:05 PM
To: Rddp
Subject: [rddp] One vs Two CRCs

After listening to the discussion after Jim William's
presentation at the IETF, I think that having two CRCs
is more flexible than having one CRC.

A single CRC restricts the implementation to one where
the check must be done in-line with the packet ingress -
which in turn implies many other design choices.

A separate header/payload CRC allows the CRC to be done
either at ingress, or at egress to the system. It does
not unduly constrain the implementations that want to
perform their checks only at the ingress. It allows both
unbuffered and buffered models. The only negative
being an extra 4-bytes.

I would suggest mandating the use of IPSec and
not having to worry about CRCs at this layer at all.

Somesh
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 22 Nov 2002 21:51:42 -0600

As I understood the charter for DDP, it is supposed to run
over IP transports: especially SCTP and TCP.

The ipsec charter specifies the goal as being to "develop
mechanisms to protect client protocols of IP".

So whether IPsec is present or not is just not relevant to
DDP, anymore than whether Ethernet or ATM is the L2
protocol. DDP rides just above L4. DDP and IPsec should
both be blissfully ignorant of each other.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Michael Krause <[email-masked]>
Date: Sat, 23 Nov 2002 07:02:03 -0800

>both be blissfully ignorant of each other.

Agreed.  This is the whole point of being layered on top of the transport 
and IP - it isn't a DDP issue at all as it just operates on the data 
payload which has been handed to it by the LLP which has determined this is 
good data from the LLP's perspective.  The LLP and IP deal with IPSec as 
this is well below layer 5 (DDP).

Mike

---

From: "serag gadelrab" <[email-masked]>
Date: Mon, 25 Nov 2002 13:11:06 -0500

It looks to me that the case for a second CRC can really be boiled down
to how we will deal with this scenario (re-phrasing from Jim's e-mails &
draft):

1. The far-end host performing a TCP re-segmentation of a FPDU that was
already handed down to it by the MPA layer and
2. These re-segmented TCP segments are received out of order at the data
sink.

In this case, checking the single CRC while the frames are coming into
the RNIC buffer is not going to work since you may not know if a TCP
segment belongs to one frame or has the fragments of more than one frame
in it (since you may not have a marker in the segment).  In this case,
the 1 CRC implementation will therefore have to:
1. Write enough data into the RNIC buffer to form a frame. 
2. Read the entire frame to check the crc of the frame and if the crc is
good then
3. Read the entire frame again to place the data into the host buffer.  
This process consumes 1 Write (entire frame) and 2 reads (entire frame)
from the memory bandwidth (bandwidth == latency and bandwidth == money).

If you have two CRCs (as per IFT) then you are able to reduce the
processing for the above scenario to:
1. 1 Write (entire frame) into the RNIC memory
2. 1 partial read (of the header only) to validate the destination
buffer in the host through a header CRC check
3. 1 full read (of the entire frame ) into the host memory and perform
payload crc check on the fly.
Thus for the 2 CRC situation, we are able to reduce the required memory
bandwidth by almost 30% (assuming that the partial read is much smaller
than the full frame read)

Both methods will of course work. The 2 CRC option does not preclude the
first method but it does facilitate the second method (which reduces the
amount of memory BW that an RNIC needs to process a frame).  It would be
nice if we can choose an option that will work nicely not only for this
out-of-order, resegmented scenario but also for the "normal" scenario
where the FPDU headers are aligned to the TCP header.

Personally, I think that the 2 CRC method facilitates better
implementations without really restricting any implementation that would
have been used for the 1 CRC method.  The 2 CRC method also facilitates
implementations that have better latency and lower cost.  Finally, the 2
CRC method allows you to have a simple way of treating the incoming
stream regardless whether it is aligned to the TCP header or not (the
scenario discussed here).

Serag GadelRab
Hardware Development
S2IO Technologies Corp.
[email-masked]

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 25 Nov 2002 12:50:11 -0600

I am going to summarize what I believe the current state of
the discussion on one CRC vs two is.

Feasability:

Objections to the complexity of two CRCs were raised. This
was countered by pointing out that the two CRCs are in fact
the same algorithm and are applied serially. The only real
complexity is that the framing protocol must be told the
size of the header when enabled (it shouldn't assume that
it is the size of a DDP header).

While this is not free, it is relatively trivial compared to
the complexity of finding the frame start after alignment
has been lost.

As that I have not seen counter-argument, I believe the
feasability of two CRCs has been established.

Benefits:

The benefit of dual CRCs has been summarized as allowing
placement of the first half of framed segments. This
scenario applies when alignment has been lost.

Theoretically up to half of a connection's TCP segments
could have been received without having any complete framed
segments. With an optimal design for allocating temporary
storage, preliminary placement could reduce the buffering
required in half. With simpler allocation strategies (not
supporting allocations less than the maximum TCP segment
size) the space saved could be considerably less.

Because the first half of an untagged message could have
already been placed, there would be a slight improvemenet in
latency when the second half arrived. Note that latency is
only relevant for untagged messages.

It is also important to note that a header does NOT enable
"cut-through" placement on a byte by byte basis. If the TCP
segment is invalid the DDP header CANNOT be considered
valid, even if its 32 bit checksum passes. Therefore, the
DDP engine MUST NOT act on the DDP header *before*
validating the TCP segment.

Weaknesses:

The second CRC provides absolutely no benefit when alignment
is maintained. By the time the TCP segment is validated, the
entire framed PDU is available.

It consumes four bytes, that could have been used for
payload. It requires extra calculations, that no matter how
trivial, will consume gates and time.

Summary:

Given that a two CRC approach is feasible, the question is
whether it is desirable. It provides *no* benefit when
alignment is maintained, which is the *intended* deployment
scenario.

On the other hand it greatly reduces the buffering
requirement when dealing with an out-of-alignment
connection.

So, is it worthwhile to slightly reduce the performance of
the protocol in its *intended* deployment in order to
improve its worst-case behavior under a scenario that the
network administrator is supposed to avoid?

The drafters clearly favored performance under intended
deployment scenarios. I think I agree with them, but it is a
valid topic for debate. I believe the other issues have all
been addressed. Can we drive to a consensus on the remaining
fundamental issue of optimizing for alignment or not?

Caitlin Bestler

---

From: "Somesh Gupta" <[email-masked]>
Date: Mon, 25 Nov 2002 12:07:23 -0800

Caitlin,

The question about whether alignment can be guaranteed or not
is of course very important, and if alignment was guaranteed
there would perhaps be not too much debate.

However, assuming alignment is not guaranteed, do we want
to choose a mechanism that does allow significant flexibility
in implementation or not. It is not purely a matter of
reducing buffering. Some may have a NIC design that never buffers
(a partial offload device that adds placement to checksum offload,
or a really smart device that targets a specific market segment).
Some may have a design that always buffers (reasons could be
leveraging an existing TOE implementation, or a software based
TOE on an adapter, or a multi-purpose device or whatever other
reasons there may be).

Having two CRCs allows for the flexibility of implementation
without really being biased towards one implementation or the
other.

Somesh

> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Caitlin Bestler
> Sent: Monday, November 25, 2002 10:50 AM
> To: [email-masked]

---

From: Michael Krause <[email-masked]>
Date: Mon, 25 Nov 2002 12:12:31 -0800

>fundamental issue of optimizing for alignment or not?

Single CRC is preferred.   Endnode memory can be used as an intermediate 
buffer to mitigate on-chip buffer requirements which is reasonable for this 
exception case and may be re-using at least a portion of the logic required 
to create highly scalable solutions (e.g. 10,000's of connections).

Mike