Subject: RE: [rddp] Header CRC mandatory?

From: "Williams, Jim" <[email-masked]>
Date: Thu, 24 Apr 2003 12:37:31 -0400

The argument for making CRC optional
is that there will be a sizeable subset of implementations
that have a software implementation on one end of the 
connection, AND the underlying network is sufficiently
reliable that the combination of TCP checksum and
link CRC provide adequate coverage.  In this case
NEITHER the header or data require CRC.

The argument against optional CRCs is that there does
NOT exist a sizeable subset of implementations with the
above properties.  In that case BOTH the header
and payload should be protected by CRC.

I believe you are suggesting that there is a third
possibility which is a ULP that can tolerate payload
errors but not header errors, possibly because the
ULP applies its own CRC to the payload.  This seems
possible, but less compelling to me because it seems
less likely.  But even if true, it doesn't invalidate
either of the above two arguments.

> -----Original Message-----
> From: Caitlin Bestler [mailto:[email-masked]]
> Sent: Thursday, April 17, 2003 1:42 PM
> To: [email-masked]
> Subject: [rddp] Header CRC mandatory?
> 
> 
> Have we reached a consensus that CRC-32 coverage of at least
> the header is a requirement?
> 
> Without CRC coverage over the header an application CANNOT
> apply its own protection to data until AFTER it invalidates
> all STags that can reference the memory.
> 
> This is far in excess of the use of "one shot" STags. The
> existence of another STag that has the option of referencing
> the same memory exposes it to an error risk -- even when
> there is a solid basis for trusting that the peer is not
> a risk for malicious tampering.
> 
> In prior discussions on STags, we rejected mandating the use
> of STag invalidation (requiring all STags to be "one shot").
> There were numerous application profiles recognized where
> flexible buffering was of value. Without header CRC
> protection, all such applications would be subject to
> increased risk from undetected TCP errors.
> 
> Further, this would be an *increased* risk.
> 
> With pure TCP, only the buffered offered for a specific read
> is ever at risk of receiving bad data. Once delivered, it is
> no longer at risk.  RDMA over TCP without header CRC
> protection would place all advertised buffers at risk
> from undetected TCP errors for the duration of the
> advertisement.
> 
> It is my understanding of our charter that we CANNOT
> increase the risk to applications over the use of non-RDMA
> enabled TCP or SCTP.
> 
> I do not believe that a variable-coverage CRC is of
> sufficient value to justify the increased complexity
> for specifying the variation, and of testing it. But such
> a decision would be valid under our charter. I do not
> believe we have an option to propose direct memory placemnt
> based upon unprotected headers.
> 
> 
> 
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Michael Krause <[email-masked]>
Date: Thu, 24 Apr 2003 10:07:39 -0700

>NEITHER the header or data require CRC.

Prefer to think of this a bit differently:

- For RDMA to have an impact on performance / functionality offered, at 
least one side of the connection / stream needs to be a hardware 
implementation of the RDDP protocol suite.

         - Note: I have not seen anyone advocate the use of RDMA technology 
in the past 9 years I've been developing this type of technology where both 
sides are software only implementations - it does not make a lot of sense 
to me what value could be derived from using RDMA over a regular 
TCP.  Also, I have not seen anyone describe such a usage configuration and 
subsequent requirements in any detail in this forum but perhaps I missed an 
e-mail that did so.

- A software implementation may be focused primarily on pushing data to the 
hardware implementation so that it can be placed (Write or Send) or sourced 
(Read).  Such implementations exist today, e.g. DAFS that use this type of 
operational model.

- A hardware implementation may be efficiently perform any type of 
operation defined by the protocol drafts and there has been reasonable 
consensus that the headers must be protected to avoid silent data 
corruption.  Based on implementation experience and the CRC analysis as 
bandwidth increases, there is also reasonable consensus that the data 
should also be protected with the existing drafts proposing using a single 
CRC for both (see previous e-mails for that justification).

 From my perspective, software costs are really about packaging the data 
into a format that hardware on the other side of the connection / stream 
can execute in an optimal, single-method manner while providing the 
requisite data integrity.  The costs to software have been about the need 
to include markers and CRC.  My contention has been that such costs can be 
mitigated with good design / implementation such that the end-to-end 
solution is optimal from a customer perspective.  I have not seen data that 
demonstrates that such designs / implementations are not possible or 
overtly burdensome to implement.

Mike

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 24 Apr 2003 19:05:07 -0500

>NEITHER the header or data require CRC.
Which presupposes that software calculation of a CRC-32
is difficult and unduly time-consuming. I disagree.
There is no inherent need for additional memory touches
to calculate the CRC-32 for the full payload -- that is
a characteristic of the algorithm. Implementation in
software and/or hardware is irrelevant. The fact that
*current* DMA hardware only performs a subset of this
function is not really a justification for an option.
As Mike already pointed out, current hardware also tends to
be deficient in support of IPv6.

>either of the above two arguments.

There are indeed applications that can tolerate payload
errors, especially when the transfer is for final use
and there is no risk of accumulating errors. Whether making
payload CRC optional is of sufficient benefit to them is
another question.

With plain TCP, an application can decide on its own that
it need no further protection of the payload. But with plain
TCP it can inspect the delivered payload *once*.

With RDDP, the payload is at risk until there is NO STag
for the Protection Domain that is valid. Further, because
the header is not validated, the ULP cannot rely upon
automatic STag invalidation. It must do so itself after
receiving a completion notification. Only *then* could it
determine if the received payload was usable.

That amount of caution in an interface is, in my opinion,
unnatural. I find it hard to believe that an application
would truly choose to take those risks with informed
consent. More likely they would expose themselves to these
risks unknowingly, not realizing that without a header CRC
that RDDP/MPA/TCP creates new risks that did not exist
for plain TCP.

RDMA has had to fight against assumptions that direct access
to user memory inherently implied weakened security and
reduced reliability. We don't need to carve out an exception
in order to actually make it true.

Caitlin Bestler

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 24 Apr 2003 19:15:00 -0500

On 4/24/03, Michael Krause wrote:

>- For RDMA to have an impact on performance /
>functionality offered, at least one side of the connection
>/ stream needs to be a hardware implementation of the RDDP
>protocol suite.
>

Not to disagree with your comment on CRCs, but I strongly
believe that RDMA defines a fundamentally improved
interaction between the parties involved (source ULP,
source transport, destination transport and destination
ULP).

That fundamental improvement enables hardware
implementation. But it is a fundamental improvement even if
implemented solely in software.

Whether implemented in hardware or software:

    RDDP still enables placement directly from the MPA Frame 
    / SCTP Data Chunk to user memory without any 
    intermediate buffering.
    
    It still minimizes the number of interactions with the
    ULP, and hence the number of required context switches.
    
    It still frees the data source to send data in the
    order most convenient to it, not an arbitrary order
    matching the data sinks pre-conceptions.
    
    It still enables the data sink to fetch data on
    demand using RDMA Reads.
    
That said, ultimately any simple algorithm with wide
applicability will be implemented better in hardware than in
software. RDMA's strengths enable it to be implemented in
hardware, but if its only strength were truly that it could
be implemented in hardware then it would not be justified.
Hardware should adjust to the needs of the network, not the
other way around. But the network should be designed to
minimize the amount of work each node must do -- and that
work is minimized no matter how it is dividied between the
main processor, co-processors, motherboards, ASICs, FPGAs or
Network Interface Cards.

Caitlin Bestler