Subject: RE: [rddp] CRC - rough consensus update

From: "Uri Elzur" <[email-masked]>
Date: Sun, 8 Jun 2003 12:28:51 -0700

Jim

The issue it that CRC as an option, prevents the App from knowing FOR
SURE the CRC is in MPA (see other mail)

The cost for CRC in SW goes away after some time (short) especially if
we eliminate the option, as everyone will have it in the HW. CRC in HW
is trivial and low cost. If we decide to keep CRC as an option, we
consciously muddy the water by not providing a clear direction for
implementations and then you will really get CRC in SW for long time
which is undesirable

Uri

-----Original Message-----
From: Williams, Jim [mailto:[email-masked]] 
Sent: Tuesday, June 03, 2003 7:43 AM
To: [email-masked]
Subject: RE: [rddp] CRC - rough consensus update

> If the _option_ to turn it off is there, it will forever add a 
> negotiation which slows down the initialization and adds complexity to

> implementations. It will be yet another knob the administrator has to 
> deal with and no one showed there is a HUGE performance hit
> 
> Uri

The complexity and performance burden of the option
should be somewhere between negligible and zero if
done right.  Implementations are free to
always use CRC and always require remote end
to use CRC if the implementers wish.  The knob only
exists if implementers feel there is demand for it.
The administrator only needs to know about the knob
if he wants to use it.

The pragmatic issue is that a customer may come to us
and say, "We haven't seen a TCP checksum error in years,
but we are wasting 10% of our compute resources on CRCs.
Can you turn off the CRCs for us?"

We will of course say yes, regardless of whether the 
standard allows it or not.  But I would rather have
a mechanism to do this defined within the standard
than have to invent an ad hoc non-standard mechanism.
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Uri Elzur" <[email-masked]>
Date: Sun, 8 Jun 2003 13:02:18 -0700

Caitlin

If indeed the end result is that ULP is _guaranteed_ to have same data
protection with all transport and needs no API to verify it, I'm
on-board.

Still not sure what this additional complexity on the side with OPTIONAL
CRC buys anyone. 
 
Uri

-----Original Message-----
From: Caitlin Bestler [mailto:[email-masked]] 
Sent: Wednesday, June 04, 2003 5:16 PM
To: Uri Elzur
Cc: [email-masked]
Subject: RE: [rddp] CRC - rough consensus update

On 6/4/03, Uri Elzur wrote:

>Caitlin
>
>Strikes me we're on the same page. However, I don't see a
>way around using an API, IF CRC is OPTIONAL.
>
Agreed. By inclination I am against options. I ended up convincing
myself that this one made sense while trying to compose a blistering
email tearing the rationale for the option to shreds. I could come up
with lots of reasons why it was far from essential, but not as to how it
hurt anybody (beyond wasting an option bit that might theoretically have
found a use sometime 30 years later).

>Let me try to clarify it, using the following scenario: in
>the case MPA protocol makes the use of CRC OPTIONAL, in a particular
>case MPA may have negotiated the connection to NO CRC. Now the ULP 
>cannot tell how MPA has  negotiated CRC without a proper API. i.e. I'm 
>not sure the ULP can query MPA without a proper API (that I also think 
>can not be part of what RDDP requires or can be guaranteed to be
>available)
>
>Therefore what will the ULP do? How will a ULP that wants
>to RELY on MPA's CRC and NOT have to provide its own error recovery do
>it?
>

There have been two criteria proposed:

First, equivalence of result - not method. The ULP does not truly care
*how* the payload is protected (well, at least it shouldn't). It cares
*that* it is protected. An end-to-end CRC32c is defined to be a "safe
harbor", that is one method is defined which will provide adequate
protection without any further questions asked.

Under the "equivalent protection" override, the CRC is suppressed for
specific paths *only* if the network administrator can vouch that these
paths truly have equivalent protections.

Personally, I would anticipate that this feature would
be used by all system administrators who are bored and
have absolutely nothing else in their in-basket. ;-)

Second, their MAY be an API that allows a ULP to volunteer
for a lower level of protection. This lower level MUST NOT
be applied unless the ULP actively invokes this API.
Ignorance of such an API MUST leave the higher standard undisturbed.

With these two rules, any ULP that requires the equivalence
of end-to-end CRC32c protection need do nothing. Any application
designed that does not understand how to lower the lower-protection API
will receive the full end-to-end CRC32c (or equivalent) protection.

So in the absence of an API to query MPA, the ULP can assume that it has
CRC32c (or equivalent) protection.

Caitlin Bestler
http://asomi.com/CaitlinBestler/

---

From: "Uri Elzur" <[email-masked]>
Date: Sun, 8 Jun 2003 13:05:47 -0700

Ok. Seems like we're converging around the aspect of CRC (or equivalent)
is always _guaranteed_ to the ULP. I'm also in support of it 

Uri

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Wednesday, June 04, 2003 5:32 PM
To: Uri Elzur
Cc: Caitlin Bestler; [email-masked]
Subject: RE: [rddp] CRC - rough consensus update

On Wed, 4 Jun 2003, Uri Elzur wrote:

> Caitlin
>
> Strikes me we're on the same page. However, I don't see a way around 
> using an API, IF CRC is OPTIONAL.

The point is we aren't making CRCs "OPTIONAL." Optional means you are
free to turn it on or off.

We've been talking about making it such that the admin MAY indicate that
CRCs can be skipped IF the admin is CONFIDENT that the connection will
be as reliable as if it had the CRC on.

Thus there's a burden on the admin that wants to mark CRCs as ignorable,
which is not there if we use the word, "OPTIONAL."

> Let me try to clarify it, using the following scenario: in the case 
> MPA protocol makes the use of CRC OPTIONAL, in a particular case MPA 
> may have negotiated the connection to NO CRC. Now the ULP cannot tell 
> how MPA has  negotiated CRC without a proper API. i.e. I'm not sure 
> the ULP can query MPA without a proper API (that I also think can not 
> be part of what RDDP requires or can be guaranteed to be available)
>
> Therefore what will the ULP do? How will a ULP that wants to RELY on 
> MPA's CRC and NOT have to provide its own error recovery do it?

The ULP will do the only thing it can. It will know that regardless of
the CRC setting, it will have a link as reliable as if it has the
CRC-32C verification. Because the only time(s) it is valid for the CRCs
to be neglectable is when other factors known to the admin(s) indicate
that the connection is reliable.

Take care,

Bill

---

From: "Teisberg, Robert" <[email-masked]>
Date: Mon, 9 Jun 2003 11:03:14 -0500

> From: Uri Elzur [mailto:[email-masked]] 

> A. If the proposal is that it takes one side that requires 
> CRC to mandate CRC for the particular flow, it allows "knob' 
> free implementations for the side where CRC is always on. It 
> still requires a mechanism for the ULP to know this kind of 
> HW is installed. How is that achieved?

I don't see why the ULP needs to know.  I'd bury the code to
turn on the "I don't need CRCs" bit in the RNIC driver.  Then
it's the RNIC implementor's job to make sure the driver and
RNIC agree.  An RNIC (or its driver) that always insists on CRCs
would hardwire the bit appropriately.  An RNIC that doesn't would
need an (out-of-scope for IETF purposes) administrative interface
to control the bit.

If the ULP gets a vote regarding turning off CRCs, that requires
a simple API, but the mechanism is the same.

> 
> B. If one side can force CRC, how can the other side turn it 
> off? Or is the idea that that side will not initiate comm 
> with CRC on but if required by remote peer it will be 
> supported??? So in any case the SW needs to have the 
> capability to compute/check CRC.

I've been interpreting the "no CRCs" bit as a preference.  Each
end sends an MPA mode declaration indicating its CRC preference.
If the preferences don't match, the endpoints conservatively
agree to generate and check CRCs.

Some suggested requirements:

After receipt of an MPA mode declaration indicating that its peer
requires CRCs, an MPA instance MUST continue generating and checking
CRCs until the connection terminates.  If an MPA instance has
declared that it does not require CRCs, it MUST turn off CRC checking
immediately after receipt of an MPA mode declaration indicating that
its peer also does not require CRCs.  It MAY continue generating
CRCs.  Until its partner's preference is confirmed, it MUST generate
and check CRCs.  The MPA mode declaration MUST contain a valid CRC.

So, yes, I agree that a software MPA implementation needs to be
able to compute and check CRCs.  We may be able to get out of running
the CRC code most of the time, but we can't escape implementing it.
Apologies to lazy programmers everywhere :-)

[snip]

> 
> D. not sure I followed the idea of destination masks, I guess 
> this is just for the side with optional CRC. This clearly 
> adds additional complexity (un-bound) to remember all 
> destination and their preference. Is it the right tradeoff?

The complexity in a hardware RNIC is trivial: just a one-bit register
and an AND gate.  The administrative and software complexity is
another matter.  Those who think it's worth the price will gladly
pay it. Those who don't will mandate CRCs.  The market will choose.

> Sent: Tuesday, June 03, 2003 8:08 AM
> To: Williams, Jim
> Cc: [email-masked]
> On 6/3/03, Williams, Jim wrote:
> 
> >
> >> If the _option_ to turn it off is there, it will forever
> >> add a negotiation which slows down the initialization
> >> and adds complexity to implementations. It will be yet 
> another knob 
> >> the administrator has to deal with and no one showed there 
> is a HUGE 
> >> performance hit
> >> 
> >> Uri
> >
> Good points. What makes the proposal acceptable to me
> is precisely that it can largely be ignored, and that
> it does not even require any setup delay to negotiate
> because the active side can simply send the first FULPDU
> with a CRC before finding out whether the other side will 
> consent to suppressing CRCs.
> 
> But it would be best to view them as "knobs". If they exist, 
> and a stack have no ability to suppress CRCs at all is 
> clearly acceptable, they need to be destination dependent. 
> For example: try to suppress CRCs with destinations that 
> conform to this list of IP masks.
> 
> All valid arguments on why standards SHOULD be as
> accomodating as possible.
> 
> One small disagreement: any customer who is wasting 10% of 
> their compute resources on CRCs is using the wrong algorithm 
> or has an amazingly underpowered processor.
> 
> But the degree of waste is only relevant if the cost of the 
> option is signifigant. As proposed, the cost of this option 
> is that some bits in the MPA Mode frame are allocated, and 
> that it has to be documented properly.
> 
> A "All CRCs All The Time" implementation does not even have
> to *examine* those bits, it merely has to set its bits to 
> indicate that it will insist on CRCs. That's a compile-time 
> constant initializing a byte within a packet. A packet that 
> would have been sent anyway, and a byte that was part of the 
> packet anyway. You can't get options must cheaper than that.
> 
> Even for a *slight* advantage for *some* implementations,
> that is a very low cost. I see no reason not to allow it.
> 
> 
> Caitlin Bestler

---

From: [email-masked]
Date: Wed, 11 Jun 2003 14:45:41 -0600

Jim,

I can say the almost the same thing about CRC calculation (even in software). A number of algorithms that are out there are not highly optimized. With a good algorithm, it shouldn't be taking 10% of processing. 

For instance, we have found that we cut processing time in half if we went from the traditional byte at a time software CRC calculation to our algorithm which handles a 32-bit word at a time:

Loop for each data word
XOR current CRC result with next data word.
Do a table lookup for each byte of the data word. 
(4 tables are used; one for each byte position)
XOR the values from the tables to get the new CRC result.

This gets rid of all the shifts and half the XORs that the other method uses. I can go into more detail, but it is a bit of a pain to show the math in flat ASCII.

Regards,
Pat