Subject: RE: [rddp] CRC - rough consensus update

From: John Hufferd <[email-masked]>
Date: Wed, 28 May 2003 16:45:51 -0700

I do not think it is an overhead issue, but it may simplify some of the 
actions that need to be handled with HW RNICs, because the packets are 
manipulated at different points with different state engines etc. 

However, if it does not matter to the RNIC vendors, then with the overhead 
low, it should just always be done.  And in that case there is no Admin 
action for CRC.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: [email-masked]

"Talpey, Thomas" <[email-masked]>
Sent by: [email-masked]
05/28/2003 02:24 PM

        To:     John Hufferd/San Jose/IBM@IBMUS
        cc:     <[email-masked]>
        Subject:        RE: [rddp] CRC - rough consensus update

At 02:35 PM 5/28/2003, John Hufferd wrote:
>So, lets propose that based on the perceived fact 
>that SW overhead is NOT an issue, we just say that the requirement from 
the 
>iWARP layers is to provide to the ULP protection at least as strong as 
that 
>provided by CRC32c. 
...
>If the above is reasonable, then since we have stated that if either side 

>wants it, the CRC must be sent.  And if a side can not tell if it has 
IPSec 
>Encryption but the other side can and says that it does not want the CRC, 

>then don't generate the CRC.  However, if the other side does not 
explicitly 
>say it does not want the CRC, then it should be provided. 

If overhead is not an issue, then what difference does it make whether the
CRC is included or not? In other words, why bother to negotiate leaving it 
out?
Just calculate it and let the receiver decide whether to pay attention.
Everyone wins.

This has been my point, at least.

Tom.

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: [email-masked]
Date: Wed, 28 May 2003 23:47:52 +0000 (UTC)

On Wed, 28 May 2003, Caitlin Bestler wrote:

First off, thank you for the thoughtful note.

> On 5/28/03, [email-masked] wrote:
>
> >On Tue, 27 May 2003, Caitlin Bestler wrote:
> >
> >> On 5/27/03, Michael Krause wrote:
> >>
> >> >>My point is that the data belongs to the application,
> >> >>not to the system administrator.
> >> >
> >> >The system belongs to the administrator / customer not
> >> >the application designer or the ULP.
> >>
> >> On the contrary, it is the ULP.
> >
> >I'm sorry. I'm missing something. I go out and buy ten
> >computers. One of them will be a server to the other nine.
> >I pay for them myself. I install the OS myself. I buy &
> >install the applications myself. I seed the data myself.
> >At what point did the computers, software, and data start
> >belonging to themselves and not me?
> >
> And if you want to use email on those computers that you
> just bought, do you sit down and decide to do it on your
> own? Of course not, you use an existing Upper Layer
> Protocol.

True, but the data still belong to me, don't they?

I'm not sure if EMail is the best example here, for three reasons. First,
EMail is an application, not just a ULP. Second, Email gets transfered
over a number of different protocols. While SMTP is the most common, POP
and IMAP and other protocols get used. Third, EMail is, well, complicated.
:-) It's a mess. Configuring sendmail or postfix is something I'd rather
not do. I'd rather build a new kernel file system or iSCSI stack or RDMA
implementation instead. ;-)

> Or, occassionally, you design your own ULP to meet your
> own custom needs.
>
> The point is that it is the ULP that decides if additional
> data integrity is required, because it is the ULP that
> must specify what form (if any) that additional integrity
> will take.

I definitely agree that the ULP knows better than the RDMA layer how risk-
sensitive it is.

My whole point though is that the ULP should listen to the admin about
relaxing requirements on CRCs. i.e. for just the ULP & RDMA, the ULP knows
better. But when we add the admin to things, the admin should be further
up the authority chain than either the ULP or RDMA.

> For example, RDDP offers no services to authenticate the
> end user. A ULP that requires those services must add
> them on their own. Virtually everyone agrees that this
> is a good thing because the needs for user authentication
> vary so widely amongst different ULPs. It is not a natural
> service for a transport layer.
>
> So when I design a ULP using RDMA, can I rely upon
> end-to-end CRC32c (or equivalent) data protection or not?

Yes.

I think it is reasonable to assume that you will have CRC32C or equivalent
data protection. The only time you wouldn't have CRCs in active use are
times when the admin(s) have decided that the links a connection will
travel over will be safe enough with the TCP checksum.

Given how contentious this point is, I think it would be reasonable to add
text indicating that SHOULD not disable RDMA CRCs (well indicate a desire
to ignore them) without assessing both the links such a connection will
traverse and the error sensitivity of the ULP. Yes, it's not enforceable by
RDMA, but it can serve as an agreement between the ULPs and the admins as
to when CRCs will be ignorable.

> If I have to actively *check* something that I really do
> not. An active control is an additional failure point.
> It is one more thing that has to be tested and monitored
> forever. It is a permanent ongoing cost for all
> applications, well at least those that do QA.
>
> And for what benefit? So *some* implementations can
> safe a very small amount of CPU power? When *most*
> will either perform the check in hardware and/or
> have adequate CPU power?
>
> Reduced data-integrity checking is a specialized
> need. I have no problem allowing it, provided that
> it costs the general application nothing.

And I don't think this will really cost the general application anything.

Everything we're saying (or everything I've been hearing) indicates that
everyone is fine with CRCs defaulting to enabled, and having either side
being able to require them. Thus for them to NOT be active would mean that
a deliberate policy decision by the admin(s) covering the connection has
been made.

> That means that an affirmative step should be required,
> *for each ULP* and *for each path* to enable voiding CRCs.
>
> Anything that resembles a "big red switch" to disable CRCs,
> that applications must check to make sure it wasn't thrown
> on accident, is imposing an unacceptable cost on all
> applications for the benefit of a small handful.

Then let's make that part of the spec.

I think it is appropriate for CRC disabling to be performable on a port by
port basis. We can make it a SHOULD in the RFC. In fact, it sounds like we
should. I've certainly had "it's not a big red switch" in my mind as to
how things should be.

Actually, something closer to how IPsec policy is set up would be best.
Then you can throw address ranges into the policy. So you can permit not
using CRCs over the direct connection you have to a box, but not permit
them being skipped over the connection that goes through the wiring
closet. If you want.

It's sounding like an informational guide for administrators RFC may be in
order for the working group to create.

> That means the ULP must take an affirmative act to indicate
> a willingness to accept less than end-to-end CRC32c
> protection. Avoiding taking an affirmative act is not
> something that you have to test for with each release
> (after all, there an an infinite number of affirmative
> actions that any application has not taken, they obviously
> cannot be tested for).

If we do it like that (say something like the TCP_NDELAY option), then the
ULP or its implementing application really MUST be configurable to forgo
requiring CRCs.

Otherwise the ULP is making a policy decision for the admin, when it
should be the other way around.

> While I cannot think of a way to make it an enforceable MUST
> or SHOULD, any determination by a network administrator that
> a given path offers the equivalent of end-to-end CRC32c
> protection should last only as long as that path is stable.

At some point we have to say, "Yes, there's the rope. If you hang
yourself, that's your own foolishness." Because every time we prevent
admins from doing something stupid, we also prevent them from doing
something clever.

> Further, the network administrator MUST NOT make predictions
> for any path that goes through any middle-box not under
> their exclusive control. To do otherwise creates an
> obligation on a middlebox to continue to function as
> it has in the past. Any such requirement would be contrary
> to the fundamental nature of an IP network.
>
> Ultimately, the futility of CRC suppression based on
> "network quality" is that the environments where that
> assertion can be made are exactly those where software
> implementations are the least likely.
>
> Any real value will more likely be found with software
> stacks running on remote clients for those applications that
> can actively assert that an end-to-end 16-bit TCP checksum
> is adequate for their needs. The gotcha there is that
> calculating a CRC32c for even T1 speeds is trivial.

Since we're talking hypotheticals, it's not to hard to find a counter
example. :-)

I think a common and reasonable case would be one where we have end-to-end
IPsec. It is my understanding that the cryptographic hashes in IPsec are
stronger than CRC32C, so the CRC will, in a well-crafted system, add no
value. So there's a case that will be using high-performance hardware
where an application/ULP will see CRC32C equivalent protection w/o a CRC.

I agree that IPsec in and of itself isn't the cure-all, since there may be
perfectly-valid systems that have the IPsec calculation far enough away
that the TCP checksum isn't good enough. But there can (and I expect will)
be times when it is.

Also, haven't we been talking about (as one installation scenario) the case
of fan-in systems, where a number of clients fan into one system? While
it's not hard to calculate the CRCs at T1 rates, if you have a few
thousand simultaneous feeds (or ten thousand feeds), you start to notice.

Take care,

Bill