Subject: Re: [Tsvwg] [SCTP checksum problems]

From: Mark Bakke <[email-masked]>
Date: Thu, 19 Apr 2001 12:49:55 -0500

Chip-

You are correct on all counts.  However, this does not reduce the
need for an iSCSI-level CRC; it does mean that the type of integrity
provided depends somewhat on network engineering:

For a layer-2 network, with no routers or middle boxes, there's no
need for iSCSI CRC; this is analogous to a Fibre Channel network
today.

For a carefully-built layer-3 network, the TCP checksum can be
good enough, and again, the iSCSI CRC is not always required.

That leaves us with two other types of networks:

1. Networks where the quality of the various routers and middle
   boxes are not certain, and may cause accidental harm that is
   not detected by TCP.  These might include larger, private
   networks, leased metro networks, or networks subject to
   cheap equipment purchases (small business, home, etc).

2. Networks where there is a real threat of intentional harm to
   the data.

Note that network type (1) would also include networks of type (2)
where there may be minimal threat, and the cost of dealing with
the threat is not justified.

Network type (2) requires a signature on the data; IPSec would
probably be used for that case.  The expense for doing IPSec
would be justified.

Network type (1) can use a non-cryptographic, iSCSI-level CRC.

I have a few more comments below.

--
Mark

Chip Sharp wrote:
> 
> As was pointed out previously, middle box operations (such as NATs) tend to
> creep up the protocol stack and into applications.

This will certainly happen for iSCSI.

> Take SIP for example.  It includes IP addresses in its INVITE.  In order to
> work across a NAT, the IP addresses it exchanges have to be replaced with
> the NATed address.  One way is for the NAT to reach up into the SIP INVITE
> and change the address.  This modifies the TCP or UDP checksum.  Now SIP
> could have included its own integrity check to protect against corrupted or
> modified TCP checksums, but all that would have happened is that NATs would
> have changed the SIP checksum in addition to the TCP/UDP checksum.
> 
> Therefore, even if iSCSI included its own integrity check, if a middle box
> is going to futz with iSCSI packets it will just strip the check, do
> whatever it does and then recalculate the check.

That is the reason that we have separate integrity checks on the header and
the data.  If the middle box is trusted to do so, we have to enable this
sort of behavior.  By separating the checks, we at least have end-to-end
integrity on the data, and could (in theory) do a modification of the header
check without completely throwing it away and recalculating it.  This would
be expensive to do on the data, but would probably not be prohibitive for
the headers (someone correct me if I'm wrong, but I recall some discussions
on the possibilities for modifying CRCs in this way).

> If this is what you want to protect against you will have to go to some
> type of digital signature.

Again, there are times when you want to protect against middle boxes.  In
this case, the digital signature is needed, and perhaps IPSec is the solution.
There are other times when this behavior must be enabled.  In either case
(even with IPSec), the iSCSI CRC can provide end-to-end protection.

> At 12:22 PM 4/19/2001, [email-masked] wrote:
> >Stephen,
> >
> >I have to admit that I do not have much direct experience with middle boxes,
> >BUT I did have fairly direct and recent experience with a popular NAT router
> >from a popular vendor that was corrupting data in a network of Macintoshes.
> >
> >Apple's TCP was unaware of any problem as was Apple's Filing Protocol and
> >most applications. The only applications that detected the corruption were
> >those that performed an integrity check of their own. Those applications
> >that assumed a reliable transport (and file system) were doomed to
> >experiencing the indirect effects of the corruption at some later time. The
> >corruption only happened when large amounts of data were transferred
> >quickly.  The router vendor fixed the problem once; then fixed it again;
> >then fixed it one last time before the data corruption finally
> >"disappeared". After several weeks of continuous operation the router
> >appeared to get into a mode where it was once again corrupting data. Power
> >cycling the router "fixed it". The story apparently has not yet ended.
> >
> >I admit I may have given too much significance to this single incident that
> >I have personally experienced but on the other hand I don't see the
> >mechanisms in place to prevent this type of problem in the future other than
> >the end to end integrity checks.
> >
> >Incidentally this incident change my behavior when transferring data over a
> >network. I will always use a compression utility; not only for reducing the
> >data to be transmitted but to ensure the integrity of my data is protected
> >end to end by the utility's CRC mechanism.
> >
> >I believe quite firmly that we DO need a mechanism to allow us to tolerate
> >poor implementations of middle boxes and cannot simply hope that eventually
> >such poor implementations will vanish, nor that we will have the luxury of
> >being able to select only good implementations for every component of our
> >storage network.
> >
> >Vince
> >
> >|-----Original Message-----
> >|From: Stephen Bailey [mailto:[email-masked]]
> >|Sent: Wednesday, April 18, 2001 3:09 PM
> >|To: CAVANNA,VICENTE V (A-Roseville,ex1)
> >|Cc: 'WENDT,JIM (HP-Roseville,ex1)'; [email-masked]';
> >[email-masked]; [email-masked]; 'Craig Partridge'; Jonathan Wood;
> >[email-masked]; Jonathan Stone; Randall Stewart
> >|Subject: Re: [Tsvwg] [SCTP checksum problems]
> >|
> >|
> >|Vince,
> >|
> >|> I don't think iSCSI can be completely relieved of performing
> >|some data
> >|> integrity checking as long as there exists the possibility
> >|of "middle boxes"
> >|> opening up the transport protocol's packet and thus
> >|potentially invalidating
> >|> any reliability guarantees the transport protocol makes.
> >|
> >|Any protection provided against this failure mode will only be
> >|transient, so we must temper the desire to introduce such a
> >|requirement with reality.
> >|
> >|Middleboxes can just as easily open up to the iSCSI layer and tinker
> >|with the payload, as they do with other ULPs running on TCP (e.g HTTP)
> >|today.  Short of securing the connection, there is ALWAYS a
> >|possibility of a middlebox terminating and reoriginating an integrity
> >|check.  In case you think this is a farfetched scenario, I do get the
> >|impression that there is a high level of interest in `actively
> >|middling' iSCSI once the specs crystalize.  Who shaves the barber?
> >|
> >|An integrity check is not necessary as long as some lower layer
> >|provides adequate integrity guarantees.
> >|
> >|Adding an integrity check above the transport layer is based upon
> >|documentation of the presence of a lot of crappy network hardware and
> >|software and analyses of the transport integrity check (TCP checksum)
> >|which suggests it might not be adequately strong against some such
> >|observed errors.
> >|
> >|I claim that the high incidence of `broken' (corruption introducing)
> >|components is a result of a variety of factors which have shaped the
> >|development of network components thus far.  The fact that integrity
> >|checks are assumed to be performed in a network context substantially
> >|lowers the bar for implementation correctness.
> >|
> >|In a storage (or CPU) context, these types of implementation errors
> >|are a) more easily detectable (more fatal) b) more carefully avoided
> >|during implementation (because of the cost of a potential fatal
> >|error).  If network components magically reached the same `quality
> >|level' as storage and CPU components, there might be no justification
> >|for additional integrity checks above the transport.  Similarly if the
> >|transport (or whatever lower layer) integrity checks are very strong
> >|(e.g. IPSec), there is, again, no need for a higher level integrity
> >|check.
> >|
> >|I am not disagreeing that we need an additional integrity check over
> >|TCP in the present target environment, but I do disagree that iSCSI
> >|will always need such a check, independently of what is running
> >|beneath it.
> >|
> >|Steph
> >|
> 
> -------------------------------------------------------------------
> Chip Sharp                       Consulting Engineering
> Cisco Systems
> -------------------------------------------------------------------

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Chip Sharp <[email-masked]>
Date: Thu, 19 Apr 2001 12:53:53 -0400

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 19 Apr 2001 15:26:53 -0500

Hi, all,

Please correct me if I am under the wrong impression, but what I've been
hearing so far from the discussion in this thread seems to be:

 1) There is uncertainty on what error model should be used for
    evaluating transport error check mechanisms;

 2) iSCSI level integrity/authentication seems inevitable not matter how
    strong the transport error check is;

 3) Some uncertainty about the implementation cost (hw and sw) on some
    of the proposed CRC-based schemes, but almost certainly any 
    CRC-based scheme will be more expensive than checksum schemes;

 4) Discussion of requirements has been so far very much limited to
    iSCSI data transport, and it seems that a CRC-based scheme is 
    strongly desirable for iSCSI case _if_ the transport error check 
    alone is all we will get to meet the iSCSi requirements.

Now my question to the group is: if a weaker but much cheaper SCTP error
checking mechanism can be found to be sufficient to the majority of the
applications (remember SCTP is a general purpose transport), and the
ultra-low error rate applications such as iSCSI will eventually rely on
their own integrity check, should we adopt a selection approach which
put more emphasis on the ease-of-implementation side?

(Maybe after all the Adler-32 in the current RFC2960 is good enough :-)
)

regards,
-Qiaobing

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 19 Apr 2001 16:20:32 -0700

Qiaobing,

In essence, the problems for iSCSI and SCTP are similar.  Both wish to
improve error detection over TCP checksums.  Examination of Adler-32
indicated weak burst error detection and low sensitivity for small frames, a
principle uses of SCTP at this point.  A realistic means of examining the
transport end-to-end error detection is needed for both of these protocols,
although SCTP is indeed more generalized.  Moving forward, it would be
beneficial if a common scheme were employed and agreed adequate.  There are
problems with various Telco components over more robust Ethernet equipment.
DSLAM interfaces and poorly aggregated SONET packets may be areas of greater
concern than Gaussian noise or burst errors.  Without a better understanding
of the present error sources, assessing any scheme is difficult.

A 128k byte table for CRC is not practical for most applications so CRC
requires about 8 instructions per byte with a 1K lookup table.  The modified
Adler-32 where S1 becomes a simple unsigned summation of 16 bit values
seeded with 0x5555 but where S2 retains the Adler modulo only requires about
2 instructions per byte versus about 3 of the prior algorithm.  This
modification reduces the present overhead depending the host to network
endian arrangement and should help with both burst error detection and
smaller packets.  This modification would also reduce the overhead
associated with a hardware implementation of this scheme.

Those working with storage products are expert at assessing burst error
detection and it is not likely any other scheme other than CRC will provide
superior fault coverage for burst errors.  This assessment is not likely
realistic if burst errors or Gaussian noise are not a predominate source.
Virtually any error scheme done in addition to the IEEE CRC found on the
various physical serial transports will provide adequate coverage.  The
problems arise from errors outside this CRC protective blanket.  The
assessment may concern the bit distribution of algorithms and susceptibility
to various pathological patterns.  Again, CRC hold advantages in this area
but it may also become a weakness with respect to some error modes where bit
coloring can be advantageous.

This area is not easily resolved because of the many facets involved.  It
would appear with the problems uncovered with Adler-32, this work should be
revisited and hopefully put to rest.

Doug

---

From: Randall Stewart <[email-masked]>
Date: Thu, 19 Apr 2001 16:20:55 -0500

Q:

Good thoughts except for one minor detail... see below...

> (Maybe after all the Adler-32 in the current RFC2960 is good enough :-)

The major weakness of Adler-32 (if I understand Jonathan and Craig
correctly) is that in
short messages (very common in signalling) the upper two bytes of the
checksum are not
being incremented very often. This is due to the 8bit adds happening at the
input of
data. I am not sure of the strenght of this sum but it is on the order of
weaker than
TCP (if I have heard right). A SIMPLE fix to this is to change the adds to
16 bit
increments.. so input to the c-sum is an array of 16 bit int's instead of 8
bit ints. This
1/2s the work you must do to produce the checksum and causes the upper bits
to
be incremented better... So, I think no matter what we do CRC or checksum a
change is needed. I think to take the minimalist approach changing the
suming to
16 bits is the way to go... Other options such as 32 bit TCP type scares me
a bit
since it will require a few mor operations in any machine that does not have

a 64 bit accumulator... (which is many many many small devices). ...

I see the real issue here is we can either choose a
CRC-32 variant or
one
modified alder..

R