Subject: RE: [Tsvwg] Re: IPsec AH

From: RJ Atkinson <[email-masked]>
Date: Wed, 16 May 2001 12:00:28 -0400

At 11:41 16/05/01, Fairlie-Cuninghame, Robert wrote:

>Also IPSEC AH requires Security Associations for all permutations of source and destination addresses in an association as well as a key-management and distribution mechanisms.

        True in a pedantic sense, but misleading.  IN_ADDR_ANY is 
a possible valid address for any IPsec address binding, either
source or destination.  If the same key (e.g. all 1s) were
used when the goal were checksum but not security, then
key management/distribution is not an issue either.

        That noted, I'm not advocating using AH to provide a
mundane checksum, just noting that the computational costs
of *any* cryptographic MAC will be unduly high when a mundane
checksum is all we need.

        To be honest, I think a Fletcher checksum would be more
than adequate here.  I don't think we need perfection for our
mundane checksums.

Ran

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 17 May 2001 10:58:33 -0700

Ran,

From my testing, Fletcher-16 can only detect one out 50 stuck bit errors.
If you view the checksum's job is to provide a level of protection from
hardware related errors not associated with the actual transfer of the
information which is typically checked with CRC-32, then Fletcher-16 is
perhaps the worst check you can make.  Adler-32 is many many orders of
magnitude better at that function.  Adler-32 is weaker in respect to burst
errors, but is that the check that needs to be made here?  I ague no.  The
digest chunk was defined to allow parity with iSCSI.  I'll make a new
proposal to be a means to select either Adler-32 or CRC-32 with reserve bits
that may be used to track iSCSI if they create their own level of security
that then requires this flexibility.  If you are running digests, then these
checksums are redundant.  I am not suggesting that digests will be used in
preference to simpler checksums if security is not required.  It also allows
checksums to be elevated if the processor can support it.

Doug

> >Also IPSEC AH requires Security Associations for all
> permutations of source and destination addresses in an
> association as well as a key-management and distribution mechanisms.
>

---

From: RJ Atkinson <[email-masked]>
Date: Thu, 17 May 2001 14:33:38 -0400

At 13:58 17/05/01, Douglas Otis wrote:
>If you view the checksum's job is to provide a level of protection from hardware related errors not associated with the actual transfer
>of the information which is typically checked with CRC-32, 
>then Fletcher-16 is perhaps the worst check you can make.  

You do a nice job of begging the question:

        "What problem is the SCTP checksum trying to solve ?"

I don't think that question has actually been sorted out so far,
perhaps because we got bogged down early in the minutiae of
various checksums and their properties.   I do think that
question needs to be sorted out with some SCTP community 
consensus one way or the other before we pick a checksum
algorithm.

        For the problem being addressed by the TCP checksum, the 
existing Fletcher checksum has more than 15 years of operational 
experience indicating that it works JUST FINE.  Theory aside, 
it in fact just works fine for TCP.

> The digest chunk was defined to allow parity with iSCSI.  

Forgive my directness, but:
                - I seriously doubt iSCSI will get their own security
                  mechanism through the IETF approval process.  Just
                  my personal opinion, but I have been watching IETF
                  stuff for the past decade or so...
                - SCTP is not iSCSI; we ought not slavishly repeat
                  their design/architectural mistakes in SCTP.

> I'll make a new proposal to be a means to select either Adler-32 
> or CRC-32 with reserve bits that may be used to track iSCSI 
> if they create their own level of security that then requires 
> this flexibility.  

        I think that interoperability is paramount.  So I think
we really ought to pick exactly *1* checksum to be used with SCTP
and always use it.  I could live with either of those checksums,
if either developed community consensus in future, but PLEASE 
let us just pick *1* and be done with it.

        In many cases, IETF experience is that optional things
don't actually get widely implemented and often lead to 
interoperability problems.  So even if more than 1 checksum
is defined, only *1* ought to be mandatory-to-implement 
(to ensure interoperability).  Note that the optional checksum
probably won't get widely implemented, so it will probably
just be implementation complexity and code footprint that goes
unused.

        To repeat my above comment, I think that it is a serious
mistake to design around the more misguided stuff currently being
marketed in the iSCSI context.  SCTP ought to be able to stand
on its own.  Note that I took roughly the same position in the
past when some folks wanted SCTP to be tuned specially for
SIGTRAN WG applications.

        [Per request from Scott Bradner, I'm moving this thread
to the TSVWG mailing list; replies to tsvwg only, please edit
your headers :-]

Regards,

Ran
[email-masked]

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 17 May 2001 11:11:35 -0700

Sorry,

Fails to detect one of 50 stuck bit errors.

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 17 May 2001 14:02:24 -0700

Ran,

> >If you view the checksum's job is to provide a level of
> >protection from hardware related errors not associated with the
> >actual transfer

If CRC-32 is too difficult to implement in smaller processor environments,
then a simpler checksum process could be used.  In my opinion, Adler-32 is a
dramatic improvement over Fletcher-16.  Sending network data back and forth,
data corruption has been seen by myself often enough to think that things
are not just fine.  I never had that problem with storage devices and that
still amazes me.  It was not that long ago that when storage products
exceeded 20 mega-bytes, CRC-16 was no longer was suitable as a means of data
protection on a sector basis.  CRC-16 is still used on floppies limited to
small sector sizes.  The last time I used CRC-16 was to create erasure flags
for Reed-Soloman algorithms with 3 syndrome packets for every 32 (done in
software albeit at 2mb/s with 386 processors) to improve this floppy
interface integrity for larger capacities.  It also allowed the loss of a
packet without the need to retry.  A network has no natural data size and
concerns get raised based on calculations of soft error rates.  A network is
a difficult thing to assess.  You do not know even the equipment that you
are using or its raw error rate.  Worse yet, you can not be sure of the
types of errors you may see.

> it in fact just works fine for TCP.

I do not agree with this assessment.  If this checking was handling runway
reservations, what error rate would be okay?  If this checking was to ensure
the integrity of your bank accounts, what error rate would be okay?  If it
means you have to down load the file again because the Adler-32 built into
Zip indicated the file was corrupted, perhaps that would be okay.  If it
means you get a strange response that causes you to refresh your browser
page, perhaps that too would be okay.  If a business is using network based
files and storage, what error rate would be okay?  What causes great concern
about Adler-32 is that it is susceptible to 17 bit burst errors.  Now that
is perhaps better than CRC-16, but who said CRC-16 is worth much.  It gets
distilled into the question, does this checking need to protect against
burst errors.  If it does not, would the market place accept this weakness?
I suspect this is a foregone conclusion.  No, if the data matters; yes, if
it does not.  If you are shipping voice packets, then the checking is simply
to ensure the headers are reasonably valid.  Corrupted data is the least of
your concerns.  Perhaps there are indeed two levels of checking that should
be accommodated.

>                   stuff for the past decade or so...

I would love to see IPS use more of the existing architecture.  If that were
to happen, then the IAB or ADs should have provided them more guidance than
they have received if you are right.  As it is now, I suspect that boat has
sailed.  You may wish to stand up to this, but I am not convinced you will
be able to quell their zeal for their own environment.  In addition, IPS
wish to make TCP a framing protocol similar to SCTP.

The draft about to be submitted to tsvwg is
ftp://ftp.ietf.org/internet-drafts/draft-williams-tcpulpframe-01.txt

A quote from this draft:
"The framing connection algorithm is viewed as a transition step between
current deployed TCP protocol stacks and the eventual deployment of SCTP.
Because SCTP supports framing directly, rather than requiring a shim to be
implemented on top of it, it is expected that this framing mechanism will
eventually be replaced by SCTP when SCTP becomes pervasive."

It would seem that IPS can do anything out of the architectural norm they
desire and if David Black's comments are correct, this proposal is about to
become an Informational RFC.  I seriously doubt your doubts.

>                   their design/architectural mistakes in SCTP.

I am not attempting to slavishly repeat anything.  I am only attempting to
remain functionally compatible while using SCTP.  As I have no control over
the architectural directions allowed IPS, their decisions creates problems
with respect as to how SCTP can be made compatible.  I'll take a better look
at the IPSEC proposal, but the bottom line, there will need to be at least
two forms of error checking if CRC-32 is not used.  Until IPS resolves their
security, this too may become an issue.

I have just completed 128 byte packet tests, in 315 million packets, only
the Modified Alder algorithm failed.  Adler-32 and Modifed-Fletcher never
failed to detect the errors.  Fletcher-16 still fails 2% of the time.

Doug

---

From: RJ Atkinson <[email-masked]>
Date: Thu, 17 May 2001 18:31:44 -0400

At 17:02 17/05/01, Douglas Otis wrote:
>dramatic improvement over Fletcher-16.... [stuff deleted here]

        Thanks for making my point.  You dove immediately into 
the minutiae of checksum algorithm alternatives, without addressing 
the firstlevel question I carefully indented for emphasis above.

Let me try to rephrase:

What problem do YOU think the SCTP checksum is trying to solve ?

        I'm interested in an answer that is a problem statement, 
with some bounds, not a discussion of particular algorithms.

>I do not agree with this assessment.  If this checking was handling runway reservations, what error rate would be okay?  

        TCP is currently being used 24x7 in life critical applications
inside the US DoD. So far, people aren't dying because of the TCP
checksum.  Since these applications also run over really noisy
2.4 Kbps HF radio links (and have done for over a decade),
there is some substantial data that Fletcher is OK for the
current application mix.

        Ignoring those, I use NFS/TCP and ftp/TCP all day
and have (the latter at least; the former I used to run NFS/UDP
before NFSv3) for more than a decade.  I haven't lost a file 
yet to checksum failure.  In fact, there aren't even any 
good over-beer stories about losing files sent via TCP 
due to checksum problems.

        So the fact remains clear that the Fletcher checksum
is in fact OK for TCP and its applications.

>they have received if you are right.  

        IAB *has* told them to use more of the existing architecture,
speaking as one member of the IAB.  However, IAB only offers 
advice and does not tell WGs that they "can't" do something.  
IESG manage the standards process day-to-day, not IAB.

>In addition, IPS wish to make TCP a framing protocol similar to SCTP.

        I'm aware there are multiple problems, but yes,
thanks for mentioning that.  If SCTP doesn't tweak itself
in bad architectural ways to accomodate the confusion in
iSCSI-land, then it might help cause the iSCSI folks to 
step back and try to get an architectural reality check.

In any event, bad architecture doesn't justify bad architecture.

>It would seem that IPS can do anything out of the architectural norm they desire and if David Black's comments are correct, this 
>proposal is about to become an Informational RFC.  I seriously 
>doubt your doubts.

        Thanks for the heads up.  I'll be sure to comment
on that when the IETF Last Call happens.

Cheers,

Ran
[email-masked]

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 17 May 2001 20:32:51 -0700

Ran,

I will start with a quote from "When The CRC and TCP Checksum Disagree" by
Jonathan Stone and Craig Partridge:
"Traces of Internet packets from the past two years show that between 1
packet in 1,100 and 1 packet in 32,000 fails the TCP checksum, even on links
where link-level CRCs should catch all but 1 in 4 billion errors.  For
certain situations, the rate of checksum failures can be even higher: in one
hour-long test we observed a checksum failure of 1 packet in 400.  We
investigate why so many errors are observed, when link-level CRCs should
catch nearly all of them.

We have collected nearly 500,000 packets which failed the TCP or UDP or IP
checksum. This dataset shows the Internet has a wide variety of error
sources which can not be detected by link-level checks.  We describe
analysis tools that have identified nearly 100 different error patterns.
Categorizing packet errors, we can infer likely causes which explain roughly
half the observed errors. The causes span the entire spectrum of a network
stack, from memory errors to bugs in TCP.

After an analysis we conclude that the checksum will fail to detect errors
for roughly 1 in 16 million to 10 billion
packets. From our analysis of the cause of errors, we propose simple changes
to several protocols which will decrease the rate of undetected error. Even
so, the highly non-random distribution of errors strongly suggests some
applications should employ application-level checksums or equivalents."

If you wish to open a beer, I have found file errors using DSL connections
that disappear when downloaded over a T1.  I had one only last night. The
ftp completed but the file was corrupt.  Thankfully the executable wrapper
for the application was able to detect the corruption.   Something is not
happy with my DSL link but it is not from equipment that I own.  In talking
to others, they warn me of the DSLAM equipment that may be causing these
errors.

Your positive experience with NFS is likely the result of this being within
a LAN environment were IT managers are doing their job monitoring links.  If
the equipment between two points is good, the media CRC does an excellent
job at keeping data error-free.  As NFS is not checking the integrity of
those files, you would not know if the file was corrupted but you may
experience perhaps an odd array of symptoms but I doubt it would say
checksum error.

In repeating Jonathan's and Craig's summation, [Network related error
distribution] strongly suggests some applications should employ
application-level checksums or equivalents.  If you review the complexity of
iSCSI, much of this is found trying to add a layer of error protection at
the application.  I do not wish to repeat this exercise in building double
layers of error protection.  In my opinion that is the wrong approach.
Moving toward the block level applications, error handling at that level
does not robust and makes additional demands on the transport to ensure
integrity.  It is not my desire to redesign this layer nor to place a second
layer of error checking within a transport.

Once a highly reliable interface is provided connectivity into the WAN
environment, the demands on the transport increase.  As you increase the
data rate of these devices, the likelihood for data corruption within the
various networking components is also likely to increase.  As you increase
the network volume, the level of protection required will increase.
Jonathan and Craig have documented router memory errors within their data
set.  Although you do not like theory, Fletcher-16 will not reliably detect
errors repeated on the same word bit.  This is a real-world problem beyond
the control of those using the WAN.  The only solution for overcoming this
problem is through protocol error checking or additional error checking
within the application.

The problems SCTP checksum will solve:
 - 1 Error checking to be done once within the transport and not the
application.
 - 2 Transport error checking is to provide the level of integrity demanded
by the application.
 - 3 The transport should make few assumptions as to the nature of errors.
 - 4 The consumer is to depend on the transport to ensure integrity.
 - 5 If high levels of data integrity are not required, such as with
switched voice, a simpler algorithm may be used to save software overhead.

CRC-32 can be justified for critical data.  The IPS group also claim that
CRC-32C can be justified by the larger frame size offered by 1GE.  Adler-32
can be justified for applications where data is not critical or the
environment is controllable.  Error checking done only within the transport
can be justified.

I believe your comments about your experiences within LAN environments or
perhaps with WAN connections at the DoD.  As for packet radio links,
remember my concern with this algorithm was with parallel data whereas
serial noise is ideal for this algorithm.  You should look at the level of
error protection that was required to be added to various applications to
guard against data corruption not caught by the transport.  I do not wish to
say that TCP is bad, it is just not good enough for some WAN environments if
the data is not protected by some other means.  If the IETF is to introduce
yet another transport, they should take a careful look at these assumptions.
In the long run, getting this wrong is going to be expensive.

Doug

---

From: Jacob Heitz <[email-masked]>
Date: Thu, 17 May 2001 21:44:43 -0700

We have competing requirements:
Strong error checking vs. fast computation.

Is it possible to get a crc/checksum that is computationally
efficient, yet strong for small frames, but weaker for
larger frames ?

Then those that need strong error checking can set themselves
a small maximum MTU. SCTP will segment and reassemble, so it
should not be a problem. Small MTU gets better throughput in
high error environments anyway.

Those without the strong error checking requirement can then
choose a high maximum MTU and still have a computationally
efficient checksum.

-- 
// Jacob Heitz Tel:510-747-2917

---

From: Lloyd Wood <[email-masked]>
Date: Fri, 18 May 2001 13:34:27 +0100 (BST)

On Thu, 17 May 2001, RJ Atkinson wrote:

> checksum. 

How do you know? please prove that assertion.

> is in fact OK for TCP and its applications.

Clear to you, perhaps. Please define 'OK'.

L.

thinks you should just put an overengineered CRC on it, and be done.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 18 May 2001 11:22:02 -0400

At 23:32 17/05/01, Douglas Otis wrote:

>Your positive experience with NFS is likely the result of this being within a LAN environment 

        Actually it was within a trans-continental US WAN; making
assumptions about another's environment is often risky when one
has no real data.

>In repeating Jonathan's and Craig's summation, [Network related 
>error distribution] strongly suggests some applications should 
>employ application-level checksums or equivalents.  

        Their approach of using application-level integrity 
checks for the very few applications that might need more than 
we currently have seems quite reasonable.  It places the extra
cost of more computationally expensive checks onto only the
few users that need those checks, while avoiding penalising
other more normal users with undue performance penalties.

>If you review the complexity of iSCSI, much of this is found 
>trying to add a layer of error protection at the application.  

        IMHO, much of the complexity has to do with needless bad
architecture over in iSCSI-land.

> - 1 Error checking to be done once within the transport 
>   and not the application.
> - 2 Transport error checking is to provide the level of 
>   integrity demanded by the application.
> - 5 If high levels of data integrity are not required, such as 
>   with switched voice, a simpler algorithm may be used to save 
>   software overhead.

        Thanks for writing those 5 down.  It is always helpful to
keep that basic problem statement information documented in I-Ds
and RFCs so that everyone can sync up on why the document exists.

        IMHO, the *WG as a whole* need to mull over whether they 
agree that those are the right objectives for SCTP and mull over 
how much computational penalty is appropriate to burden on all 
users of SCTP.

        If the WG decides to have multiple algorithms, then 
the simpler (less computationally expensive) algorithm should be 
the only one that is mandatory to implement, again keeping to the
notion that normal users should not all be penalised because of the
abnormal requirements of one or two not widely used applications.

>CRC-32 can be justified for critical data.  The IPS group also 
>claim that CRC-32C can be justified by the larger frame size 
>offered by 1GE.  Adler-32 can be justified for applications 
>where data is not critical or the environment is controllable.  
>Error checking done only within the transport can be justified.

        IFF the group as a whole concurs, then I'd say that Adler-32
should be mandatory-to-implement and default to use, with CRC-32C 
being purely optional to implement or use.  There will need to
be Sockets API changes to support multiple algorithms, if the
WG makes the decision to have multiple algorithms.

>You should look at the level of error protection that was 
>required to be added to various applications to guard against 
>data corruption not caught by the transport.  

        Nothing special was needed.

>I do not wish to
>the data is not protected by some other means.  

        Operational evidence for 10-15 years indicates that
TCP is JUST FINE for the wide range of environments it has been
used in so far.

>If the IETF is to introduce yet another transport, they should 
>take a careful look at these assumptions.  In the long run, 
>getting this wrong is going to be expensive.

        I agree with that the issue needs more thought -- in 
particular using a checksum that is way overkill for the wide range 
of current applications and adding significant computational burden 
on small embedded systems could significantly impair the ability 
of IP to continue to be used in many environments where it exists
nicely today.

        What is more interesting to me at this point is to hear
from other folks in the working group about what their views
are about this topic, including at least:
        - does the SCTP checksum need to be changed ?
        - should there be more than one checksum in SCTP ?
        - if there are multiple, which *1* should be
          mandatory to implement ?
        - if we need a new checksum (or new checksums),
          which ones are appropriate ?
        - are the 5 objectives outlined above the right ones ?
        - should we distort and twist SCTP just so that folks
          in iSCSI and IPS can avoid putting appropriate functions
         into their extremely unusual and unique and not widely
          deployed (now or likely in future) application protocols ?

        I'd encourage other folks to comment to the list so
that some WG consensus could be more apparent...

Cheers,

Ran
[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Fri, 18 May 2001 14:29:31 -0500

....
> > - 1 Error checking to be done once within the transport

I remember there was a discussion thread [~4/19 on this list] about the
above topic and the conclusion seemed to be that since the potential
involvement of middle boxes in the data path the application should
never count on the transport error checking along for "end-to-end" data
integrity.

> >   integrity demanded by the application.

Which kind of application? We all know different application demands
different level of integrity.

> > - 3 The transport should make few assumptions as to the

Before we can come up with a universally good yet cheap-to-implement
solution, I'd say this is just a wishful thinking.

> > - 4 The consumer is to depend on the transport to ensure integrity.

See my comments to 1 above.

> >   with switched voice, a simpler algorithm may be used to save

True, in fact there are increasing number of media applications that do
NOT want any transport level error checking (e.g., the UDP-Lite story).

-Qiaobing

---

From: "Randall R. Stewart" <[email-masked]>
Date: Fri, 18 May 2001 14:40:48 -0500

> thinks you should just put an overengineered CRC on it, and be done.
Lloyd:

I won't let you off that easy. if you are going to "think that
we should put an overengineered CRC on" you must declare which
one :)

R

-- 
Randall R. Stewart
[email-masked] or [email-masked]
815-342-5222 (cell) 815-477-2127 (work)

---

From: "Randall R. Stewart" <[email-masked]>
Date: Fri, 18 May 2001 14:51:59 -0500

Ran:

I am normally in agreement with you but
I have a couple of nits on some of
your suggestions... see below..

>         - does the SCTP checksum need to be changed ?

If the answer is NO, we can throw away the BIS and hold
off on all the changes from the bake-offs and wait to go
to draft standard before incorporating them ... My feeling
was that the answer though is YES due to Jonathan and Craigs
Small packet problem with the current Adler-32...

>         - should there be more than one checksum in SCTP ?

No, I am strongly against this. Chip and I wrote a draft on
this (we have it somewhere) and it becomes a NIGHTMARE to support.
Even worse when one wants to do hardware acceleration.. We need
to pick ONE and only ONE checksum. It should NOT be general
enough but not weak... I know its tough to find this but or
even get close to agreeing ... since we will get the iSCSI folks
demanding CRC-32C and the other side pushing for something more
along the lines of the current TCP sum... sigh... I really don't
care but thats probably because I don't think I know which one
is best :/

>         - if there are multiple, which *1* should be

See above... multiple is a bad bad thing... if you want
to do multiple do it has a seperate extension that no one
has to implement but those who want it. Burn up a chunk 
or something but please please don't do 2 in the base
protocol..

>           which ones are appropriate ?

This is the question that I would love answered... I know I don't
know...

>         - are the 5 objectives outlined above the right ones ?

I don't think so... they are a start but I fall more along the
lines of Qiaobings response...

>          into their extremely unusual and unique and not widely

NO, NO NO. Give them an option or let them do something at the
application level. This whole thing to me involves NOT what 
iSCSI wants but instead what Craig and Jonathan found on small
packets..

> that some WG consensus could be more apparent...

the more comments the better but history tells me we will
not get as many as I would like to see :<

R

---

From: Shyamal Prasad <[email-masked]>
Date: 18 May 2001 16:37:12 -0500

"Randall" == Randall R Stewart <[email-masked]> writes:

    Randall> RJ Atkinson wrote:
    >> 
    >> I'd encourage other folks to comment to the list so that some
    >> WG consensus could be more apparent...

    Randall> the more comments the better but history tells me we will
    Randall> not get as many as I would like to see :<

Hey Randy, some of us read this stuff, but unfortunately have little
information to add. But here's one comment towards your wish.....;-)

I've seen no evidence that the current 16 bit IP checksum has resulted
in a noticeable failure of the Internet. Taking that as a cue, and the
fact that no one has said that the Adler32 we currently use in SCTP is
WORSE than current TCP/UDP checksums, I'm unable to understand the
excitment about Adler32.

This might just be my ignorance, but I've seen no evidence that the
current SCTP checksum is *broken* for transport of data that we send
over TCP/UDP today. There have only been statements that there are
*better* checksums for "small" messages (which is what?  I saw a 128
byte figure somewhere).

It would be really nice if some one could describe precisely (well,
probabilistically) what we would gain by leaving Adler32 for, say, 16
or 32 bit Fletcher as you suggested in one of your posts. Unless there
is a checksum that is as fast as, and all around superior to Adler32,
I don't see any reason to change. Count me in as one of ignorants on
the sidelines until I understand this ;-)

Is this just an iSCSI problem, or something the rest of us 16 bit
UDP/TCP checksumming louts should worry about? Why replace one
checksum with another? 

Cheers!
Shyamal

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 18 May 2001 18:57:29 -0400

>care but thats probably because I don't think I know which one

        I agree that more than one checksum is not desirable.

>... but I fall more along the lines of Qiaobings response...

        Agree.

>iSCSI wants but instead what Craig and Jonathan found on small

        Agree.  Lets all drop iSCSI and IPS as reasons to change
SCTP starting now.  We need to design SCTP for the general case.
Applications that need special stuff can use application-layer
mechansims to provide that special stuff.

>the more comments the better 

        Agree.  More input really is better.

Ran
[email-masked]

---

From: "Douglas Otis" <[email-masked]>
Date: Fri, 18 May 2001 15:50:48 -0700

Shyamal,

Moving into the next 15 years with a new standard transport, there are a few
aspects that should be considered.  This SCTP transport, unlike TCP, is
suitable for automation and will likely be one of the first to see 10 GE.
In hardware, CRC is a no-brainer and is highly preferred.  This transport
should also consider the weakness in the existing infrastructure when
supporting 10 GE at 8k packets due to a weakness in media CRC-32.  Not being
able to depend on the transport to delivery valid data is a tremendous
architectural flaw.  The level of complexity of checking and then rechecking
together with a second tier of acknowledgements as now even an
acknowledgement from the transport is depreciated due to a weakness in data
integrity makes this technology far less attractive for system reliance.

It would be nice if the Telco equipment all used error checking between
interfaces, the routers and translators all ensured memory integrity, and
the amount of data remained relatively small.  This is not the reality today
nor likely within the next 15 years.  Even UDP can ignore the checksum and
for some applications, this is a valuable feature as the raw error rate of
the network is adequate for many applications carrying non-critical data.
Over a LAN environment, if there is a component problem, the user is
responsible for any resulting data corruption.  Over the WAN, the underlying
equipment may be changing on a minute by minute basis providing unknown
integrity.  The only defense in this environment is the transport error
check or the application is forced to recheck and re-acknowledge data.

Why should "okay for voice or multimedia presentations" set the standards
for error checking?  Should a transport be designed to support the simple
applications and expect critical applications to suffer the consequence of
replicate checking and acknowledgement?  I find it hard to reconcile the
need for minimal error checking and assured data integrity without allowing
an option within the transport.  If the transport is handled by a processor
that is heavily burdened doing CRC, then silently discarding packets with
the option set would be a low effort negotiation.   It would seem this
option could be enabled at the system level.

Adler-32 is the same as Fletcher-16 for small packets where both result in a
32 bit checksum.  Adler-32 is more susceptible to burst errors and
Fletcher-16 is highly susceptible to stuck bit errors.  Adler-32 is a bit of
a pain to implement in hardware but not impossible.  CRC-32 takes about 4
more instructions per byte to implement in software and depending on the
processor cache size and relative bus speed, this may not be noticeable.  If
done in hardware, it takes less effort.  All of these provide a 32 bit
result.  Does IETF wish to provide assurance of data integrity, or should
those using these transports be expected to design their own protocol on top
of these IETF protocols.  I do not think that is a good choice as the
protocol itself is the real burden and forcing it to be done twice is
unconscionable.

One additional technique may be to zero the checksum field and add four
bytes to the packet to signal the use of CRC.

Doug

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 18 May 2001 18:55:15 -0400

At 15:29 18/05/01, Qiaobing Xie wrote:
>I remember there was a discussion thread [~4/19 on this list] 
>about the above topic and the conclusion seemed to be that 
>since the potential involvement of middle boxes in the data path 
>the application should never count on the transport error 
>checking along for "end-to-end" data integrity.

        This is an excellent point and it TOTALLY demolishes
any argument that relies on iSCSI or IPS to justify changing
the checksum or to justify a computationally expensive checksum,
IMHO.

>different level of integrity.

        Agree.  I'd go with the typical *existing* application
as where we should target our checksum.  If some application
needs better, it ought rely on application-layer protections
-- see again the middle-box comment above.

        Agree.

        Agree.

>True, in fact there are increasing number of media applications that do NOT want any transport level error checking (e.g., the UDP-Lite story).

        For interoperability reasons, I think its best to always
have a checksum.  The historical experience of UDP with checksums
vs UDP without checksums leads to the conclusion that that sort
of option creates interoperability issues that are needless.

        Thanks much for the input.

Ran
[email-masked]

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 18 May 2001 19:27:46 -0400

At 18:50 18/05/01, Douglas Otis wrote:
>This SCTP transport, unlike TCP, is suitable for automation and 
>will likely be one of the first to see 10 GE.

        Packet Engines, which was affiliated with SGI, had TCP
implemented in silicon in the very early 1990s.  TCP too
could be put into silicon.  Packet Engines discovered that
there wasn't enough of a marketplace for their protocol ASICs
at that time, hence they seem to have gone away.

        Curiously, a lot of your arguments sound like those
made by XTP advocates in the late 80s and early 90s.

>In hardware, CRC is a no-brainer and is highly preferred.  

        Arguable that it is "highly preferred" by ASIC designers.

>This transport
>supporting 10 GE at 8k packets due to a weakness in media CRC-32.  

        IEEE has not changed the Ethernet MTU size for 10 GigE.
The proposal was made, but was not accepted by IEEE.  No doubt
some vendors will support a ~8K MTU despite this.

>Not being able to depend on the transport to delivery valid data 
>is a tremendous architectural flaw.  

        The data so far is that Fletcher as used in TCP works fine.
If it didn't, the Internet wouldn't have been this successful.

>Should a transport be designed to support the simple applications 
>and expect critical applications to suffer the consequence of
>replicate checking and acknowledgement?  

        Common applications don't need the level of protection
you are advocating.  Specialised applications should not force
all applications to pay the penalty/costs associated with 
properties not needed by most applications.  That's part of
making good engineering tradeoffs in system design.

>Does IETF wish to provide assurance of data integrity, or 
>should those using these transports be expected to design 
>their own protocol on top of these IETF protocols.  

IETF has historically said that the few applications with 
unusually high requirements should not impose the costs 
of those requirements on all other applications.

Use of application-layer extensions for checksums or other
properties for applications with unusual needs is an accepted 
part of the Internet Architecture, IMHO.

Ran
[email-masked]

---

From: "Randall R. Stewart" <[email-masked]>
Date: Fri, 18 May 2001 18:28:19 -0500

Comments below and something for
Craig and Jonathan...

> the sidelines until I understand this ;-)

This all started when Craig and Jonathan put forward that the
Adler-32 WAS weaker for small packets. At least that is
my understanding... Jonathan and Craig will have to explain...

In effect being weaker for small packets causes great problems
since this is what are first target was.. signalling messages...

Now I really don't care if Adler-32 is not strong enough for
iSCSI... They can do something at the app layer... BUT if
Adler-32 IS weaker than UDP/TCP in small packets then I think
we do need to do something about it... I put fletcher-16
(which is a 32 bit sum) in the current BIS since it is already 
an option for TCP...

Jonathan/Craig... could you elaborate the problem for
small packets....

R

---

From: "Douglas Otis" <[email-masked]>
Date: Fri, 18 May 2001 17:39:55 -0700

Ran,

> made by XTP advocates in the late 80s and early 90s.

I wonder what point you are making.  You are saying that I am not the first
to raise this concern or that automation will not happen?  Packet Engines
was acquired by Alcatel for $315 million.  With respect to high data rates,
there are few alternatives to automation.

>         Arguable that it is "highly preferred" by ASIC designers.

I'll concede that it may only be preferred.

> some vendors will support a ~8K MTU despite this.

If you check the marketplace, ~8K MTU is the standard offering for 1GE
adapters.  Are you suggesting these features should not be used?  Using the
end-to-end check within the transport is a means to enable this option
safely.  Are you suggesting this is not important to support?

> If it didn't, the Internet wouldn't have been this successful.

Sigh.  One can only imagine the success data integrity at the transport
would allow a WAN application without inordinate layering of additional
protocol error checking and re-acknowledgements.  All this because there was
not a perceived need for a high level of integrity at the transport.  The
defensive stance already used with respect to integrity has not been helpful
in that cause.  It is because of the success of the Internet, this issue
becomes even more urgent.

> making good engineering tradeoffs in system design.

You are not allowing a tradeoff by preventing the transport from providing
integrity out of a sense of elegance or complacence.  Data integrity becomes
substantially more expensive within a WAN environment if there is a need to
implement a second tier of transactions in such applications.  There are few
applications that should turn a blind eye to data integrity while people
have become accustom to low-grade results such as programs that exhibit
fatal exceptions or files that are corrupted during a transfer.

> of those requirements on all other applications.

I would suspect few would think that a data integrity requirement is
unusually high.  Either you can provide it, or you can't.

Doug

---

From: Jonathan Stone <[email-masked]>
Date: Fri, 18 May 2001 19:42:09 -0700

>Jonathan/Craig... could you elaborate the problem for

Briefly, the problem is that, for very short packets, Adler32 is
guaranteed to give poor coverage of the available bits.  Don't take my
word for it, ask Mark Adler. :-).

Adler-32 uses two 16-bit counters, s1 and s2.  s1 is the sum of the
input, taken as 8-bit bytes.  s2 is a running sum of each value of
s1. Both s1 and s2 are computed mod-65521 (the largest prime less than
2^16).  Consider a packet of 128 bytes. The *most* that each byte can
be is 255.  There are only 128 bytes of input, so the greatest value
which the s1 accumulator can have is 255 * 128 = 32640.  So for
128-byte packets, s1 _never_ wraps.  That is critical. Why?

The key is to consider the distribution of the s1 values, over some
distribution of the values of the individual input bytes in each
packet.  Because s1 never wraps, s1 is simply the sum of the
individual input bytes. (even Doug's trick of adding 0x5555 doesn't
help here, and an even larger value doesn't really help: we can get at
most one mod-565521 reduction).

Given the further assumption that the input bytes are drawn
independently from some distribution (they probably aren't: for
filesystem data, it's even worse than that!), the Central Limit
Theorem tells us that that s1 will tend to have a normal distribution.
That's bad: it tells us that the value of s1 will have hot-spots at
around 128 times the mean of the input distribution: around 16k,
assuming a uniform distribution.   That's bad. We want the   accumulator
to wrap as many times as possible, so that the resulting sum has 
as close to a uniform distribution as possible. (I call this "fairness").

So, for short packets, the Adler-32 s1 sum is guaranteed to be unfair.
why is that bad? it's bad because the space of valid packets-- input
data, plus checksum values -- is also small. If all packets have
checksum values very close to 32640, then the likelihood of even a
`small' error leaving a damaged packet with a valid checksum is
higher than if all checksum values are  equally likely.

It's not possible to say exactly how bad this effect is, without
access to traces of signalling-protocol packets to feed into a
simulation.  The only high-quality data I have is on the contents of
files drawn from filesystems.

Last: If the bytes are not independent, but positively autocorrelated
within a packet, that just makes things even worse: intuitively, that
makes the s1 value approach a normal distribution even more tightly
than the Central Limit theorem suggests.

The story for s2 is more complicated; I'd prefer to leave
that for now.

Craig and I both did some fairly crude simulations back in January. I
still have some results about how the second-order sum (s2) is
distributed, given uniform input.  I'm not sure whether i still have
on the distribution of s1; anything I do still have would be on
archival storage.  And it would be low-quality: small samples.

I never went back and did high-quality simulations of 128-byte
filesystem data, as I didn't (and still don't) know enough about
telephony signalling in general (or SS7 in particular) to know whether
filesystem data are a suitable proxy for signalling packets.  
If it is, I can certainly look at the distribution of Adler32 sums
over very short `packets' drawn from filesystem data.
I will  probably be doing that sometime in June anyway.

hope that helps.  For more detail than that, I'll have to start
circulating around postscript copies of the draft chapter
from my dissertation.

--Jonathan

---

From: Lloyd Wood <[email-masked]>
Date: Sun, 20 May 2001 22:37:15 +0100 (BST)

> we should put an overengineered CRC on" you must declare which

CRC-32.

CRC-16 just isn't overengineered _enough_.

Given that protocol standards should last last longer than individual
implementations in endhosts and that endhost processing power
increases over time, I simply don't buy any 'computational overhead'
argument.

Taking a programming complexity viewpoint, moving between more than
one different type of checksum under certain conditions (length of
packet, say, as has been suggested) just lends itself to host of
complex interoperability problems if someone mistakes a >= for a >.
Don't go there.

Taking a programming comprehensibility viewpoint, more people are
likely to know how to or find out how to implement a CRC than some
tweaked variant checksum that we invent or decree gets implemented for
the first time here, only to discover the flaws in same several years
down the line.

Either SCTP is a reliable protocol, or it isn't. If it isn't and you
think applications using SCTP should implement their own additional
checksums if they need to, why are you implementing anything at all?

If you're going to implement a 32-bit integrity check, make it CRC.

L.

this is, no doubt, a minority viewpoint.

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 21 May 2001 11:52:14 -0500

Lloyd Wood wrote:
...snip...
> checksums if they need to, why are you implementing anything at all?

I wonder whether this is a legitimate question. Just try to ask the same
question about the media layer..

-Qiaobing