Subject: [Sigtran] Connect-a-thon SCTP testing summary

From: Randall Stewart <[email-masked]>
Date: Thu, 07 Mar 2002 01:57:59 -0600

Greetings:

We have now reached the conclusion of the SCTP testing at 
connect-a-thon and Michael Tuexen and I decided we had best
write a summary of all that has transpired.

We have had 6 company's with 9implementations.

Using RFC1025 we extrapolated a similar set of point scoring
as follows:

10 points for a complete conversation with another
   implementation. This involves setting up the association,
   data transfer in both directions and a graceful shutdown... followed
   by a repeat of the procedures with opposite end initiating the 
   events (e.g. the one that sends the INIT first in the second
   pass receives the INIT).
  
10 points for handling out of order data.

10 points for correct handling of sequence number wrap around (both
   TSN and SSN)

10 points for multiple connections at the same time.

10 points for handling duplicate SACK without a false fast retransmit.

10 points for show that a peer does not handle the checksum (version
   -03 of the draft) correctly.

10 points for performing ADD-IP correctly.

10 points for successful U-SCTP.

20 points for handling a SCTP X-mas tree packet containing either
   (Cookie-ack, ECN, SACK, ASCONF, Unrecognized Chunk, SHUTDOWN)
     or
   (Cookie-ack, SACK, ASCONF, Unrecognized Chunk, SHUTDOWN) if
   the receiver does not handle ECN.

10 extra points for bundling all of the X-mas tree response in
   one packet.

20 points knocking out a peer with a non-valid SCTP packet.

30 points for knocking out a peer with a valid SCTP packet.

The scoring results are as follows:

Implementation Number              Score

1                                  275
2                                  225
3                                  125
4                                   60
5                                   30
6                                   20
7                                   20
8                                   20
9                                   10

We had three implementations that correctly performed U-SCTP and
two implementation that handled the Add-IP/ASCONF draft. 

We found no real specification issues with RFC2960. There was
quite a bit of churn at the beginning to get the new checksum in
place for all testing however. The checksum that was used is the
new draft that was just submitted (and until the draft hits
the IETF web site can be found at http://www.sctp.org).

One predominant theme that many implementations did have an
issue with was the proper handling of unrecognized chunks. In
particular the upper two bits for reporting and skipping were
not always handled correctly. 

In general the bake-off went well considering that many of the
implementations were quite new and kernel implementations.

Randall and Michael

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

---

From: Randall Stewart <[email-masked]>
Date: Fri, 08 Mar 2002 08:08:13 -0600

Hmm.. my mail server seems not to have
forwarded this for some reason.. attempt number two :>

R