Subject: SCTP socket API and M2PA changeover

From: [email-masked]
Date: Tue, 30 Oct 2001 12:47:55 +0530

Hi ,

This is with reference to the "Sockets API Extensions for SCTP  " draft, July
2001.

It lacks complete support for undertaking changeover by M2PA.
For instance, it does not allow unsent/unacked data to be retrieved from SCTP
for a given socket.

If the draft(socket api)  is subject to change, we should include such a support
otherwise M2PA will be
unable to integrate with a standard socket based SCTP kernal implementation.

regards
Harsh

---
You are currently subscribed to sigtran as: [email-masked]
To unsubscribe send a blank email to [email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 01:24:13 -0600

Harsh,

[email-masked] wrote:                              (Tue, 30 Oct 2001 12:47:55)
> unable to integrate with a standard socket based SCTP kernal implementation.

When did this INFORMATIONAL INTERNET-DRAFT become a "standard"? ;)

It is clear that M2PA needs an application-level acking function
for other reasons and that SCTP retrieval will not be necessary (as
M2PA can hold its own copy of unacknowledged messages).

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 13:16:51 +0530

Hi,
  brian see the comments  manish>>>>>

"Brian F. G. Bidulock" <[email-masked]> on 10/30/2001 12:54:13 PM

Please respond to [email-masked]

To:   "sigtran" <[email-masked]>
cc:    (bcc: Manish Sharma/HSS)

Subject:  Re: SCTP socket API and M2PA changeover

Harsh,

[email-masked] wrote:                              (Tue, 30 Oct 2001
12:47:55)

It is clear that M2PA needs an application-level acking function
for other reasons and that SCTP retrieval will not be necessary (as
M2PA can hold its own copy of unacknowledged messages).
manish >>>>>>but brian till now we don't have sequence number at m2pa level.
             and it doesn't make any sense to make queue at m2pa level.
             Tom told me that you are in support for using sequence no.
             at m2pa level but we are getting support from SCTP itself.
             can you explain me why to use sequence no. at m2pa level.

--brian

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---
You are currently subscribed to sigtran as: [email-masked]
To unsubscribe send a blank email to
[email-masked]

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 13:47:12 +0530

Talking of queues at M2PA level suggests that we are trying to mimick MTP2 ,
which in my opinion is incorrect as we already have reliable service from
SCTP as transport protocol for M2PA.
Also, how is M2PA supposed to know which message is acknowledged and
which is unacknowledged to maintain these queues!!

---Vipin

To:   "sigtran" <[email-masked]>
cc:    (bcc: Vipin K Gupta/HSS)

Harsh,

--brian

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---
You are currently subscribed to sigtran as: [email-masked]
To unsubscribe send a blank email to
[email-masked]

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 13:55:23 +0530

Hi Brian,

     I see no value addition to the M2PA Stack by
adding Seq. Numbers, moreover i see an additional
overhead.

We will first acknowledge messages at the SCTP level.
then these messages are sent to the M2PA Queue.
There we will again ACK these messages at the M2PA level.

This is required in MTP2 since MTP2 does not have a Reliable medium
to transfer messages, Thereby Compelling MTP2 to maintain Queues
and implement it's own mechanish to Ensure Delivery.

Since M2PA has an Already proven and reliable medium.
I do not find any positive traits by adding SSN to the M2PA Stack.

regards
Dipak

[email-masked] on 10/30/2001 01:47:12 PM

To:   "sigtran" <[email-masked]>
cc:    (bcc: Dipak Aggarwal/HSS)

---Vipin

Harsh,

--brian

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---
You are currently subscribed to sigtran as: [email-masked]
To unsubscribe send a blank email to
[email-masked]

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 03:53:56 -0600

Dipak, Vipin, Harsh,

The difficulty has been presented before, but I will present
it one more time...

There are two situations where SCTP/M2PA cannot provide the
the services expected of MTP2 from MTP3: BUSY or LPO resulting
in changeover.  Here is an example message flow:

     M2PA       SCTP                       SCTP       M2PA     
       |          |                         |          |       
 _MSU1_|___DATA___|______                   |          |       
       |          |      \                  |          |       
       |          |       \                 |          |       
 _MSU2_|___DATA___|______  \                |          |       
       |          |      \  \               |          |       
       |          |       \  \              |          |       
 _MSU3_|___DATA___|______  \  \             |          |       
       |          |      \  \  \            |          |       
       |          |       \  \  \           |          | MSU1  
 _MSU4_|___DATA___|______  \  \  \----------|----------|------>
       |          |      \  \  \____SACK___/|   DATA   |  LPO  
       |          |       \  \ /\ x+++++++++|++++++++++|++++++<
 _MSU5_|___DATA___|______  \  X  x----------|--------->|RB     
       |          |      \  \/ \x           |   DATA   |       
       |          |       \ /\ x\           |          |       
 _MSU6_|___DATA___|______  X  x  \----------|--------->|RB     
       |          |      \/ \x \____SACK___/|   DATA   |       
       |          |<-----/\ x\ /\           |          |       
 _MSU7_|___DATA___|______  x  X  \----------|--------->|RB     
       |          |      \x \/ \            |   DATA   |       
 <+++++|++++++++++|++++++x\ /\  \           |          |       
   RPO |    ^     |        X  \  \----------|--------->|RB     
       |    |     |       / \  \____SACK___/|   DATA   |       
       |    |     |<-----/   \ /\           |          |       
       |    |     |           /  \----------|--------->|RB     
       |    |     |          / \            |   DATA   |       
       |    |     |         /   \           |          |       
       |    |     |        /     \----------|--------->|RB     
       |    |     |       /     ____SACK___/|   DATA   |       
       |    |     |<-----/     /            |          |       
       |    |     |           /             |          |       
       |    |     |          /              |          |       
       |    |     |         /               |          |       
       |    |     |        /                |          |       
       |    |     |       /                 |          |       
       |    |     |<-----/                  |          |       
       |    |     |                         |          |       
       |    |    RTX                        |          |       
       |    |   Empty                       |          |       
       |    |     |                         |          |       
      ...   |    ...                       ...        ...      
       |    |     |                         |          |       
       |    |     |                         |          |       
       |    |     |                         |          |       
       |    |     |               __________|__________|_RESUME_
       |    |     |              /          |          |       
       |    |     |             /           |       Flush RB   
       |    |     |            /            |          |       
       |    |     |  ABORT    /             |        MSU2      
       |    |     |    or    /              |         to       
  _OOS_|____v_____|___OOS   /               |        MSU7      
       |          |      \ /                |      Discarded   
       |          |       X                 |          |       
       |          |      / \                |          |       
       |          |         \               |          |       
   RETRIEVE       |          \              |          |       
  ---->|--------->|           \             |          |       
       |        TX/RTX         \            |          |       
   Nothing to    are            \           |          |       
   Retrieve     Empty            \          |          |       
  <----|<---------|               \         |          |       
       |          |                \________|          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       
       |          |                         |          |       

The diagram illustrates that and indeterminate number of MSUs
are discarded for changeover resulting from LPO.  The same is
true also for BUSY, but more messages are discarded.

In SS7 no messages are lost in this scenario because MTP2 start
witholding acknowledgements immediately upon LPO or receive
congestion discard.  Q.782 conformance (changeover test cases)
requires zero message lost in this scenario.

MTP/M2PA/SCTP cannot meet this Q.782 conformance case.
Retrieval is of no use in these scenarios because by the time
that retrieval is performed, SCTP has transferred all its
transmit messages to the other end and there is nothing left to
retrieve.

Acks at M2PA level with RTX buffering at M2PA removes the
difficulty.   Dorking with a_rwnd and SACKs is useless (and
maybe dangerous) and is anyway likely not supported by most SCTP
implementations.

In light of this, please see comments below...

--brian

[email-masked] wrote:          (Tue, 30 Oct 2001 13:55:23)
> 
> Hi Brian,
> 
> I see no value addition to the M2PA Stack by adding Seq.  Numbers,
> moreover i see an additional overhead.

MTP will otherwise fail Q.782 conformance under load.  It is of
great value to permit MTP to conform.  The overhead is minute.

> We will first acknowledge messages at the SCTP level.  then these
> messages are sent to the M2PA Queue.  There we will again ACK
> these messages at the M2PA level.

SCTP must ack for RTT.  M2PA will not.  M2PA will ack when it
knows that the message is deliverable to MTP3.  This is
essential to permit MTP3 to meet Q.782 conformance.

> This is required in MTP2 since MTP2 does not have a Reliable
> medium to transfer messages, Thereby Compelling MTP2 to maintain
> Queues and implement it's own mechanish to Ensure Delivery.

In contrast, MTP2 does not calculate RTT on ack, it uses queue
depths to detect congestion.  MTP2 uses acks only to tell the
peer that it can strike the message from its RTB and also that
it is no longer necessary to keep the message available for
retrieval.  SCTP has no concept of changeover between
associations: it uses acks to calculate RTTs and for simple
retransmission only.  It was never intended that SCTP changeover
between associations.

> Since M2PA has an Already proven and reliable medium.  I do not
> find any positive traits by adding SSN to the M2PA Stack.

Do you now see why sequence numbers are necessary?  It has
nothing to do with the reliability of the medium, and a lot
to do with the lack of assurance of the communication between
MTP2 and MTP3 (LPO/BUSY).

> Dipak

[snip]
> 
> Talking of queues at M2PA level suggests that we are trying to
> mimick MTP2 , which in my opinion is incorrect as we already have
> reliable service from SCTP as transport protocol for M2PA.  Also,
> how is M2PA supposed to know which message is acknowledged and

Does it make sense now?

[snip]
> > This is with reference to the "Sockets API Extensions for SCTP
> > " draft, July 2001.
> >
> > For instance, it does not allow unsent/unacked data to be
> > retrieved from SCTP for a given socket.
> >
> > If the draft(socket api)  is subject to change, we should
> > include such a support otherwise M2PA will be unable to
> > integrate with a standard socket based SCTP kernal
> > implementation.
> >
> 
> for other reasons and that SCTP retrieval will not be necessary
> (as M2PA can hold its own copy of unacknowledged messages).
> 
> --brian
> 

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 15:56:02 +0530

Brian,

Your figure assumes that M2PA has its own queues (RB in figure) and
it keeps on receiving messages from the SCTP even after MTP3 has
given an LPO indication to it.
In my opinion it is purely an implementation issue, I agree that the send
queue of sending SCTP will become empty, but if the M2PA does not
accept messages from SCTP after the LPO indication then SCTP at the
receiving end is going to maintain these messages in its Receive queue.
    After receiving _RESUME_ from the MTP3, if M2PA retrieves the data from
SCTP and passes it to MTP3, then no MSUs will be discarded.

--- Vipin

"Brian F. G. Bidulock" <[email-masked]> on 10/30/2001 03:23:56 PM

Dipak, Vipin, Harsh,

     M2PA       SCTP                       SCTP       M2PA
       |          |                         |          |

--brian

[snip]

Does it make sense now?

[snip]

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 04:33:49 -0600

Vipin,

[email-masked] wrote:                                (Tue, 30 Oct 2001 15:56:02)
> given an LPO indication to it.

No assumption made.  See redrawn diagram.  It does not matter
where they are buffered.

> SCTP and passes it to MTP3, then no MSUs will be discarded.

No, then it will violate Q.782 in two ways instead of one.  RB contents
must be flushed on resume, otherwise, the MSUs will be delivered after
MSUs sent on the new link (the one the transmitter changed over to) and
mis-sequencing will also occur.  (Mis-sequencing is considered worse in
SS7 than discard.)

You have no solution to this difficulty and calling it implementation
dependent is an inadequate dodge.  Do you see a way other than M2PA-
level acks of getting the sender to hold a copy of MSUs 2 through 7
in preparation for changeover?

--brian

      M2PA       SCTP                       SCTP       M2PA
        |          |                         |          |
  _MSU1_|___DATA___|______                   |          |
        |          |      \                  |          |
        |          |       \                 |          |
  _MSU2_|___DATA___|______  \                |          |
        |          |      \  \               |          |
        |          |       \  \              |          |
  _MSU3_|___DATA___|______  \  \             |          |
        |          |      \  \  \            |          |
        |          |       \  \  \           |          | MSU1
  _MSU4_|___DATA___|______  \  \  \----------|----------|------>
        |          |      \  \  \____SACK___/|   DATA   |  LPO
        |          |       \  \ /\ x+++++++++|++++++++++|++++++<
  _MSU5_|___DATA___|______  \  X  x----------|RB        |  
        |          |      \  \/ \x           |          |
        |          |       \ /\ x\           |          |
  _MSU6_|___DATA___|______  X  x  \----------|RB        |  
        |          |      \/ \x \____SACK___/|          |
        |          |<-----/\ x\ /\           |          |
  _MSU7_|___DATA___|______  x  X  \----------|RB        |  
        |          |      \x \/ \            |          |
  <+++++|++++++++++|++++++x\ /\  \           |          |
    RPO |    ^     |        X  \  \----------|RB        |  
        |    |     |       / \  \____SACK___/|          |
        |    |     |<-----/   \ /\           |          |
        |    |     |           /  \----------|RB        |  
        |    |     |          / \            |          |
        |    |     |         /   \           |          |
        |    |     |        /     \----------|RB        |  
        |    |     |       /     ____SACK___/|          |
        |    |     |<-----/     /            |          |
        |    |     |           /             |          |
        |    |     |          /              |          |
        |    |     |         /               |          |
        |    |     |        /                |          |
        |    |     |       /                 |          |
        |    |     |<-----/                  |          |
        |    |     |                         |          |
        |    |    RTX                        |          |
        |    |   Empty                       |          |
        |    |     |                         |          |
       ...   |    ...                       ...        ...
        |    |     |                         |          |
        |    |     |                         |          |
        |    |     |                         |          |
        |    |     |               __________|__________|_RESUME_
        |    |     |              /          |          |
        |    |     |             /           |       Flush RB
        |    |     |            /            |          |
        |    |     |  ABORT    /             |        MSU2
        |    |     |    or    /              |         to
   _OOS_|____v_____|___OOS   /               |        MSU7
        |          |      \ /                |      Discarded
        |          |       X                 |          |
        |          |      / \                |          |
        |          |         \               |          |
    RETRIEVE       |          \              |          |
   ---->|--------->|           \             |          |
        |        TX/RTX         \            |          |
    Nothing to    are            \           |          |
    Retrieve     Empty            \          |          |
   <----|<---------|               \         |          |
        |          |                \________|          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |
        |          |                         |          |

> You are currently subscribed to sigtran as: [email-masked]

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 16:18:59 +0530

Brian,

Can you please explain why mis-sequencing will take place? I think it will
take place only if retransmission of MSU2 -7 takes place. But in this case
there are no messages in the sending SCTPs send queue, so there will be
no retransmission or mis-sequencing.

---Vipin

"Brian F. G. Bidulock" <[email-masked]> on 10/30/2001 04:03:49 PM

Vipin,

[email-masked] wrote:                                (Tue, 30 Oct 2001
15:56:02)

--brian

      M2PA       SCTP                       SCTP       M2PA
        |          |                         |          |

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 05:00:37 -0600

Vipin,

SS7 MTP (Level 3) flushes buffers if a resume occurs after a LPO
has persisted long enough to cause the other end to perform a
changeover.  This is because MSU-8, 9, ... may have already been
delivered to the destination over another link.  You cannot send
the old MSU 2 to 7 after MSU 8 to 8+n  have already arrived.
Serious mis-sequencing will result.  This is why flushing buffers
was added to the SS7 specifications.

Failure to flush buffers when LPO is of a "long term" will violate
SS7 MTP (Level 3) specifications and fail MTP3 conformance.

--brian

[email-masked] wrote:                                (Tue, 30 Oct 2001 16:18:59)
> You are currently subscribed to sigtran as: [email-masked]

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: Randall Stewart <[email-masked]>
Date: Tue, 30 Oct 2001 06:14:04 -0600

Harsh:

I think you miss a important point here. It does too allow you
to get the unsent/unacked data. It may not be the way you
expected but it is present. The following is from the next
rev of the document so the numbering may be off from the -00 
version.... 

"
5.3.1.4 SCTP_SEND_FAILED

    If SCTP cannot deliver a message it may return the message as a
    notification.

    struct sctp_send_failed {
        uint16_t ssf_type;
        uint16_t ssf_flags;
	uint32_t ssf_length;
        uint32_t ssf_error;
        struct sctp_sndrcvinfo ssf_info;
        sctp_assoc_t ssf_assoc_id;
        uint8_t ssf_data[0];
    };

    ssf_type:

    It should be SCTP_SEND_FAILED.

    ssf_flags: 16 bits (unsigned integer)

    The flag value will take one of the following values

    SCTP_DATA_UNSENT  - Indicates that the data was never put on
                        the wire.

    SCTP_DATA_SENT    - Indicates that the data was put on the wire.
                        Note that this does not necessarily mean that
the
                        data was (or was not) successfully delivered.

    ssf_length: 32 bits (unsigned integer)

    This field is the total length of the notification data, including
    the notification header and the payload in ssf_data.

    ssf_error: 16 bits (unsigned integer)

    This value represents the reason why the send failed, and if set,
    will be a SCTP protocol error code as defined in [SCTP] section
    3.3.10.

    ssf_info: sizeof (struct sctp_sndrcvinfo)

    The original send information associated with the undelivered
    message.

    ssf_assoc_id: sizeof (sctp_assoc_t)

    The association id field, sf_assoc_id, holds the identifier for the
    association.  All notifications for a given association have the
    same association identifier.  For TCP style socket, this field is
    ignored.

    ssf_data: variable length

    The undelivered message, exactly as delivered by the caller to the
    original send*() call.
"

Note that you just subscribe to this type of event and then if
a connection fails you get one of the above for each message
that was inqueue.

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 15:38:31 -0500

Brian,

I also think that  ACK functionality at the m2pa level is redundant
overhead.  An alternative to consider is the ATM HSL approach, fail the
link when a LPO occurs.

Regards
Ray

"Brian F. G. Bidulock" <[email-masked]> on 10/30/2001 04:53:56 AM

To:   "sigtran" <[email-masked]>
cc:    (bcc: Ray P. Singh/Telcordia)
Subject:  Re: SCTP socket API and M2PA changeover

Dipak, Vipin, Harsh,

     M2PA       SCTP                       SCTP       M2PA
       |          |                         |          |

--brian

[snip]

Does it make sense now?

[snip]

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---
You are currently subscribed to sigtran as: [email-masked]
To unsubscribe send a blank email to
[email-masked]

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 15:06:47 -0600

Ray,

ATM drops the SCCOP connection when LPO occurs, it does not
tear down the entire VC.

Even still this might be a viable approach for LPO, but it
is not for the BUSY scenario which exhibits the same difficulty.
Sending ABORT on BUSY is a poor idea.  How will you handle
the problem for BUSY?

Interestingly enough SSCOP uses an Ack which is not used to
calculate RTTs or QOS parameter either.  Thus, SSCOP's Ack
is well-suited to SS7 MTP2 and can be withheld at any time,
particularly in MTP3 Busy conditions.  SCTP cannot withold
its Acks.

A witholdable Ack is required for proper MTP3 operation.  How
will it be provided without sequence numbers?

--brian

[email-masked] wrote:                                (Tue, 30 Oct 2001 15:38:31)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: Jacob Heitz <[email-masked]>
Date: Tue, 30 Oct 2001 13:41:51 -0800

Why can SCTP not withold its ACKs ?
Have you tried ?
If so, what happened ?

If MTP2 witholds its ACKs for too long, the link drops.
The same would be true for SCTP. Is that a problem ?

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 16:28:02 -0600

Jacob,

Jacob Heitz wrote: (Tue, 30 Oct 2001 13:41:51)
> Why can SCTP not withold its ACKs ?

The situation described would require delaying acks 1 to 2 seconds.
Perhaps I should have said: SCTP cannot withold its ack long enough.

Secion 6.2 RFC 2960:

	"... Specifically, an acknowledgement ... SHOULD be generated
	 within 200 ms of the arrival of any unacknowledged DAA chunk."

	"An implementation MUST NOT allows the maximum delay to be
	 configured to be more than 500 ms.  In other words an
	 implementation MAY lower this value below 500ms but MUST NOT
	 raise it above 500ms."

> If so, what happened ?

SCTP cannot exceed 500ms per the "MUSTs" above, regardless of whether on
considers whether it "should" be done (impact on RTT calculation,
congestion avoidance, potential triggering of slow start, excessive
and unnecessary retransmissions, ...).

The RFC is clear again that:

	"Specifically, an acknowledgement SHOULD be generated for at
	 least every second packet ..."

I believe the intention of the 200ms limit is to set a limit on the time
that SCTP will wait for another message with unacknowledged DATA chunks
before issuing a SACK between the "every second packet".

Witholding acks for any other reason is not recommended:

Section 4.2 RFC 2581:

	"The delayed ACK algorithm specified in [Bra89] SHOULD be
	 used by a TCP receiver.  When used, a TCP reciver MUST NOT
	 excessively delay acknowledgements.  Specifically, an ACK
	 SHOULD be generated for a least every second full-sized segment,
	 and MUST be generated within 500ms of the arrival of the first
	 unacknowledged packet."

Again it is clear that delaying acks is intended for the "every second"
algorithm.  Delaying longer than every second packet, or generating a
"Stretch ACK violation" and its impact on performance is detailed in
RFC 2525.

The topic on this thread included the SCTP Socket API.    There is no
mechanism in the SCTP Socket API for the socket user to ask SCTP to
delay Acks.  The maximum gap between acks for successive packets can
be set, but there is no mechanism to ask SCTP to start witholding acks.

It really doesn't appear to be a option for M2PA.  M2PA should not
rely upon implementation quirks in specific SCTP implementations for
its support.

--brian

Jacob Heitz wrote:                                          (Tue, 30 Oct 2001 13:41:51)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Tue, 30 Oct 2001 17:25:02 -0500

Brian,

See comments below.

Regards
Ray

"Brian F. G. Bidulock" <[email-masked]> on 10/30/2001 04:06:47 PM

Ray,

Even still this might be a viable approach for LPO, but it
is not for the BUSY scenario which exhibits the same difficulty.
Sending ABORT on BUSY is a poor idea.  How will you handle
the problem for BUSY?
     [Ray] I agree that ABORT is not needed for the BUSY case.  My view is
that the BUSY case is really an m2pa flow control issue similar to how link
status busy is used for MTP2 flow control in T1.111.3.  An ack function at
m2pa seems to be over kill as a solution for m2pa flow control.  May be a
simple solution to consider could be,  remote m2pa just don't send more new
messages down to SCTP when BUSY status is received.

     [Ray] Again I view this as a m2pa flow control issue and not an MTP3
busy condition.

--brian

[email-masked] wrote:                                (Tue, 30 Oct
2001 15:38:31)

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

---

From: Tom George <[email-masked]>
Date: Tue, 30 Oct 2001 17:14:19 -0600

> The difficulty has been presented before, but I will present
  [...] 
> maybe dangerous) and is anyway likely not supported by most SCTP
  [...]

Brian,

Could you give some more explanation here? 

a_rwnd: I think the problem here is that even if the upper layer (M2PA)
had a way of telling SCTP to set its a_rwnd to 0, there could still be
packets in flight. Those messages could be lost. 

SACKs: Are you referring to the idea you mentioned earlier of sending
GAP ACKs with no gap in them? or something else?

-- 
* Tom George        972-519-3168         [email-masked]   *
* PB2-1323       Intranet:  http://www.ssd.usa.alcatel.com/~tgeorge/  *
* Alcatel USA, Inc.; 1000 Coit Rd, MS PB2-STPA; Plano TX 75075-5813   *
* I actively disclaim speaking for Alcatel USA, Inc. I do not purport *
* to advertise or represent its position,  operations, or activities. *

---

---

From: Tom George <[email-masked]>
Date: Tue, 30 Oct 2001 17:25:14 -0600

Brian,

My understanding is that a time-controlled changeover is performed after
the Remote Proc Outage. In a time-contolled changeover, only the
untransmitted messages are sent over the alternate link. So it seems to
me that even if you have a mechanism for keeping messages 2-7 in the
transmit queue, they would still be flushed. The only difference is that
they are flushed by the sender instead of the receiver. Or am I missing
something?

Maybe I am mixed up by the timing. Where is MTP3 timer T1 expiring on
your diagram?

Also, what is causing the OOS at the bottom of the diagram? Is it
independent of the RPO?

Thanks,
tomG.

P.S. Excellent ASCII diagram.  8-)

---

---

From: Tom George <[email-masked]>
Date: Tue, 30 Oct 2001 17:33:12 -0600

Comment below...

> Sending ABORT on BUSY is a poor idea.  How will you handle

One problem I see with ABORTing the association on LPO is that a number
of incoming messages may be in the SCTP receive queue before it goes to
LPO. It might even be in a Busy condition. If you ABORT the association,
all those messages are lost. 

  [...]

---

---

From: Tom George <[email-masked]>
Date: Tue, 30 Oct 2001 17:36:30 -0600

Is there some reason that the SCTP spec could not be modified so that it
allowed the ULP to withhold ACKs? 

tomG.

  [...]

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 18:37:40 -0600

Tom,

Tom George wrote:  (Tue, 30 Oct 2001 17:25:14)
> they are flushed by the sender instead of the receiver. Or am I missing

If a changeover order is received within time T1, MSUs 2-7 could be
collected by a shift to the normal procedure per Q.704 5.6.2:

	"If time-controlled changeover has been initiated
	according to case ii) above [processor outage indication
	received] and if a changeover order is received from the
	remote and during the Time T1, it is advantageous to
	switch to the normal changeover procedure including
	retrieval because unnecessary message loss or sending of
	old messages is avoided in a simple way.  The ability to
	perform this switch is considered to be implementation
	dependent.  A changeover acknowledgement however, must
	be returned in any case in order to assure the normal
	completion of the changeover procedure at the remote
	end.  If a changeover order is received after timer T1
	has expired time-controlled changeover is completed (if
	not yet done) and an emergency changeover acknowledgement
	is sent to the remote end.

If MSUs 2-7 have been moved to the remote, there is no opportunity
to retrieve them an "unnecessary message loss or sending of old
messages" cannot be avoided in this "simple way."
> Maybe I am mixed up by the timing. Where is MTP3 timer T1 expiring on

My diagram does not show that well...  It works like this:

        MTP3           Links                               MTP3
         |             0   1                 0   1           |
         |             |   |                 |   |           |
         | ---MSU1---> | <----MSU1 Acked---->| ----MSU1----> |
         | ---MSU2---> |   |        ___SIPO__|<xxxxxxxxxxxxx>|------
         | ---MSU3---> |           /         |    Failure    |   ^
         | ---MSU4---> |   |      /          |   |           |   |
         | ---MSU5---> |         /           |        Detects|   |
         | ---MSU6---> |   |    /            |   |  Permanent|   T1
         | ---MSU7---> |       /             |        Failure|   X 
  _______| <---RPO---- |<-----/              |               |
     ^   |             |   |        ----ECO--|---|<---ECO----|
  ---|-->|MSU8         |           /         |               |
  ---|-->|MSU9  CO     |   |      /          |   |           |
  ---|-->|MSU10 Buffer |         /           |               |
  ---|-->|MSU11        |   |    /            |   |           |
     |   |             |       /             |               |
    T1   | <---ECO-----|---|<--              |   |           |
     X   |             |                     |               |
  Switch | --GetBSNT-->|   |                 |   |           |
    to   | <--BSNT-----|                     |               |
  Normal | ----COA-------->|------COA----------->|----COA--->|
         | --ClearRTB->|                     |               |
         | --Retrieve->|                     |               |
         | <--MSU2---- |                     |               |
         | <--MSU3---- |   |                 |   |           |
         | <--MSU4---- |                     |               |
         | <--MSU5---- |   |                 |   |           |
         | <--MSU6---- |                     |               |
         | <--MSU7---- |   |                 |   |           |
         |             |                     |               |
         | ---MSU2-----|-->| <----MSU Acked--|-->| --MSU2--> |
         | ---MSU3-----|-->  <----MSU Acked--|-->  --MSU3--> |
         | ---MSU4-----|-->| <----MSU Acked--|-->| --MSU4--> |
         | ---MSU5-----|-->  <----MSU Acked--|-->  --MSU5--> |
         | ---MSU6-----|-->| <----MSU Acked--|-->| --MSU6--> |
         | ---MSU7-----|-->  <----MSU Acked--|-->  --MSU7--> |
         | ---MSU8-----|-->| <----MSU Acked--|-->| --MSU8--> |
         | ---MSU9-----|-->  <----MSU Acked--|-->  --MSU9--> |
         | ---MSU10----|-->| <----MSU Acked--|-->| --MSU10-> |
         | ---MSU11----|-->  <----MSU Acked--|-->  --MSU11-> |
         |             |   |                 |   |           |
         |             |                     |               |
         |             |   |                 |   |           |
         |             |                     |               |
         |             |   |                 |   |           |

A Link processor generates SIPO when it loses communication
with the local MTP3.  The actions taken by the MTP3 processor
that it has lost communication with are independent.  If MTP3
detects that the failure is permanent immediately or at least
within its T1, it is pefectly welcome to initiate emergency
changeover.

Thus "unnecessary message loss or sending of old messages is
avoided in a simple way."

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: Cliff Thomas <[email-masked]>
Date: Tue, 30 Oct 2001 18:53:18 -0600

Tom,

  I didn't follow this on.  Couldn't we retrieve those messages? 
Isn't that what the BSNT is all about?

Cliff

Tom George wrote:
> You are currently subscribed to sigtran as: [email-masked]

-- 
==================================================================
Cliff Thomas                          [email-masked]
M/S STP
Alcatel USA                                          (972)477-7015
1000 Coit Rd.
Plano, TX 75075

I only speak for Alcatel when I say that I don't speak for Alcatel
==================================================================

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 21:29:39 -0600

Tom,

Tom George wrote:                                             (Tue, 30 Oct 2001 17:14:19)
> packets in flight. Those messages could be lost. 

RFC 2960 6.1 A)

	"A) At any given time, the data sender MUST NOT transmit new
	    data to any destination transport address if its peer's
	    rwnd indicates that the peer has no buffer space (i.e.
	    rwnd = 0, see Section 6.2.1).  However, regardless of the
	    value of rwnd (including if it is 0), the data sender
	    can always have one DATA chunk in flight to the receiver
	    if allowed by cwnd (see rule B below).  This rule allows
	    the sender to probe for a change in rwnd that the sender
	    missed due to the SACK having been lost in transit from
	    the data receiver to the sender."

cwnd always permits 1 MTU slop.  Rule A permits one data chunk in
flight.  Section 6.2 requires acknowledgement of every second
message containing new data.

There is no requirement in RFC 2960 for the receiver to withold
acknowledgements when it advertises a rwnd = 0.  In fact, many OS
implementations of the equivalent TCP mechanism (and some
implementations of SCTP that we have discussed off-line) allow the
ULP's receive buffer to slop over as well and continue acknowledging
data as normal even though the connection's receive buffer has been
exceeded.

Some implementations of TCP send duplicate segments to probe the
rwnd when rwnd is 0.  SCTP does not use this procedure but is
permitted to send new data (one DATA chunk in flight).

So, setting a_rwnd to zero does not really guarantee that data is
not going to be acknowledged by the receiver.  It does not
necessarily withold any acknowledgements.

Setting a_rwnd to zero does at least slow the transmitter, however,
the LPO or BUSY will make it there just as fast (or just as slow,
depending on how you look at it) as the SACK containing the a_rwnd =
0.  It is just another case of shutting the barn door after the
horse has fled.  The receiver can continue to ack MSU2 through MSU7
which were in flight before the transmitter received a_rwnd = 0.  It
may have a slowing effect on MSUs queued for transmission at SCTP
that the M2PA cannot pull back after receiving LPO or BUSY, but
after the SACK(a_rwnd=0) and LPO arrive at about the same time, the
M2PA will stop delivering messages to SCTP anyways.  MSU2 thru 7 are
still trapped on the wrong side of the interface.

Setting a_rwnd = 0, therefore, does not solve the problem, and may
have some negative side-effects from letting the ULP dork around
with the a_rwnd mid-association.  Receive buffer sizes are really
meant to be set up when the connection begins and left alone
afterwards.

M2PA cannot recommend dorking around with a_rwnd values and
artificially droppping them in the transmitter's face.  The
SCTP transmitter forms a view of the peer's receive buffer by
adding and substracting various values from the advertised value.
By changing the advertised value non-linearly, the peer's view
of the receive buffer might very well misrepresent the situation.

If for example, the LPO situation were to be asserted and then
clear quickly (within T1), even bound back and forth from LPO to
no LPO in an ocillatory fashion, the impact on the throughput of
the assocation could be not what one wanted.

M2PA should institute a sequence number and ack to provide the flow
control that MTP3 is expecting rather than relying on SCTP quirks
that may never make it into an implementation.

> GAP ACKs with no gap in them? or something else?

One way that was discussed here and on tsvwg about 6 months ago now
was to have SCTP send a GAP ACK block that is contiguous wtih the
cumulated ACK (i.e., has no gaps).  The SCTP would GAP ACK DATA
chunks until they were somehow acknowledged by the ULP.  This would
have the effect of an advisory ACK that would not mess up the
transmitter's RTT calculations or RTOs, but would force the
transmitter to keep a copy of the DATA (because GAP ACKs can be
reneged).  However, RFC 2960 does not describe GAP ACKs being used
in this fashion and pretty much indicates that GAP ACKs are used
to report gaps (not to take advantage of their advisory nature).

It is quite unlikely that an SCTP implementation would support to
the ULP the ability to respond with GAP ACKs instead of Cumulative
ACKs.

M2PA should not take advantage of this quirk which is not provided
for in the SCTP RFC and over which it is likely that no SCTP
implementation will provide ULP control.

It is a little late to go changing SCTP in these regards.

Again, this is easily solved by giving M2PA its own ack to withold.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 21:40:47 -0600

Tom,

Tom George wrote:                                             (Tue, 30 Oct 2001 17:33:12)
> all those messages are lost. 

Now that problem is pretty implementation dependent.  Just don't
queue messages between SCTP and M2PA and that won't be a problem.

ABORT would actually get rid of the problem for LPO (but not BUSY).
But ABORT brings with it a new range of considerations.  The peer
is going to try to re-establish the association.  What does the
M2PA do?  Refuse the association?  This means that if the LPO condition
clears, it will be necesary to reform the association, align it,
prove it, and go through slow start.  SSCOP has none of this.  MTP3b
permits foregoing proving, it takes two (very small) messages (one
each way) to establish an SSCOP session, and SSCOP has no initial
congestion avoidance and transfers full traffic right off the bat.
Bringing up SCTP association is much more involved (cookies and acks
and doing MD5 hmacs, MD4 or 5 swirling for verification tags, ramping
up from slow start...

Dropping an SSCOP session does not drop the ATM VC and the channel
still exists.  In fact the disconnect and connect messages could be
viewed as simply LPO and LPR messages because they are that simple.

So M2PA should not ABORT associations just because it wants to
withhold acks.  The cost of the ABORT is high and the M2PA has no
understanding as to the extent or the duration of the problem.

One might be asserting LPO briefly just to swap processors.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 21:58:18 -0600

Tom,

I asked for RFC 2960bis to contain the GAP Ack approach (ie., permit
the receiver to GAP Ack without GAPs) as a MAY kinda thing.  I even
proposed text.  Others felt that application level acks at ULP were
preferrable to altering SCTP (which is at Proposed Standard and in
widescale implementation).

I as told that it might work, it might ABORT the association.  There
is no way that any of these approaches can be forced upon existing
implementations.  It is better to address it in M2PA.  Most other
applications for SCTP have no need for this capability.

The correct place to deal with this is in M2PA.

--brian

Tom George wrote:                                             (Tue, 30 Oct 2001 17:36:30)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Oct 2001 22:01:53 -0600

Cliff,

When a receiver goes LPO, the local MTP processor has lost contact
with the link that it must retrieve from.  Tom is talking of MSUs
queued to M2PA before M2PA issues and ABORT request.  The messages
could be retrieved by M2PA, but it is loss of communication between
M2PA and MTP3 locally that caused the situation and it is therefore
not possible for MTP3 to retreive the messages.  In fact, if the
LPO condition clears after T1, these messages must be discarded per
Q.704.

--brian

Cliff Thomas wrote:                                         (Tue, 30 Oct 2001 18:53:18)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Wed, 31 Oct 2001 09:42:44 +0530

Brian,

I think the problem should be addresed at the SCTP level. Today we
have a requirement in M2PA, tommorow the same thing may come up
in some other layer.
Also SCTP-SCTP reliability is useless if SCTPULP-SCTPULP reliability
is not guaranteed.

---Vipin

"Brian F. G. Bidulock" <[email-masked]> on 10/31/2001 09:28:18 AM

To:   "sigtran" <[email-masked]>
cc:   [email-masked] (bcc: Vipin K Gupta/HSS)

Tom,

--brian

Tom George wrote:                                             (Tue, 30 Oct 2001
17:36:30)

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

---

From: [email-masked]
Date: Wed, 31 Oct 2001 10:34:10 +0530

Hi ,

The current M2PA draft v03 states in section 4.2.6 'Changeover':
"....SCTP must retain the
messages for retrieval by MTP3/M2PA whenever an association is
aborted. SCTP must be able to return messages to M2PA so that M2PA can
perform retrieval for MTP3....."

Is there  a previous thread where m2pa level buffering of user messages was
discussed and a conclusion made that support at SCTP
level is insufficient or not necessary? i.e a conclusion contrary to what is
stated in the current draft.
Or is this the first time this is being dicsussed?
If this has been discussed before, can somebody please tell me the topic of the
thread to follow?

thanks
Harsh

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 31 Oct 2001 00:07:54 -0600

Harsh,

The complete discussion goes back over a year.  The more recent
thread goes back to May of this year and is entitled "M2PA
Acknowledgement Problem".  It was also discussed on tsvwg as far
back as the beginning of the year in conjunction with both
retrieval and shutdown in the sockets API.    Sigtran was copied
on these messages as well.

The M2PA draft cannot place a requirement on the SCTP RFC.

--brian

[email-masked] wrote:                              (Wed, 31 Oct 2001 10:34:10)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 31 Oct 2001 00:10:00 -0600

Vipin,

It has nothing to do with reliability and everything to do with
application flow control; however, if it is addressed at the
SCTP level in an SCTP extension draft, perhaps we should shelf
M2PA until such a draft becomes available.

--brian

[email-masked] wrote:                                (Wed, 31 Oct 2001 09:42:44)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

---

From: [email-masked]
Date: Wed, 31 Oct 2001 11:51:12 +0530

Brian

Why not, I suggest that we should rather propose something like
MTP2 over SCTP instead of M2PA because adding queues and
acks at M2PA level makes it MTP2. Then probably we will not even
need SCTP.

---Vipin

"Brian F. G. Bidulock" <[email-masked]> on 10/31/2001 11:40:00 AM

Please respond to [email-masked]

To:   Vipin K Gupta/HSS@HSS
cc:   sigtran <[email-masked]>

Vipin,

--brian

[email-masked] wrote:                                (Wed, 31 Oct 2001
09:42:44)

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---