Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue (fwd)

From: [email-masked]
Date: Tue, 30 Sep 2003 08:56:49 -0400

Hello All,

Interesting discussion...

>> Perhaps you would prefer the original text that said "SHOULD" and
"promptly".

I still prefer the original draft text and think it sufficiently addresses
this issue.
I do not agree with the newly proposed 'MUST' or 'immediately' regarding
sending ACKs.

Regards,

Jeff

                      "Brian F. G.                                                                                                
                      Bidulock"                To:      Nitin Vairagare <[email-masked]>                               
                      <bidulock@openss         cc:      Javier Kim <[email-masked]>, [email-masked]                            
                      7.org>                   Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue (fwd)                          
                      Sent by:                                                                                                    
                      sigtran-admin@ie                                                                                            
                      tf.org                                                                                                      

                      09/26/03 10:53                                                                                              
                      PM                                                                                                          
                      Please respond                                                                                              
                      to bidulock                                                                                                 

Nitin,

SCTP acks packets (TSNs) not M2PA messags (SSNs).  SCTP is already able to
wait briefly (the Nagle algorithm with Minshall modifications is quite
useful
int his regard) and coallesce M2PA messages into SCTP packets.  Do no
assume
that 1 M2PA will be placed into each SCTP packet.  The SACK engine is
independent of the M2PA ack mechanism.  Also, in SCTP, delaying SACKs is
completely optional in circumstances that permit it and forbidden in
circumstances that doe not (duplicate TSNs reports, gaps, etc).

While the Nagle algorithm benefits the network by reducing the number of
undersized packets in the network (and thus the packet overhead for a given
amount of application data), it does so at cost to latency.  Delaying
SACKs (with the hope of bundling them with outher chunks) is far less
beneficial to the network than Nagle.  Even still both are optional,
neither
are required to protect the network, and they becom operational decisions.

However, delaying SACKs too long *is* harmful to the network (because it
cause
unnecessary retransmissions), and this is why SCTP places absolute maximums
on
the amount of time that one can delay a SACK.  (An SCTP that always SACKs
immediately and without delay is completely conformant to RFC 2960.)  Also,
delaying SACKs at all under some circumstances is harmful to the network
(dup
reports, gaps, etc.) because they again cause an excessive numebr of
retransmissions.  Thus RFC 2960 (and similarly for TCP) makes requirements
upon when an SCTP receiver MUST send a SACK immediately, and on its own
if there are no other messages with which to bundle it.

When you consider Nagle (with Minshall modifications) and delayed SACK, it
is
quite likely the case under normal operation that if there is no DATA, that
successive DATA-ACKs generated by M2PA will in fact be bundled, and they
will
have a delayed SACK piggybacked with them.

But this is the domain of SCTP an well outside the scope of M2PA.  M2PA
should
send acknowlegements immediately and let SCTP sort out precisely when to
send
acks and SACKs.

On of the major differences between MTP Level 2 ack mechanism and SCTP SACK
mechanism is that MTP Level 2 only acknowledges messages once they are
assured
to be delivered to MTP Level 3.  So when there is a processor outage, or
receive congestion towards MTP Level 3, acknowledgements are withheld.
This
is why we abandonned using the SACK mechanism for acknolwedgement in M2PA.
It
was impossible for an SCTP ULP to suppress sending SACK in these two
circumstances.  So, whereas SCTP's ack is an acknowledgement that the
message
was received correctly, M2PA's ack is an acknowledgement to the M2PA peer
that the message was actually delivered to MTP Level 3.

I would even go so far as to say that we should have put acknowledgements
on
stream 0 in M2PA.  This is because it might be a good idea for
acknowledgements to not be delayed waiting for a HOL blocked data message
in
stream 1.  Consider the following:

               _______
                | | | |
                | | | |
        ------->| | | |------------------------>
                | | | |
               _|_|_|_|
                                        _______
                                       | | | |
                                       | | | |
        <------------------------------| | | |<---
                                       | | | |
                                       |_|_|_|_

Both peers have 1 second of DATA messages backed up in queue to be
transported
by SCTP.  When one peer receives a message, it responds immediately.
However,
because the DATA-ACK goes in stream 1, it will have to wait 1 seconds to be
delivered to the peer.  The messages received are already 1 second old, so
the
DATA-ACK will arrive at the peer 2 seconds after the message was sent.  If
T7
is 1.2 seconds, this link wll fail due to excessive delay of
acknowledgement,
causing changeover and increased loss of data in ther SS7 network.  If
acknowledgements are sent on stream 0, a different queue applies and SCTP
can
deliver DATA-ACKs ahead of the data.  Also the expediting nature for stream
0
of some SCTP implementations could be utilized.  In that case the link will
not fail and failure detection has dropped to a about 1.0 seconds from 2.0
seconds.

Consider that SCTP SACKs can be delayed about 200ms (or sometimes every
other
message) and then they are sent immediately.  If M2PA acknowledges messages
immediately, they are still queued in SCTP and may take some time to reach
the
other end.

Given that bundling is performed by SCTP (Nagle/Minshall), and M2PA
acknowledgement indicates successful delivery to MTP3 rather than
successful
receipt, it is best if M2PA does not delay at all.

Because of the effect described above, we should have probably kept a
separate
DATA-ACK message that was sent on stream 0.  That would have also relieved
our
problems with LSB, LSBE, LSPO, and LSPR because the acknowledgements would
be
in the same stream as these messages that indicate delay or withholding of
acknowledgement.

So, you see, MTP Level 2 does delay acknowledgements, after sending SIB or
SIPO.  The rest of the time they are immediate.  For an M2PA implementation
to delay acknowledgement without sending LSB or LSPO is to court failling
links due to excessive delay of acknowledgement.  MTP Level 3 is especially
succeptable to catastrophic cascade failures due to link failures under
excessive delay of acknowledgement because, upon changeover, MTP Level 3
transfers large buffers to the next link (which fails for the same reason)
and then adds them all to the next link (which fails for the same reason)
etc.  M2PA must meet MTP3's needs.  It cannot harm the network because it
is an SCTP ULP, not a transport protocol.

That said, send them ever other received message if you would like.  The
proposed text said "immediately" and did not say "before processing any
other received message".  Just don't delay.  So, if you can process
20 received message in a batch, acknowledge them with one DATA-ACK.  Just
don't go away and wait for the next received message before sending it.
But in the wild that won't happen, because DATA is moving in both
directions at about the same rate.

Perhaps you would prefer the original text that said "SHOULD" and
"promptly".

--brian

Nitin Vairagare wrote:
(Fri, 26 Sep 2003 17:39:15)
> Javier/Brian,
>
> We need to understand that the underlying infrastructure for
> traditional ss7 links (56/64kbps or HSL 2.048Mbps) is different
> from the M2PA. In most of the cases we have much more bandwidth
> available for M2PA. At the same time we do not have stringent
> constraints for declaring bad link in absence of FISUs or
> immediate ACK.
>
> The SCTP is already doing Acks for every other message in absence of
> user data messages. So not sending M2PA Ack for 200Ms or till 8 messages
> is not going to prevent SCTP from generating that extra SACK traffic.
> Instead we should take the advantage and synchronize the M2PA with
> underlying SCTP. We should be able to use the same procedure as the SCTP
>
> and prevent SCTP from generating that extra SACK.
>
> So, I think what M2PA should do is, in absence of any User
> Data traffic generate ACK for every other received packet. If no other
> Packet is received then the timeout should generate the ACK. The timeout
>
> Value should be set sufficiently less than SCTP Receive timer so that
> M2PA generates the ACK (which is user data for SCTP) before SCTP
> generates SACK.
>
> Regards,
> Nitin
>
> -----Original Message-----
> From: Javier Kim [mailto:[email-masked]]
> Sent: Friday, September 26, 2003 4:26 PM
> To: [email-masked]
> Subject: RE: [Sigtran] M2PA MSU_ACK frequency issue (fwd)
>
>
> > Date: Fri, 26 Sep 2003 09:29:20 -0500
>
> Hi all,
>
>
> We seem to be having a problem in resolving a clarification of the ACK
> procedure for M2PA.  What is at stake is some fairly ordinary concerns
> for the regular or exceptional operating situations for the operation of
> the protocol.  Both regular and exceptional protocol situations must be
> covered, but an unreasonable burden of overhead should not be introduced
> into the standard at this time (or any time).  Particularly changes
> should
> not be done for no or negligible benefit.
>
> There is a proposal to send ACKs with ordinary data MSUs or in the
> absence
> of outbound data MSUs, ACK with an zero-length MSU after reception
> of at most 8 received MSUs or the passage of 200mS whichever comes
> first.
>
> With respect to the recent remarks concerning the pressing need for
> immediate M2PA acknowledgment packets, the comment is largely without
> insight into how real protocol stacks work in real time.  When receiving
> an MSU, the majority of cases stacks have no queued nor unsent outbound
> MSUs.
> Therefore nearly all received MSUs would result in a needless ACKs were
> they to be required IMMEDIATELY.  However, typically within microseconds
> an MTP3 or higher layer response will likely arrive at M2PA that can
> carry the
> same ACK information in a user content carrying MSU (obviating the
> separate
> ACK).
>
> The proposal to always ACK when the M2PA layer has no data waiting to
> send is un-insightful of this last point and therefore "wastes" the
> link and host bandwidth by not postponing the ACK a very short interval.
> Further the writer confuses the situation by stating "that delaying ACKs
> would degrade M2PA...".  This couldn't be further from the truth.  It is
> precisely the same reason that SCTP delays SACKs (so that immediately
> forthcoming response DATA chunks may carry a SACK) to not waste
> bandwidth
> and CPU perturbation.  There is no loss and no degradation in a short
> delay with only a few intervening received MSUs until a response (which
> is almost surely forthcoming) arrives.  The savings to M2PA and the
> underlying machine is very substantial (up to 50% reduction of overhead)
> and the most of the unlikely penalty is 1 to 8 MSUs stored in the
> sending
> host's retransmit buffer pending retrieval in the rare event of link
> loss and changeover.
>
> The writer points to degradation.  Where is the degrading?  At the worst
> case (I suggest a brief review of mini-max principles)  0-7 messages can
> be retrieved rather quickly by the sender.  However, of significant
> interest to protocol designers is the average or nominal case of
> routine operation.  Therefore, to saddle M2PA  with a certain 30-50%
> packet overhead (IMMEDIATE acknowledgment overhead) forever is a real
> and certain degradation.
>
> Best regards,
>
> Javier
>
>
> > Javier,
> >
> > 200ms is too long.  Waiting 1 received message is too long.  The text
> in
> > M2PA-09 attempts to state the differences between M2PA and Q.703 with
> > regards to positive acknowledgment.  Q.703/5.2.3 says:
> >
> >   "The receiving signalling link terminal acknowledges the acceptance
> of
> >    one or more message signal units by assigning the forward sequence
> >    number value of the latest accepted message signal until to the
> >    backward sequence number of the next signalling unit sent in the
> next
> >    signal unit sent in the opposite direction."
> >
> > Now, Q.703 has busy transmitters, so if there are no MSUs available
> for
> > transmission in the opposite direction, then the next FISU to be sent
> > will carry the acknowledgment.  This means that the mean delay before
> > emission of an acknowledgment is the mean of the emission time for an
> SU
> > in the opposite direction.  For a 64kbs link that is idle in the
> > opposite direction with 1 flag between FISUs is about 375 microseconds
> > (yes, MICROseconds).  Because the delay depends on SU emission time
> > only, the delay for a 2.048Mbps Annex A link (idle in the opposite
> > direction) is 11.7 microseconds.  The mean emission times and the
> > acknowledgment delay associated with mean emission times based on MSU
> > size and Erlang loading of a signalling link are considered in the
> > queuing model for MTP presented in Q.706.  Queuing models (Q.706) for
> > MTP assume 30ms loop delay for terrestrial links INCLUDING PROCESSING
> > TIME AT THE SIGNALLING TERMINAL (caps for emphasis).
> >
> > If a Q.703 (or Annex A) link waited 200ms to acknowledge receipt of an
> > MSU it would destroy the usability of the link for use with Q.704.  So
> > what makes you think that 200ms is any better than the 500ms
> experienced
> > in inter-op testing?  Either would make an M2PA link unsuitable for
> > replacement of even a 64kbps Q.703 link, regardless of the performance
> > of the underlying SCTP transport.
> >
> > Back to Q.703/5.2.3 above and M2PA-09, M2PA-09 says:
> >
> >   "If there is no other User Data messages to be sent when there is a
> >    message to acknowledge, M2PA SHOULD send a User Data message with
> no
> >    data payload.  The FSN for this empty User Data message is not
> >    incremented. It MUST contain the same FSN as the most recently sent
> >    User Data message containing Data. This message SHOULD be sent
> >    promptly in order to prevent poor SS7 performance."
> >
> > This is an attempt to mimic the passage from Q.703/5.2.3 above given
> the
> > consideration that M2PA does not have a busy transmitter (i.e. does
> not
> > idle FISUs).  So the idea is to say that the acknowledgment goes out
> on
> > the next DATA message available for transmission (to SCTP) and that if
> > one is not immediately available for transmission (to SCTP), then a
> > DATA-ACK message must be generated and transmitted (to SCTP).
> >
> > So, I disagree strongly with any statement that would give one the
> > impression that waiting any amount of time is acceptable.
> >
> > Considering this, I don't see a problem with rewording the passage
> from
> > M2PA-09 as follows:
> >
> >   "When there is a message to acknowledge, M2PA MUST acknowledge the
> >    message with the next User Data message sent.  If there is no User
> >    Data message available to be sent when there is a message to
> >    acknowledge, M2PA MUST generate and send a User Data message with
> no
> >    data payload, without delay.  The FSN for this empty User Data
> >    message is not incremented. It MUST contain the same FSN as the
> most
> >    recently sent User Data message containing Data."
> >
> > I believe that makes the statements be closer to the effect of the
> > statements in Q.703.  An implementation that artificially waits 2ms to
> > acknowledge a message is in violation of both.
> >
> > So, you see, stating 200ms would not improve M2PA, it would degrade
> it.
> > It was always the intention that M2PA generate a DATA-ACK to carry the
> > acknowledgment that Q.703 would carry in the next outbound FISU.  If
> you
> > can propose better language to that effect, please send it without
> delay
> > ;)
> > But waiting even so much as 1ms, or even 1 inbound message, to
 > generate
> > an acknowledgment is IMO unacceptable.
> >
> > On the other side, the sender had better be prepared to buffer T7
> worth
> > of messages at maximum throughput or institute N2 RTB full and Q.704
> > transmission congestion procedures in conformance with the portions of
> > Q.703 that are unchanged by M2PA, because it may take SCTP just less
> > than T7 to deliver the ack.  (See test case 3.8.3 "Check RTB full" in
> > draft-bidulock-m2pa-test-04.txt and test case Q.781/8.3.)  At 20Mbps
> > effective throughput and 64 byte User Data messages with a 1.0 second
> > T7, that would mean buffering about 40,000 messages awaiting ack in
> the
> > RTB or instituting RTB full and Q.704 transmission congestion
> procedures
> > before that point (setting N2 < 40,000).
> >
> > Taking the link out of service at 1500 messages isn't acceptable
> either.
> > If your RTB holds a maximum of 1500 messages, set N2 to a maximum of
> > 1500.  If your RTB is byte limited instead of message limited, use N1.
> > Set your Q.704 transmission congestion thresholds based on TB, TB/RTB
> or
> > RTB occupancy appropriately as well to signal congestion before
> > congestion discard occurs.  (Note that with the "Check RTB Full" test
> > cases, the link must remain in service.)
> >
> > --brian
> >
> > Javier Kim wrote:
> > (Thu, 25 Sep 2003 18:01:54)
> > >
> > > Brian,
> > >
> > > The proposed text is offered for section 4.2.1. Are you suggesting
> to
> > > add Q.2110(MaxPD, etc) style method in the M2PA draft?
> > >
> > > 200ms is proposed to set maximum time for INFO_TRANSFER state, not
> in
> > LPO
> > > or any other exceptional states, which should be discussed elsewhere
> > in
> > > the draft anyway.  Rationale is that in one hand, we don't want
> > penalize
> > > implementaions that wish to wait for a reasonable time for ourgoing
> > > MSUs from MTP3, and  the other hand, we don't want to accumulate too
> > many
> > > MSUs queued in the re-transmission queue either. The proposed text
> > will
> > > allow to send MSU_ACK as frequently as or as fast as one elects to
> do
> > so. I
> > > believe that 20 or 30ms is too short.  We don't want to agrevate
> > throughput
> > > by specifying shorter timer than SCTP's.  I welcome your porposal
> for
> > > reasonable timer value.
> > >
> > >
> > >  The actual event experienced during the plugtest 'forced' us to
> wait
> > > 500ms.  Nobody violated any rule described in the draft v09, it led
> to
> > > unacceptable situation.  How are we going tell that the
> implementation
> > is
> > > sloppy or bad?  As far as I can see,  they have violated nothing.
> > Without
> > > a specific method, it did not work, and it will not work, no matter
> > how
> > > many times or how loudly we repeat the word "promptly".
> > >
> > > Yet, I need to hear one compelling reason why we don't want to
> improve
> > the
> > > draft by adding a simple and straight forward rule.  A protocol
> > without
> > > specific rule to cover problematic senario like this, only promote
> > > on-going IG, and diminish its applicability.  Why bother?
> > >
> > > Best regards,
> > >
> > > Javier
> > >
> >
> >
>
>
> _______________________________________________
> Sigtran mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/sigtran
>
> _______________________________________________
> Sigtran mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/sigtran

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

_______________________________________________
Sigtran mailing list
[email-masked]
 https://www1.ietf.org/mailman/listinfo/sigtran

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

---

From: "Tolga Asveren" <[email-masked]>
Date: Tue, 30 Sep 2003 10:36:18 -0400

Brian,

[..snip..]
> stream 1.  Consider the following:
[TOLGA]Considering that the reason for two streams was to prevent problems
with HOL as much as possible and that we like to think about extreme
scenarios (remembering the ones presented to justify T7 per message ;) ) , I
tend to think that acknowledging on stream 0 might make sense (or maybe
better unordered delivery to prevent HOL problems among ACKs themselves -at
the cost of losing sequential behaviour between ACKs and LS messages-).

---

From: "Tolga Asveren" <[email-masked]>
Date: Tue, 30 Sep 2003 11:46:43 -0400

Brian,

After thinking a bit more, two questions below....

> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Tolga Asveren
> Sent: Tuesday, September 30, 2003 10:36 AM
> To: [email-masked]
[TOLGA]Are we sure for the different transport queues per stream?
> > of some SCTP implementations could be utilized.  In that case the
[TOLGA]What do you mean exactly with  "expediting nature of stream 0"?
Different transport queues for different streams (the above case) or
something different?

---

From: Andrew Booth <[email-masked]>
Date: Tue, 30 Sep 2003 14:47:32 -0400

Hi All,

I agree with Jeff.  I prefer the draft text.

Andrew

-- 
Andrew Booth, Software Developer
Signaling Systems Group, Performance Technologies
613-237-4344

Visit www.pt.com/mailing.html to subscribe to our Packets & Signals eLetter.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Sep 2003 13:26:43 -0600

Andrew,

I agree with the draft text too.  I was just trying to
accomodate Javier.

If we have concensus on the draft text, can we drop this
issue?

--brian

Andrew Booth wrote:                                                                 (Tue, 30 Sep 2003 14:47:32)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/