Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue (fwd)

From: Javier Kim <[email-masked]>
Date: Thu, 25 Sep 2003 10:12:42 -0700 (PDT)

Hi Tom,

For some reason, my response did not go out to you yesterday.  Please
read my proposal below.

Best regards, Javier

---------- Forwarded message ----------
Date: Wed, 24 Sep 2003 18:11:01 -0700 (PDT)
From: Javier Kim <[email-masked]>
To: Tom George <[email-masked]>
Cc: [email-masked]
Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue

Hi Tom,

Thank you for your following up on this issue. I believe that the word
"promptly" is not sufficient. We can hope for the best and prepare for
the worst.  Both Q.703 and Q.2110 have their own methods to cover this
specific circumstance and do not leave it to the open interpretation of
 the implementors.

I'd like offer the following text in the section 4.2.1, after
"This message SHOULD be sent promptly in order to prevent
poor SS7 performance.":

"In any event an MSU with updated BSN or acknowledgement MSU MUST
be sent no less frequently than every 8 MSUs received or 200ms whichever
occurs first."

Best regards,

Javier

On Wed, 24 Sep 2003, Tom George wrote:

> Date: Wed, 24 Sep 2003 10:31:44 -0500
> From: Tom George <[email-masked]>
> To: [email-masked]
> Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue
>
> MTP2 is always transmitting something (MSU, LSSU, FISU), so it will
> (almost) immediately ack an incoming MSU. M2PA, on the other hand, is
> not transmitting continuously. So I can see a need for a requirement
> that M2PA not delay acknowledgement too long.
>
> But how should the requirement be stated? We already have a non-specific
> statement in the draft (4.2.1):
> "This message SHOULD be sent promptly in order to prevent poor SS7
> performance."
>
> If you want something more specific, it would be something like this:
>
> =====
> M2PA SHOULD acknowledge a received User Data message within 20ms.
>
> M2PA SHOULD NOT allow more than 8 messages to be unacknowledged at any
> time.
> =====
>
> But there would be exceptions to this, since in some cases M2PA does not
> ack incoming data messages (LPO).
>
> How much more helpful is it to make a more specific than to just say
> "promptly"?
>
> tomG.
>
> Nitin Vairagare wrote:
> > Brian,
> >
> > Your inputs are helpful to those poorly written implementations and
> > provide ways to defend themselves, however, unless standard enforces
> > constraints regarding acknowledgements, there will always be such
> > poorly written implementations. And we all agree that those
> > implementations are dangerous to networks.
> >
> > Regards,
> > Nitin
> >
> > -----Original Message-----
> > From: Brian F. G. Bidulock [mailto:[email-masked]]
> > Sent: Monday, September 22, 2003 2:35 PM
> > To: Nitin Vairagare
> > Cc: Tom George; [email-masked]
> > Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue
> >
> > Nitin,
> >
> > Q.703 requires that *every* MSU be acknowledged.  The SDLs show MSUs
> > being
> > acknowledged immediately and never delayed.
> >
> > Any implementation that waits 0.5 seconds is just poorly written.
> >
> > There is, however, a way to defend yourself against an implementation
> > that
> > does this, at the cost of throttling the link.  That is to set your N1
> > and N2
> > values (see Q.703) to some tight values: such as 32 msgs and 1024 bytes.
> > This
> > means that a Q.703 conforming state machine will stop transmitting new
> > messages when 32 messages or 1024 bytes of MSUs have been sent, until
> > those
> > messsages are acknowledged.
> >
> > You might want to make these even more restrictive, depending on the
> > maximum
> > throughput that you wish to sustain without acknowledgement.  You also
> > might
> > want to set your transmission congestion onset and abatement thresholds
> > down
> > lower than N2.
> >
> > For Q.2210 based links, there are similar transmit window configuration
> > parameters.
> >
> > Buffering large amounts of data in the Transmit Buffer can be
> > disasterous for
> > SS7 in live networks.  Transferring full Transmit Buffers on changeover
> > can
> > lead to cascading congestive failure resulting in total network
> > isolation.
> >
> > Because the transmitter already can enforce constraints in terms of both
> > the number of outstanding messages (N1) and the number of bytes
> > outstanding
> > (N2) that are acceptable within a time period (T7), by configuration you
> > can force whatever constraint you want on the receiver.  Therefore IMO
> > it is
> > not necessary to devise constraints for the draft.  It is sufficient to
> > say
> > that the receiver should acknowledge messages immediately, as the draft
> > already states.
> >
> > --brian
> >
> > Nitin Vairagare wrote:                         (Mon, 22 Sep 2003
> > 14:16:23)
> >
> >>Tom,
> >>
> >>I am copying my comments that I exchanged with you earlier...
> >>
> >>
> >>Yes, But there are no rules to do it..
> >>
> >>As mentioned in original mail it happens only after
> >>timeout occurs. This means the risk of holding lots of
> >>messages in buffer (approx 1500 msgs for transfer rate of 3K and
> >
> > timeout
> >
> >>occurring at 0.5sec). Which is rare in MTP2 in the presence of Acks
> >
> > thru
> >
> >>FISUs?
> >>
> >>Synchronizing it with SCTP might be helpful...
> >>
> >>SCTP requires that in absence of data traffic
> >>from receiver end to Sender, every other data message be
> >>acknowledged or at the expiration of receive timer.
> >>If we synchronize M2PA with SCTP then, SCTP do not need to
> >>generate SACK for every other Data message also.
> >>
> >>In absence of any such formal rules, the loss of data is larger (when
> >>M2PA thread/Process dies). Which could be easily restricted.
> >>
> >>Some formal rules in the M2PA standard may help synchronize the
> >>implementations.
> >>
> >>Regards,
> >>Nitin
> >>
> >>-----Original Message-----
> >>From: Tom George [mailto:[email-masked]]
> >>Sent: Monday, September 22, 2003 1:48 PM
> >>Cc: [email-masked]
> >>Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue
> >>
> >>Javier,
> >>
> >>This was worded non-specifically in the draft, section 4.2.1:
> >>
> >>=====
> >>If there is no other User Data message to be sent when there is a
> >>message to acknowledge, M2PA SHOULD send a User Data message with no
> >>data payload. The FSN for this empty User Data message is not
> >>incremented. It MUST contain the same FSN as the most recently sent
> >>User Data message containing Data. This message SHOULD be sent
> >>promptly in order to prevent poor SS7 performance.
> >>=====
> >>
> >>Waiting 0.5 sec is way too long. On the other hand, even if there are
> >>only a few messges (less than 8), it is still not good to wait 0.5
> >>seconds to ack.
> >>
> >>To cover both cases, there would need to be both a time limit and a
> >>number-of-messages limit.
> >>
> >>tomG.
> >>
> >>Javier Kim wrote:
> >>
> >>>Hi All,
> >>>
> >>>
> >>>From the recent Plugtest in Sophia Antipolis it was observed that
> >>
> > some
> >
> >>>M2PA implementations didn't send MSU_ACKs except by long timer
> >>
> > trigger
> >
> >>>(.5 second).  On a 3000 message-per-second link this could lead to
> >>>recovery queues of 1500 MSUs or more.  It was determined that more
> >>>frequent MSU_ACKs would be better and add little or no overhead for
> >>
> >>most
> >>
> >>>bi-directional message flows.
> >>>
> >>>We suggest that the draft be amended to require an MSU or MSU_ACK be
> >>>returned no less frequently than every 8 MSUs received.  This would
> >>>keep the M2PA recovery queues shorter, and faster to process, in the
> >>
> >>event
> >>
> >>>of a change-over.
> >>>
> >>>Best regards,
> >>>
> >>>Javier Kim
> >>
> >>
> >>--
> >>* Tom George       PB7-24001 (near B-21)      [email-masked]
> >
> > *
> >
> >>* 972-519-3168   Intranet:  http://www.ssd.usa.alcatel.com/~tgeorge/
> >
> > *
> >
> >>*   I actively disclaim speaking for Alcatel.  I do not purport to
> >
> > *
> >
> >>*   advertise or represent its position, operations, or activities.
> >
> > *
> >
> >>
> >>_______________________________________________
> >>Sigtran mailing list
> >>[email-masked]
> >>https://www1.ietf.org/mailman/listinfo/sigtran
> >>
> >>_______________________________________________
> >>Sigtran mailing list
> >>[email-masked]
> >>https://www1.ietf.org/mailman/listinfo/sigtran
> >
> >
>
>
>

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 25 Sep 2003 11:46:35 -0600

Javier,

Javier Kim wrote:                                                                   (Thu, 25 Sep 2003 10:12:42)
> be sent no less frequently than every 8 MSUs received or 200ms whichever

This text will not do.  When M2PA is in LPO it does not acknowledge messages,
contradicting "In any event" and "MUST".

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Javier Kim <[email-masked]>
Date: Thu, 25 Sep 2003 12:08:19 -0700 (PDT)

Brian,

That's a good point. I reedited it addressing your concern.  The new
proposed test is:

"Unless M2PA is in LPO, an MSU with updated BSN or acknowledgement MSU
MUST be sent no less frequently than every 8 MSUs received or 200ms
whichever occurs first."

I believe that my proposed text will make M2PA protocol much more roburst,
and will be in alignment with sprit of other existing signaling protocols.
Please remember that this is needed based on actual experience during the last
plugtest in Sophia Antipolis. The implementors who took .5 second in
sending MSU acknowlegement was firmly convenced that they were complying
the M2PA draft version 9 completely. They were right. In the meantime,
there were around 1500 MSUs in the retransmission queue waiting for their
"prompt MSU_ACK". This is utterly not acceptable to most of us.

Best regards,

Javier

On Thu, 25 Sep 2003, Brian F. G. Bidulock wrote:

> Date: Thu, 25 Sep 2003 11:46:35 -0600
> From: Brian F. G. Bidulock <[email-masked]>
> To: Javier Kim <[email-masked]>
> Cc: [email-masked], [email-masked]
> Subject: Re: [Sigtran] M2PA MSU_ACK frequency issue (fwd)
>
> Javier,
>

---

From: Tom George <[email-masked]>
Date: Thu, 25 Sep 2003 15:04:51 -0500

Javier,

I am still unconvinced this needs to be changed. But just in case:

1. M2PA does not ack messages during Busy condition, either. There are 
probably other unusual cases as well, e.g., user data message received 
while the link is OOS.

2. IMHO 200ms is too long. If M2PA has received fewer than 8 user data 
messages in 200ms, there is no reason to wait that long to ack them. 
That is why I suggested 20ms. But still 20ms (and 8 messages) were 
arbitrarily chosen.

tomG.

-- 
* Tom George       PB7-24001 (near B-21)      [email-masked]  *
* 972-519-3168   Intranet:  http://www.ssd.usa.alcatel.com/~tgeorge/  *
*   I actively disclaim speaking for Alcatel.  I do not purport to    *
*   advertise or represent its position, operations, or activities.   *

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 25 Sep 2003 14:08:20 -0600

Javier,

What about receive congestion?  A link withholds acknowledgements during
receive congestion and is permitted to withhold messages up to T6 which
is much greater than 200ms.

Also, Q.2210 based links will have their own special cases for withholding
acknowledgements.

"SHOULD send promptly" is strong enough language.

Javier Kim wrote:                                                                   (Thu, 25 Sep 2003 12:08:19)
> the M2PA draft version 9 completely.

They are not sending acknowlegements promptly, which they SHOULD.
See RFC 2119:

  3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
     may exist valid reasons in particular circumstances to ignore a
     particular item, but the full implications must be understood and
     carefully weighed before choosing a different course.

Those delaying 0.5 chose poorly.

> They were right. In the meantime,

You chose poorly with regard to transmission congestion procedures and buffer
occupancy.  Transmission side congestion is a function of buffer occupancy in
the TB/RTB.  If 1500 MSUs is unacceptable it is unacceptable under any
circumstances.  What happens in your implemetation when the peer's ack gets
delayed by 0.5 seconds by the network?  It is still acceptable from a T7
standpoint!

Expecting the speed of acknowledgements to keep your TB/RTB buffer size within
limits is just as bad an idea as artificially delaying acks.

You need to implement Q.703 where M2PA does not provide special procedures,
and transmission congestion base don TB/RTB occupancy is one of them.  If you
are using Q.2210 based links, the SSCOP flow control mechanism applies.

You seem to think that this is like an SCTP or TCP SACK or something.  It is
not.  If an SCTP SACK is delayed too long, the sender thinks that packets have
been lost and start retransmission which is harmful for the network.  That is
why "MUST" limits are set on delaing SACKs.

For M2PA, just as for SS7, excessive delay of acknowledgement is measured
with three timers (T6 and T7 in MTP2 and T1 in MTP3) and three procedures
(receive congestion, transmit congestion and processor outage).

Also, performance requirements for delay and acknowledement are provided in
Q.706 and Q.709.  They do not need to be stated in M2PA.

At most, we could add a reference to the performance requirements of
Q.706/Q.709.

Saying 200ms is completely unacceptable.  The draft does not provide values
for T6 or T7 (or MTP3 T1).  T7 could be set to 150ms, in which case 200ms
would be disasterous.  (Not to meantion that delaying 200ms would be foolish
in any case: that is way too long to delay an ack.) "Promptly" means now.  If
you want to change "promptly" to "immediately" I ok with that.  "immediately"
could mean far less than 200ms.  "immediately" means after receiving 1
message.  It would be extremely foolish to wait for 8, so that doesn't make
sense either.

But then "immediately" kinda means before anything else (like processing
non-maskable interrupts and things).  So "promptly" is a probably a better
word.  How about "expediently".  That kinda means ahead of some other things
but not without reason.

I suggest that we change "promptly" to "expediently" and leave it at that.
Just leaving it at "promptly" is fine by me too.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Javier Kim <[email-masked]>
Date: Thu, 25 Sep 2003 18:01:54 -0700 (PDT)

Brian,

The proposed text is offered for section 4.2.1. Are you suggesting to
add Q.2110(MaxPD, etc) style method in the M2PA draft?

200ms is proposed to set maximum time for INFO_TRANSFER state, not in LPO
or any other exceptional states, which should be discussed elsewhere in
the draft anyway.  Rationale is that in one hand, we don't want penalize
implementaions that wish to wait for a reasonable time for ourgoing
MSUs from MTP3, and  the other hand, we don't want to accumulate too many
MSUs queued in the re-transmission queue either. The proposed text will
allow to send MSU_ACK as frequently as or as fast as one elects to do so. I
believe that 20 or 30ms is too short.  We don't want to agrevate throughput
by specifying shorter timer than SCTP's.  I welcome your porposal for
reasonable timer value.

 The actual event experienced during the plugtest 'forced' us to wait
500ms.  Nobody violated any rule described in the draft v09, it led to
unacceptable situation.  How are we going tell that the implementation is
sloppy or bad?  As far as I can see,  they have violated nothing. Without
a specific method, it did not work, and it will not work, no matter how
many times or how loudly we repeat the word "promptly".

Yet, I need to hear one compelling reason why we don't want to improve the
draft by adding a simple and straight forward rule.  A protocol without
specific rule to cover problematic senario like this, only promote
on-going IG, and diminish its applicability.  Why bother?

Best regards,

Javier

On Thu, 25 Sep 2003, Brian F. G. Bidulock wrote:

> Date: Thu, 25 Sep 2003 14:08:20 -0600

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 26 Sep 2003 03:10:14 -0600

Javier,

200ms is too long.  Waiting 1 received message is too long.  The text in
M2PA-09 attempts to state the differences between M2PA and Q.703 with
regards to positive acknowledgment.  Q.703/5.2.3 says:

  "The receiving signalling link terminal acknowledges the acceptance of
   one or more message signal units by assigning the forward sequence
   number value of the latest accepted message signal until to the
   backward sequence number of the next signalling unit sent in the next
   signal unit sent in the opposite direction."

Now, Q.703 has busy transmitters, so if there are no MSUs available for
transmission in the opposite direction, then the next FISU to be sent
will carry the acknowledgment.  This means that the mean delay before
emission of an acknowledgment is the mean of the emission time for an SU
in the opposite direction.  For a 64kbs link that is idle in the
opposite direction with 1 flag between FISUs is about 375 microseconds
(yes, MICROseconds).  Because the delay depends on SU emission time
only, the delay for a 2.048Mbps Annex A link (idle in the opposite
direction) is 11.7 microseconds.  The mean emission times and the
acknowledgment delay associated with mean emission times based on MSU
size and Erlang loading of a signalling link are considered in the
queuing model for MTP presented in Q.706.  Queuing models (Q.706) for
MTP assume 30ms loop delay for terrestrial links INCLUDING PROCESSING
TIME AT THE SIGNALLING TERMINAL (caps for emphasis).

If a Q.703 (or Annex A) link waited 200ms to acknowledge receipt of an
MSU it would destroy the usability of the link for use with Q.704.  So
what makes you think that 200ms is any better than the 500ms experienced
in inter-op testing?  Either would make an M2PA link unsuitable for
replacement of even a 64kbps Q.703 link, regardless of the performance
of the underlying SCTP transport.

Back to Q.703/5.2.3 above and M2PA-09, M2PA-09 says:

  "If there is no other User Data messages to be sent when there is a
   message to acknowledge, M2PA SHOULD send a User Data message with no
   data payload.  The FSN for this empty User Data message is not
   incremented. It MUST contain the same FSN as the most recently sent
   User Data message containing Data. This message SHOULD be sent
   promptly in order to prevent poor SS7 performance."

This is an attempt to mimic the passage from Q.703/5.2.3 above given the
consideration that M2PA does not have a busy transmitter (i.e. does not
idle FISUs).  So the idea is to say that the acknowledgment goes out on
the next DATA message available for transmission (to SCTP) and that if
one is not immediately available for transmission (to SCTP), then a
DATA-ACK message must be generated and transmitted (to SCTP).

So, I disagree strongly with any statement that would give one the
impression that waiting any amount of time is acceptable.

Considering this, I don't see a problem with rewording the passage from
M2PA-09 as follows:

  "When there is a message to acknowledge, M2PA MUST acknowledge the
   message with the next User Data message sent.  If there is no User
   Data message available to be sent when there is a message to
   acknowledge, M2PA MUST generate and send a User Data message with no
   data payload, without delay.  The FSN for this empty User Data
   message is not incremented. It MUST contain the same FSN as the most
   recently sent User Data message containing Data."

I believe that makes the statements be closer to the effect of the
statements in Q.703.  An implementation that artificially waits 2ms to
acknowledge a message is in violation of both.

So, you see, stating 200ms would not improve M2PA, it would degrade it.
It was always the intention that M2PA generate a DATA-ACK to carry the
acknowledgment that Q.703 would carry in the next outbound FISU.  If you
can propose better language to that effect, please send it without delay ;)
But waiting even so much as 1ms, or even 1 inbound message, to generate
an acknowledgment is IMO unacceptable.

On the other side, the sender had better be prepared to buffer T7 worth
of messages at maximum throughput or institute N2 RTB full and Q.704
transmission congestion procedures in conformance with the portions of
Q.703 that are unchanged by M2PA, because it may take SCTP just less
than T7 to deliver the ack.  (See test case 3.8.3 "Check RTB full" in
draft-bidulock-m2pa-test-04.txt and test case Q.781/8.3.)  At 20Mbps
effective throughput and 64 byte User Data messages with a 1.0 second
T7, that would mean buffering about 40,000 messages awaiting ack in the
RTB or instituting RTB full and Q.704 transmission congestion procedures
before that point (setting N2 < 40,000).

Taking the link out of service at 1500 messages isn't acceptable either.
If your RTB holds a maximum of 1500 messages, set N2 to a maximum of
1500.  If your RTB is byte limited instead of message limited, use N1.
Set your Q.704 transmission congestion thresholds based on TB, TB/RTB or
RTB occupancy appropriately as well to signal congestion before
congestion discard occurs.  (Note that with the "Check RTB Full" test
cases, the link must remain in service.)

--brian

Javier Kim wrote:                                                                   (Thu, 25 Sep 2003 18:01:54)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/