Subject: RE: [Tsvwg] Re: M2PA: User Level Acknowledgements

From: [email-masked]
Date: Tue, 6 Nov 2001 13:40:02 +0200

Dear all, see some comments inline:

> -----Original Message-----
> From: ext Brian F. G. Bidulock [mailto:[email-masked]]
> Sent: 06. November 2001 13:15
> To: sigtran
> Cc: [email-masked]; [email-masked];
> [email-masked]
> Subject: Re: [Tsvwg] Re: M2PA: User Level Acknowledgements
> 
> 
> Ivan.Arias-Rodriguez,
> 
> [email-masked] wrote:        (Tue, 06 Nov 
> 2001 12:57:52)
> > 	Hi all!
> > 
> > > Tom,
> > > 
> > > I think that TSV has fairly addressed this issue and has
> > > indicated that GAP ACKs with no gap is illegal, that
> > > a_rwnd = 0 cannot be permitted to stop data flow (zero
> > 
> > 	Brian, as far as I know, this is not the final 
> conclusion regarding
> > the a_rwnd... If you set it to 0, and you don't acknowledge 
> any new DATA
> > chunk received, this will stop data flow, as the sender 
> will be spending his
> > time just retransmitting a single not acknowledged TSN 
> every RTO until the
> > receiver decides whether it continues accepting new data 
> opening the a_rwnd,
> > or it tears down the association...
> 
> The receiver is allowed to acknolwedge.  This would break MTP 
> conformance.

	The receiver is not going to be allowed to acknowledge in the next
revision of SCTP. It will simply discard the received TSN since it doesn't
have space and then it WON'T acknowledge the received TSN. Sending back a
SACK that doesn't acknowledge any new TSN and still has the a_rwnd set to 0,
is not acknowledging the received TSN. I would vote for not even sending it,
as the data sender (receiver of that SACK) will simply process it by
checking that nothing new is there and finishing not doing anything at all,
thus it is a lost of time...

> It is pretty clear from the discussion that SCTP cannot require the
> receiver to not acknowledge probes even though it continues 
> to advertize

	"It is pretty clear"??? I don't think so... please read the next
note that will be added to the RFC...

"
Note: Anytime the receivers rwnd is 0, the receiver MAY drop
the DATA chunk without acknowledgment of the new DATA
chunk (since it has no room to receive the data nor validate it).
However the receiver MUST still send an acknowledgment showing
the new chunk as still missing with a r_wnd update following 
the delayed acknowledgment guidelines specified below.
"

	For me, from this text "it is pretty clear" that the DATA receiver
MUST NOT acknowledge the receiver TSN...

> a_rwnd = 0.  This is not sufficient for M2PA's needs.  The conclusion
> appears quite final to me.  The transmitter is permitted to send new
> data with a_rwnd = 0 and the receive is permitted to 
> acknowledge it with
> a_rwnd = 0.  This will not change.

	Are you sure you have read the last mails regarding this issue...
:-/. Please Brian...

> > 	You might not be reading some other people's mails... 
> or you might
> > not want to understand them... :-0. You are right that the 
> sentence in
> > section 6.2 of RFC 2960 should be changed, and that's what 
> you were asking
> > for (actually, I'm not that sure what you are asking for), 
> but it seems that
> > as soon as your wish was fulfilled you just don't want it anymore...
> 
> What we were looking for a statement which say that the 
> receive MUST NOT
> acknowledge new data while advertising a_rwnd = 0.  What has 

	About 20 hours ago, Randall wrote that text... After that there have
been 15 more messages with the same subject, 5 of them yours...

> been achieved
> is only that the receiver MAY NOT acknowledge new data while 
> advertising
> a_rwnd = 0.  No, M2PA did not get what it needed here or with GAP ACK.

	Nope... The receiver MAY drop the DATA chunk but MUST NOT
acknowledge it. In other words, if you are "lying" saying stating an a_rwnd
set to 0, while you in reality have space, you might want to keep anyway
that TSN, but as you are going to "lye" again to the data sender telling him
that you haven't received it, effectively he will behave as if you haven't
had anything, and will retransmit it over and over again...

	If you want to drop the TSN or not upon receipt (but still not
acknowledging it) is up to you. If later on you don't tear down the
association but you open the a_rwnd, you will already have that TSN, and you
could directly send a SACK already acknowledging it, thus saving some
time...

> > 	If I'm right (not sure), both the gap ack *or* the 
> a_rwnd change is
> > enough for that problem you have...
> 
> The a_rwnd change has not come about.  It would take a 
> guarantee to the
> ULP that data will not flow while artificially adversiting a_rwnd = 0.
> There are no such guarantees in the works.

	I think there is a guarantee, Brian...

> > Thanks to your comments I think we have been able to solve 
> the problem (tell
> > us if not)...
> 
> We have solved a problem (oversight in RFC pertaining to zero window
> probes) but have provided no solution to M2PA.

	See above... Thanks for your comments regarding this issue in any
case...

	BR Iván Arias Rodríguez

> > but I'm not sure you really want to solve any problem... :-0
> 
> M2PA can solve the problem with sequence numbers and acks.  
> At least it
> doesn't have to work about reordering.  It is just that there are many
> that are lithe to include it in M2PA.
> 
> --brian
> 
> > 
> > 	BR Iván Arias Rodríguez
> > 
> > > window probes), that artificially dropping a_rwnd is not
> > > advisable, and that if ULP wants to withhold acks it needs
> > > its own acks to withhold.
> > > 
> > > The situation with SCTP-level acks means also that neither
> > > TSN nor Stream Sequence Number can be used by M2PA as BSNT
> > > for changeover.
> > > 
> > > Please place sequence numbers and acks in the M2PA protocol:
> > > I will prospose text if you would like.  This has the side-
> > > benefit that multiple logical SS7 links can be placed each
> > > in a separate stream in a single M2PA association and take
> > > full advantage of SCTP's HOL blocking avoidance for improved
> > > MTP performance under noise and delay.
> > > 
> > > Comments?
> > > 
> > > --brian
> > > 
> > > 
> > > Brian F. G. Bidulock wrote:               (Wed, 25 Jul 
> 2001 06:33:34)
> > > > Tom,
> > > > 
> > > > I still see that you don't understand that M2PA needs
> > > > user level acknowledgements which SCTP does not provide.
> > > > The draft as it stands cannot handle Receive Congestion
> > > > or Remote Processor Outage conditions correctly.
> > > > 
> > > > It will be necessary to put a sequence number on M2PA
> > > > data messages and pass back an ACK at the M2PA level
> > > > to fix this.  SCTP will not help.
> > > > 
> > > 
> > > -- 
> > > Brian F. G. Bidulock
> > > [email-masked]
> > > http://www.openss7.org/
> > > 
> > > _______________________________________________
> > > tsvwg mailing list
> > > [email-masked]
> > > http://www1.ietf.org/mailman/listinfo/tsvwg
> > > 
> 
> -- 
> Brian F. G. Bidulock
> [email-masked]
> http://www.openss7.org/
> 
> 
> ---
> You are currently subscribed to sigtran as: 
> [email-masked]
> To unsubscribe send a blank email to 
> [email-masked]
> 

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 6 Nov 2001 05:59:30 -0600

Ivan.Arias-Rodriguez,

[email-masked] wrote:        (Tue, 06 Nov 2001 13:40:02)
> thus it is a lost of time...

Nobody else agreed with you.  The receiver will be permitted to Cum Ack
even though it advertises a_rwnd = 0.

> MUST NOT acknowledge the receiver TSN...

The text says "MAY", not "MUST".

> 	Are you sure you have read the last mails regarding this issue...

The text says "MAY", not "MUST".  Will it change to "MUST"?

> 
> > > 	You might not be reading some other people's mails...  or you might
> > > not want to understand them... :-0. You are right that the sentence in
> > > section 6.2 of RFC 2960 should be changed, and that's what you were asking
> > > for (actually, I'm not that sure what you are asking for), but it seems that
> > What we were looking for a statement which say that the receive MUST NOT

Hello.  If the receiver acks the data sent with a_rwnd = 0, MTP3 fails
conformance tests over M2PA.  "MAY" is not sufficient for M2PA.
"MUST" is required.  Either the transmiter MUST NOT send new data
(which was shot down) or the receive MUST NOT acknowledge it (which
was shot down).

Change "MAY" to "MUST" in the note.

> 
> > been achieved
> > is only that the receiver MAY NOT acknowledge new data while advertising
> acknowledge it.

The text does not say that the receive "MUST NOT acknowledge it".
It says that it "MAY" drop it (but is it permitted to not drop
it and Cum Ack it).

> In other words, if you are "lying" saying stating an a_rwnd

M2PA requires that the data not move whatsoever after a_rwnd = 0.

> 
> > > 	If I'm right (not sure), both the gap ack *or* the a_rwnd change is
> > The a_rwnd change has not come about.  It would take a guarantee to the

"MAY" is not a guarantee.

> 
> > > Thanks to your comments I think we have been able to solve the problem (tell
> > > us if not)...
> > 

See above... 

--brian

> > [email-masked]
> > 
> 
> 
> ---
> You are currently subscribed to sigtran as: [email-masked]
> To unsubscribe send a blank email to [email-masked]

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 6 Nov 2001 07:02:12 -0600

Ivan.Arias-Rodriguez,

[email-masked] wrote:        (Tue, 06 Nov 2001 14:42:00)
> > Ivan,
> > 
> > This only says that if I choose to drop it (MAY) that I MUST still
> > send a SACK.  There is nothing in these words precluding an
> > implementation from Cum Acking the data while still advertising
> > a_rwnd = 0.
> 
> 	Ups... my English might be even worse than I thougth... I thought
> that "however" in this context meant something like "no matter what you have
> done previously"
> 
> > It that is what was intended, I suggest the following sentence
> > be added:
> > 
> >    The receiver MUST NOT cummulatively acknowledge any DATA
> 
> 	Well... cumulatively or not... I don't think that in should
> acknowledge it in any way... Remember that if you have just lower your rwnd
> there will still be a burst of DATA chunks coming...
> 
> >    (i.e., the receiver MUST NOT move the cumulative TSN ack
> >    point) while advertising an a_rwnd of zero.
> > 
> > Clear?
> 
> 	So, if the problem is that "I MAY drop and it MAY be that I MUST NOT
> acknowledge", why not just changing the order of the sentences? So it would
> be "I MUST NOT acknowledge and it MUST be that I MAY drop"? :-)
> 
> Note: Anytime the receiver's rwnd is 0, the receiver MUST NOT
> acknowledge the receipt of the new DATA chunk (since it has no
> room to receive the data). However the receiver MUST still send
> an acknowledgment showing the new chunk as still missing with
> an a_rwnd update, and following the delayed acknowledgment
> guidelines specified below.
> This done, the receiver MAY drop the DATA chunk received or
> MAY keep it for later process and acknowledgement once the
> rwnd grows again due to upper user actions.
> 
> 
> 	We can forget about the last sentence...
> 
> 	But, in any case... The point is that you want to stop the data flow
> first, and later on you want to tear down the association... So, why not
> simply sending an ABORT chunk bundled with a SACK chunk and terminating the
> association in that very moment? Then, you can do your changeover, establish
> a new association with another host, and start sending those TSNs that were
> never acknowledged...
> 
> 	Brian, why this wouldn't work? is it because you still want to have
> the chance of not terminating the association?
> 

Because changeover risks message loss and must be avoided to meet
10^-9.  It is not the implementor's choice anyway: it is required by
the SS7 specifications.  But the approach in SS7 keeps changeover,
and the resulting message loss and misequencing to a minimum by
waiting to be confident that the changeover is necessary before
completing the action.  More often than not, the situation recovers
and a_rwnd can be opened; however, even in the more unlikely event
that the condition persists and changeover must be performed, the
implementation is not allowed to discard more data by acking it
across the link.  To do so would fail conformance tests before
the implementation ever saw the network.

Nevertheless, the ULP-level ack is superior in that it can be
performed in a similar fashion on a per-stream or per-flow-within-
a-stream basis, permitting M2PA to take advantage of SCTP's HOL
blocking solution.  M2PA currently only uses 1 stream for data
because it has no sequence number to use in changeover messages
and uses one stream's SSN instead.  If M2PA had ULP-level flow
control, it could use multiple independent sequence numbers for
the ack and the changeover while avoiding HOL blocking.

> > 2001 14:09:47)
> > > 	Brian, please, please, please, read what is written... Maybe you
> > > just have a problem to understand the difference between 
> > saving the TSN in
> > > your buffer (the MAY), and not acknowledge its receipt (the 
> > MUST)... I quote
> > > (again):
> > > 
> > > > > "
> > >                                                             ^^^^
> > >                    ^^^^^^^
> > >                          ^^^^
> > >                   ^^^^^^^^^^^^^^^^
> > > 	I don't know if it would be better for you something like:
> > > 
> > > Note: Anytime the receiver's rwnd is 0, the receiver MAY drop
> > > the DATA chunk (since it has no room to receive the data).
> > > However the receiver MUST still send an acknowledgement showing
> > > the new chunk as still missing with an a_rwnd update and following
> > > the delayed acknowledgement guidelines specified below.
> > > 
> > > 	Any better? (but is still says exactly the same as before)...
> > > 
> > > ... snip...
> > > 
> > > 	BR Iván Arias Rodríguez
> > 
> > -- 
> > Brian F. G. Bidulock
> > [email-masked]
> > http://www.openss7.org/
> > 
> > 
> > ---

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 6 Nov 2001 07:12:38 -0600

Ivan.Arias-Rodriguez,

[email-masked] wrote:        (Tue, 06 Nov 2001 14:59:08)
> > From: ext Randall Stewart [mailto:[email-masked]]
> > Sent: 06. November 2001 14:44
> > To: sigtran
> > Cc: Arias-Rodriguez Ivan (NRC/Helsinki);
> > [email-masked]; [email-masked];
> > [email-masked]
> > Brian:
> > If the WG feels this is the right idea I have no problem to changing
> > this wording to:
> > 
> > "
> > Note: Anytime the receivers rwnd is 0, the receiver MUST drop
> > As Kachong said, this is a minor point. You should have specified
> > that is what you wanted in the first place... all you would
> > have had to do is quote the sentence and ask it to be changed
> > to a MUST... 
> > 
> > What does the rest of the working group feel? 
> 
> 	I think it is nice... Just a minor question. "nor validate it", do
> you mean calculating the checksum? I guess you have to do that before you
> ever realize there is a DATA chunk inside... If so, I would take those three
> words... I would make some minor spell changes:
> 
> "
> Note: Anytime the receiver's rwnd is 0, the receiver MUST drop
>       the DATA chunk without acknowledgement of the new DATA
>       chunk (since it has no room to receive the data).
>       the new chunk as still missing with a rwnd update and following
> 	Brian? Randall?

I don't think it is necessary to require the implementation to
drop the packet.  That should be up to the implementation (just
as long as it doesn't ack it.)

The statement "(since it has no room to receive the data)" is
misleading.  In the M2PA case the ULP is intentionally and
artificially dropping its receive buffer to 0 in the sender's
face.  It has nothing to do with space considerations.

--brian

> 
> 	BR Iván Arias Rodríguez
> 
> > "Brian F. G. Bidulock" wrote:
> > > 
> > > Ivan,
> > > 
> > I don't like the above since now you get the implementation 
> > lying about where the cum-ack is. This is much harder
> > to implement... if we go down this type of route then
> > I think we should just require the chunk to be dropped.
> > 
> > We might need to add some words about the fact that a probe
> > chunk SHOULD NOT cause any adjustment to the cwnd as would
> > a normal time-out. At least from what Kacheong said this is
> > how TCP works.. so we want to make SCTP similar here..
> > 
> > R
> > 
> > R
> > 
> > 
> > 
> > > 
> > > > your buffer (the MAY), and not acknowledge its receipt 
> > (the MUST)... I quote
> > > > (again):
> > > >
> > > > > > "
> > Randall R. Stewart
> > [email-masked] 815-342-5222 (cell phone)
> > 
> > 
> > ---
> > [email-masked]
> > 
> 
> 
> ---
> You are currently subscribed to sigtran as: [email-masked]
> To unsubscribe send a blank email to [email-masked]

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: [email-masked]
Date: Tue, 6 Nov 2001 14:42:00 +0200

BR Iván Arias Rodríguez

---

From: [email-masked]
Date: Tue, 6 Nov 2001 14:50:34 +0200

Hi again!

> Sent: 06. November 2001 14:35
> To: sigtran

	... snip...

> > The base problem I think that Brian has is he cant DEPEND on
> > the new probes being dropped. A implementation MAY still
> > accept the data when the rwnd=0.  What he wants to do is
> > instead send the data to a seperate association.
> 
> Bingo!

	What I was thinking about was not acknowledging the TSN, but still
having the possibility of keeping it in my buffers as far as I don't either
tell to the upper user that I received it, i.e. I won't deliver it to the
upper user unless at the end it happens that I open the rwnd and I
acknowledge it...

	But I guess that just keeping this DATA chunk is complicating things
and leading to missunderstandings, so I guess that the best thing is not
even allowing to keep it... We are not saving that much time in any case...

	BR Iván Arias Rodríguez

> > Now what I don't understand in this is that eventually the
> > rwnd will get set back to postive and all that data that was
> > inqueue at the sender will come forward.
> 
> Treatment of inqueue data is completely under the control
> of the implementation and already adressed by MTP3 procedures.
> If the failover occurs, this data is flushed (by the
> application) on recovery.
> 
> > It has to be dealt with.
> 
> It is dealt with and the application is free to do with it as
> it chooses.
> 
> > Either that or you must abort the association.  Which in that
> > case why not abort() the association instead of doing rwnd=0..
> 
> The reason for not aborting the association is because the
> thing which cause the a_rwnd = 0 may clear in the next 2 ms.
> The cost of aborting the association for transient events
> is too great.  It is not until the condition persists that
> it is treated as a failure, but data must be held at the
> transmitter in the mean time.
> 
> > Very strange.
> 
> It's just how SS7 works.
> 
> --brian
> 
> > 
> > R
> > 
> > 
> > 
> > -- 
> > Randall R. Stewart

---

From: [email-masked]
Date: Tue, 6 Nov 2001 21:11:17 +0200

Hi all,

> From: ext Qiaobing Xie [mailto:[email-masked]]
> Sent: 06. November 2001 20:11
> To: Randall Stewart
> [email-masked]; 
> [email-masked];
> [email-masked]; [email-masked]
> Hi, Randy:
> Randall Stewart wrote:
> ...
> > I am fine with the above.  We could proably drop the (since...) it
> > does not add anything.
> 
> The above change sounds ok with me.
> 
> > 
> > We also probably need some wording for the sender side 
> something like:
> > 
> > "
> > When a sender is probing for a change in space in the rwnd (where
> > the peer has set the a_rwnd to 0 and no data is inflight), if
> > the single probe chunk is sent and then subsequently time's out (due
> > to no change in the rwnd) the sender SHOULD NOT apply any
> > adjustment to the cwnd (as described in 7.2.3) but it MUST
> > backoff the RTO time by doubling the current value.
> > "
> 
> I am a little concerned about this one, though. There is a danger of
> flooding the network with this change. For instance, if cwnd 
> was at 500k
> the moment the receiver first slammed its a_rwnd down to 0, and the
> a_rwnd remains at 0 for 10 min before it opens up again, and 
> during this
> 10 min period the sender is probing with a 1k (=PMTU) chunk. 
> Then, there
> is the possibility that the sender will flood the network 
> with upto 500k
> data at the moment the a_rwnd opens up again. This doesn't sound right
> to me. 10 min is a long time and the 500k cwnd should be discarded by
> then.

	But in that case, the other rule about the idle addresses will make
the work... Ok, we have the "MUST" vs. "SHOULD" war here again...

	In any case, I don't think people will stay 10 mins with the rwnd
set to 0... It is in any case likely that the whole association will be tore
down before that. As the retransmitted DATA chunk is not liable to be used
to measure the RTT, I think we will be sending HEARTBEATS, and even if we
have to backoff RTO it will be set every now and then to RTO.min... With 8
retransmissions before we give up I don't think the association will survive
more than 30 seconds...

	Moreover, that 0 a_rwnd story will be mostly used in that changeover
situation, which should last no more than few milliseconds...

	BR Iván Arias Rodríguez

> This change is far more fundamental than the previous one. My 
> opinion is
> the normal CC should apply and the cwnd should gradually degrade.
> 
> -Qiaobing
>

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 6 Nov 2001 16:54:37 -0600

Randall,

Randall Stewart wrote:                                    (Tue, 06 Nov 2001 16:33:45)
> > > This is NOT for M2PA. This is to make SCTP probing work the SAME
> > > EXACT way TCP does..
> > 
> > Oh?  I understood that TCP allows new data to be acked during
> > the probe.  Why would SCTP not?
> > 
> 
> Brian:
> 
> The goal was to make Congestion Control work the same
> as TCP. In this case dropping the chunk is one of the
> two options that TCP does... choosing one does not effect
> congestion control..
> 
> On the other hand, reducing the cwnd does effect cc and 
> causes SCTP to be sub-optimal in comparison to TCP in this
> case...

It's a nit pick, I know, but you did say "SAME EXACT way,"
not "compatible 'cause we chose one option way".

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 6 Nov 2001 15:29:20 -0800 (PST)

> how TCP works.. so we want to make SCTP similar here..

There is an interesting interaction.  If zero window
probe is not sent until an RTO (triggered by T3-rxt) after 
SCTP has detected that it needs to probe the other side, it
means that the pipe has been emptied for one RTO and cwnd has
to be reduced.  In TCP, if the pipe is empty for one RTO,
it will do slow start again.  I know that SCTP has a more
lax requirement that cwnd be reduced by half.  But regardless
of what SCTP implementors choose to follow, cwnd will be
changed.  

So while zero window probe itself does not change the cwnd, 
but in effect if one is sent, cwnd is reduced afterall.  It
is not because of zero window probe timeout, it is because of
another requirement.

							K. Poon.
							[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 9 Nov 2001 17:40:30 -0600

Tom,

Tom George wrote:                                             (Fri, 09 Nov 2001 17:31:31)
>
> Brian,
> 
> You are correct. M2PA draft version 4 will have M2PA-level sequence
> numbers and acks. 
> 
> Suggested text would be welcome. 
> 
> I am not sure about putting multiple links on streams within an
> association. What advantage does multiple streams give over multiple
> associations?

Improved HOL blocking performance.  Whether SCTP really accomplishes
this or not, the objective of SCTP was to provide a HOL blocking
solution and recommends multiple streams over multiple associations.
It should be possible, whether now or in extension.

P.S. Thank's for the honorable mention for Andrew!

> tomG.
> > Brian F. G. Bidulock wrote:               (Wed, 25 Jul 2001 06:33:34)
> > > Tom,
> > >
> * Tom George        972-519-3168         [email-masked]   *
> * PB2-1323       Intranet:  http://www.ssd.usa.alcatel.com/~tgeorge/  *
> * Alcatel USA, Inc.; 1000 Coit Rd, MS PB2-STPA; Plano TX 75075-5813   *
> * I actively disclaim speaking for Alcatel USA, Inc. I do not purport *
> * to advertise or represent its position,  operations, or activities. *
> 
> 
> ---

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/