Subject: Quick Failover Algorithm in SCTP - question for the WG

From: "James M. Polk" <[email-masked]>
Date: Mon, 24 Jan 2011 19:31:09 -0600

(with my chair hat on)

ISTM that the WG has to answer the following question before either 
this "-nishida-" ID or the "-tuexen-...-multipath-" ID progress (and 
if either does):

   Does the WG feel CMT needs to be accounted for (or involved) in
   whatever the solution is here?

I'm posing this more informally to the WG, but how I, as chair, take 
the answer to this question can get more formal based on the answers 
given (including if no one answers this question).

Why do I pose this?

ITSM the "-nishida-" ID address the problem area, but 
"-tuexen-...-multipath-" ID addresses a superset of the problem area.

If I've mischaracterized this - please correct me concisely. The WG 
doesn't need to be a book of an response, as that'll likely confuse 
more of the WG...

James
(again, with my chair hat on)

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 24 Jan 2011 20:37:48 -0700

James,

IMHO the sctp-multipath draft addresses the problem area
and the sctp-failover draft attempts to isolate one aspect
that cannot be isolated.  So, IMO, any solution needs to
account for or involve loadsharing (CMT or otherwise).

--brian

James M. Polk wrote:                        (Mon, 24 Jan 2011 19:31:09)
> (with my chair hat on)
>
> ISTM that the WG has to answer the following question before either this 
> "-nishida-" ID or the "-tuexen-...-multipath-" ID progress (and if either 
> does):
>
> I'm posing this more informally to the WG, but how I, as chair, take the 
> answer to this question can get more formal based on the answers given 
> (including if no one answers this question).
>
> Why do I pose this?
>
> doesn't need to be a book of an response, as that'll likely confuse more 
> of the WG...
>
> James

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 25 Jan 2011 00:25:06 -0800

Hello James,

I think the slides Jana used in his presentation at Maastricht
characterizes this well.
(http://tools.ietf.org/agenda/78/slides/tsvwg-2.pdf)
At least, it's my characterization for this one.

Also, sctp-failover can be applied to RFC4960 in addition to CMT.
(which might make things complicated? but I believe it's useful.)

Thanks,
--
Yoshifumi Nishida
[email-masked]

2011/1/24 James M. Polk <[email-masked]>:
> If I've mischaracterized this - please correct me concisely. The WG doesn't
> need to be a book of an response, as that'll likely confuse more of the
> WG...
>
> James

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 25 Jan 2011 19:08:02 +0800

On 01/25/11 09:31 AM, James M. Polk wrote:
> "-nishida-" ID or the "-tuexen-...-multipath-" ID progress (and if
> either does):
>

I think the ID "-nishida-" can be considered as "implementation
details."  The reason is that RFC 4960 does not prohibit a stack
from doing quick failover.  And as David pointed out before, there
are existing stacks which do it already.  Doing it does not require
co-operation from the receiver side.  There is not much side effect
(should not introduce congestion for example) on the network as
communication between the end points is still on one path.  Hence
it is not necessary for the WG to standardize a mechanism.  Note
that I am not objecting to such as effort, just that it is not
necessary IMHO.

OTHO, doing CMT properly and efficiently probably requires co-
operation from the receiver side.  As mentioned in the "-tuexen-...
-multipath-" ID, there are a couple obvious issues in doing CMT
in "standard" SCTP impl.  While things like avoiding false fast
retransmission may be mitigated on the sender side alone (albeit
a little bit inefficiently), things like delaying SACK require
receiver side changes.  It is a good idea if there are new protocol
features to facilitate CMT on both sides.  And since CMT can have
side effect on the network, it is important for this WG to evaluate
it.  Hence IMHO it is necessary to standardize CMT by this WG.

> doesn't need to be a book of an response, as that'll likely confuse more

I think they can be separated.  Ideally, an app can choose how
an SCTP stack utilizes the different paths.  If an app just wants
to use one path (say because alternate paths are expensive to use),
quick failover is applicable.  If an app wants to utilizes all paths
simultaneously, CMT is used and how to do retransmission becomes part
of the CMT algorithm.

-- 

					K. Poon.
					[email-masked]

---

From: "James M. Polk" <[email-masked]>
Date: Tue, 25 Jan 2011 13:16:35 -0600

in-line below

>simultaneously, CMT is used and how to do retransmission becomes part

I think I'm gathering that if these are separate efforts, they must 
align (i.e., one is the super-set of the other). That makes me 
question whether the use-case for the subset doc is worth doing if 
the super-set doc can be done effectively and in a timely manner.

Or am I oversimplifying things?

Are these two sufficiently different problem-spaces?

Obviously, if these two efforts are covering separate problem spaces, 
however close they are to each other, it appears we want to address 
both if we can. I need to understand if we need two efforts to cover 
problem spaces that are real close to each other, or can the efforts 
be combined into one effort?

James
(with my chair hat on)

---

From: "David Lehmann" <[email-masked]>
Date: Tue, 25 Jan 2011 14:56:55 -0500

IMHO, there are a few reasons to move forward with Quick Failover.
(1) Quick Failover seems to be less complex, which will allow more
timely approval/implementation. As Kacheong pointed out, "Doing it does
not require co-operation from the receiver side.  There is not much side
effect (should not introduce congestion for example)...".
(2) Although the current spec may allow quick failover, it implies an
unacceptably long wait time before failover.  I don't think Solaris or
Linux has quick failover.  Let's say that one does implement it. Now we
have two major implementations with quite noticeable behavior
differences.  IMHO, users should expect consistent behavior across all
SCTP implementations.
(3) As Dr. Dreibholz noted, CMT will also benefit from Quick Failover.
The CMT spec can reference Quick Failover.

--
David Lehmann
Ulticom, Inc.
856-787-2952

---

From: "James M. Polk" <[email-masked]>
Date: Wed, 26 Jan 2011 14:35:22 -0600

>The CMT spec can reference Quick Failover.

Do others agree with this position, with the caveat that anything 
within the multipath ID regarding Quick Failover has to be in line 
with the Quick Failover ID (i.e., they need to be in sync)?

I want to hear from many on this before proceeding with anything.

James
TSVWG chair

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 27 Jan 2011 17:58:39 +0800

On 01/27/11 04:35 AM, James M. Polk wrote:

> Do others agree with this position, with the caveat that anything within
> the multipath ID regarding Quick Failover has to be in line with the
> Quick Failover ID (i.e., they need to be in sync)?

No.  IMHO, quick failover does not need to be align with how CMT
chooses a path to do retransmission and vice versa.

-- 

					K. Poon.

---

From: Alberto Cortés <[email-masked]>
Date: Thu, 27 Jan 2011 11:30:27 +0100

On Wed, Jan 26, 2011 at 9:35 PM, James M. Polk <[email-masked]> wrote:
> Do others agree with this position, with the caveat that anything within the
> multipath ID regarding Quick Failover has to be in line with the Quick
> Failover ID (i.e., they need to be in sync)?
>

I´m not sure: Does it makes sense to use CMT and Quick Failover
*together*? If true, then they must be in sync, else they don´t need
to be in sync.

Example: Is there any scenario where an association is using CMT over
3 paths, and then (quick) failover to another 3 different new paths
(or any combination of new and old paths).

-- 
Alberto Cortés-Martín
Telematic Engineering Dept. at UC3M
http://www.it.uc3m.es/alcortes/index.html

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 14:32:13 +0100

On Jan 26, 2011, at 9:35 PM, James M. Polk wrote:
> Do others agree with this position, with the caveat that anything within the multipath ID regarding Quick Failover has to be in line with the Quick Failover ID (i.e., they need to be in sync)?
Hi James,

I think the Quick Failover makes sense for scenarios where CMT is currently
not deployed yet. These implementations focus on standards track protocols
and having a documented method for this, being a standards track document
does make sense. It can easily be deployed since it is a sender side only
thing. We already have the situation that FreeBSD, Linux and Solaris show
different behavior for detecting path failures. For writing portable
applications this is not useful.

Originally I was hesitating to support quick failover, as indicated in a
discussion at an IETF, but changed my mind. It would be helpful to have
a standardized method for this.

CMT also needs to detect a path failure. We could just refer the Quick
Failover ID for this. So we would avoid duplicating the work.

Please note the the intended status for the CMT ID is experimental.
Only have a failure detection in an experimental document does not
seem to be optimal to me.

I'm willing to continue contributing to the Quick Failover ID by
providing review comments and even text if it is applicable.
The Quick Failover ID needs a section for the socket API, for example.

Best regards
Michael

---

From: Thomas Dreibholz <[email-masked]>
Date: Thu, 27 Jan 2011 14:15:47 +0100

Dear all,

> differences.  IMHO, users should expect consistent behavior across all

Significant difference in implementation behaviour is a very bad thing. The 
experience from RSPLIB (i.e. the RSerPool implementation, see 
http://tdrwww.exp-math.uni-essen.de/dreibholz/rserpool/), which supports 
multiple operating systems and therefore different SCTP stacks, is that 
actually the application has to detect that there is no real transmission 
progress in case of a path failure -- and either wait for SCTP to switch the 
primary path (which takes unacceptably long for this kind of application) or 
trigger a session failover (expensive). From the application writer's 
perspective, a consistent behaviour of the underlying SCTP implementations 
would make things a lot easier.

> The CMT spec can reference Quick Failover.

This is currently done in draft-tuexen-tsvwg-multipath-sctp-01.

Best regards
-- 
=======================================================================
 Dr. Thomas Dreibholz

 University of Duisburg-Essen,                   Room ES210
 Inst. for Experimental Mathematics              Ellernstraße 29
 Computer Networking Technology Group            D-45326 Essen/Germany
-----------------------------------------------------------------------
 E-Mail:     [email-masked]
 Homepage:   http://www.iem.uni-due.de/~dreibh
=======================================================================

---

From: Armando Caro <[email-masked]>
Date: Thu, 27 Jan 2011 11:43:55 -0500

On 01/27/2011 08:15 AM, Thomas Dreibholz wrote:
> Significant difference in implementation behaviour is a very bad thing.

I think it's not ideal, but it may not be a very bad thing. It depends 
on how the application is written.

> From the application writer's

It might, but I don't think applications should optimize on assumptions 
about the parameter settings of the network stack.

With that said, I do think that the quick failover algorithm makes sense 
to adopt as a WG item. Research results (and I'm guessing practical 
experience) have shown little (any?) downsides to using this algorithm, 
and the advantages are clear.

Armando

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 19:11:23 +0100

On Jan 27, 2011, at 5:43 PM, Armando Caro wrote:

> I think it's not ideal, but it may not be a very bad thing. It depends on how the application is written.
> 
> It might, but I don't think applications should optimize on assumptions about the parameter settings of the network stack.
Hi Armando,

when writing an application using SCTP which has to fulfill some requirements
on failover time I have to tweak the transport parameters to get that feature.

Currently I have to do that very platform specific. Even the same parameter
means different things on different platforms.

So having a portable way to get some predefined failover behavior makes sense
to me...

Best regards
Michael
> 
> With that said, I do think that the quick failover algorithm makes sense to adopt as a WG item. Research results (and I'm guessing practical experience) have shown little (any?) downsides to using this algorithm, and the advantages are clear.
> 
> Armando
> 
> 
>

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 11:20:19 -0700

Michael,

Michael Tüxen wrote:                       (Thu, 27 Jan 2011 14:32:13)

> seem to be optimal to me.

That brings up another point: I think that the failover ID should
be INFORMATIONAL (if anything) and the CMT ID should be Standards
Track.

I see little point in making the CMT ID EXPERIMENTAL as there are many
implementations that have been running in the wild for over a decade;
and, particularly where some approaches have been identified by some
as harmful to the Internet (i.e. CMT through a shared bottleneck).

OTOH, the failover ID should be INFORMATIONAL because it is entirely
optional, is only one of many approaches for doing such a thing (none
of which have been identified as harmful); and, is a performance-only
related "implementation detail".

I agree with Kacheong's characterization of these IDs, and I believe
the change in intended status is consistent with that characterization.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 20:20:14 +0100

> be INFORMATIONAL (if anything) and the CMT ID should be Standards
I would vote for Proposed Standard for the failover ID and
Experimental for CMT.
> as harmful to the Internet (i.e. CMT through a shared bottleneck).
We would use the coupled congestion control from the MPTCP WG, which
is experimental. So I would see, at least initially, SCTP loadsharing
also as Experimental.
> related "implementation detail".
Only because it is PS doesn't mean that you have to implement it.
Supporting it is not mandatory.

Best regards
Michael

---

From: Jeff Morriss <[email-masked]>
Date: Thu, 27 Jan 2011 14:23:33 -0500

> that I am not objecting to such as effort, just that it is not

If there are many implementations already doing it then clearly there is 
a need.  (I know first hand of several such needs.)

As an SCTP user, I don't want my choice (if/when I have one) of which 
SCTP to use to be dictated by which implementations have gone beyond 
what is standardized to give me what I need.  And I don't want to be put 
in a difficult situation (well described in sections 4.1 and 4.2 of the 
I-D) when I don't have a choice of which SCTP to use.

In other words, I have the opposite conclusion: if several/many 
implementations have found it necessary then it _should_ be standardized.

> simultaneously, CMT is used and how to do retransmission becomes part

My $.02 as a user: basically every project where I've seen SCTP used has 
needed fast path failover.  Some could benefit from CMT, but it is (from 
my point of view from those projects) only an optimization.  It would be 
nice to have but I can live without it.  I have found that life without 
quick failover... Is painful at best but is sometimes impossible 
(requirements cannot be met).

---

From: Janardhan Iyengar <[email-masked]>
Date: Thu, 27 Jan 2011 14:31:36 -0500

James,

[I can review draft-nishida and contrib text.]

Some of these points have been said, but I'm simply adding my opinion to the pool.

In my presentation in Maastricht, I proposed that if CMT (draft-tuexen) was to be adopted, then draft-nishida could be brought in under that umbrella.  The PF-state may have performance impact  in SCTP, but it has significant impact on CMT performance when path failure occurs.  And so...

>>> "-tuexen-...-multipath-" ID addresses a superset of the problem area.

... I wouldn't consider draft-tuexen as addressing a superset of draft-nishida, but as a significant use case of draft-nishida.  Without draft-nishida, the PF mechanism would have to be defined in draft-tuexen.  draft-nishida can exist on its own, and if so, the mechanism will be used by CMT as well.

> I think I'm gathering that if these are separate efforts, they must align (i.e., one is the super-set of the other). That makes me question whether the use-case for the subset doc is worth doing if the super-set doc can be done effectively and in a timely manner.

They need to be "loosely aligned".  While I suspect it will not be necessary, the CMT authors can ensure that we consider any impact of draft-nishida on draft-tuexen, and raise them for consideration.  We can then simply cite draft-nishida, and not redefine the same mechanisms in draft-tuexen.

> Obviously, if these two efforts are covering separate problem spaces, however close they are to each other, it appears we want to address both if we can. I need to understand if we need two efforts to cover problem spaces that are real close to each other, or can the efforts be combined into one effort?

There are at least two reasons to consider two separate efforts:
(i) draft-nishida is useful for SCTP, even without draft-tuexen.
(ii) Practically, draft-nishida is simpler and thus likely to move faster through the wg than draft-tuexen.

- jana

-- 
Janardhan Iyengar
Assistant Professor, Computer Science
Franklin & Marshall College
http://www.fandm.edu/jiyengar

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 12:37:36 -0700

Michael,

Michael Tüxen wrote:                           (Thu, 27 Jan 2011 19:11:23)

> So having a portable way to get some predefined failover behavior makes sense

Your problem appears to be more of an API issue than a standardization
of "implementation details".  An API feature that indicates that the
application wants 'quickest failover' would appear to provide what you
need without unnecessarily constraining how the implementation achieves
that.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 12:55:00 -0700

Janardhan,

Janardhan Iyengar wrote:                      (Thu, 27 Jan 2011 14:31:36)
>    through the wg than draft-tuexen.

To date I don't think that anyone has identified the advantages of
PF over just setting PMR to 0/1.  Simply setting PMR to 0/1 appears
to deliver the same advantages in reducing failover times as PF
without any implementation changes.  This is true for both RFC and
CMT SCTP.

In particular the ID as it stands states that no data is sent to a
PF destination (just as in the RFC4960 PMR case), only HEARTBEATS
(just as in the RFC4960 PMR case).

So why is PF necessary at all?

Michael, why are you not just setting PMR to 0 on all implementations
when you need quick failover?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 20:59:15 +0100

> need without unnecessarily constraining how the implementation achieves
Brian,

the point is that sometimes I not only want what the implementation
considers 'quickest failover', but I want some time limits to be
satisfied. I agree, I do not care how this is done in detail, but
when we were investigating FreeBSD, Linux and Solaris, we had
to figure out how these implementations do path supervision and
how specific parameters in the API influence that. Based on that
we could write code which can achieve the performance constraints.
Part of the problem was API, this is smoothed out in the current
version of the socket API ID, but another part was just different
behavior. Solaris has some sort of fast path failure detection.

Having a document describing such a mechanism would possibly result
in having such a mechanism available on several implementations.
I can speak for the FreeBSD implementation: We would provide such
an implementation pretty soon such a document is on track for PS.
Some sort of it is already implemented, but it needs some adoption.
I would guess, it would also be implemented in Linux. I can't talk
about other implementations.
But every implementation is free to implement it or to implement even
mechanisms which behave better in some metric in specific situation.
This is the nice feature of the suggested method. It is sender side
only...

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 13:21:19 -0700

Michael,

Michael Tüxen wrote:                     (Thu, 27 Jan 2011 20:59:15)
> This is the nice feature of the suggested method. It is sender side

If it is "just a suggestion" why does it need to be Standards Track?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 21:24:37 +0100

> when you need quick failover?
When using PMR == 0 I would do a failover on a single timeout
and report the path as failed to the upper layer.
Using the PF state I can use a small value to move the traffic away
without having a lot of false alarms that a path is broken.

Another thing I can not get by setting PMR=0 is that if I saw
some errors on an idle path, I send the HEARTBEATs without
taking HB.Interval into account. This is something already
done by Solaris, I think. But it helps.

Best regards
Michael

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 21:26:47 +0100

> If it is "just a suggestion" why does it need to be Standards Track?
I consider everything described in an ID as a suggestion...
Maybe I should have used "of the method described in the ID". It is what
I meant.

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 13:37:54 -0700

Michael,

Michael Tüxen wrote:                     (Thu, 27 Jan 2011 21:24:37)
> without having a lot of false alarms that a path is broken.

Well PF "suggests" a similar indication anyway.  Really, when setting
PMR = 0 the path failure indication is not an indication of a broken
destination: it is an indication that the destination has been moved
to the inactive state (no longer offered traffic).  PF as "suggested"
does the same thing.

> done by Solaris, I think. But it helps.

I don't this this should be advised.  If the endpoint figures that
the destination is in temporary congestion, it should never _increase_
the rate at which it sends messages to that destination.

Is that all the advantages of this "suggestion"?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 13:40:52 -0700

Michael,

Michael Tüxen wrote:                                               (Thu, 27 Jan 2011 21:26:47)
> Maybe I should have used "of the method described in the ID". It is what

Buf if "every implementation is free to implement it", or not, or better
ones, how is it more than a suggestion or "implementation detail"?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 21:47:13 +0100

> ones, how is it more than a suggestion or "implementation detail"?
Yes. An SCTP implementation is free to implement RFC 4895 (SCTP-AUTH)
or RFC 5061 (ADD-IP) or not, both a proposed standard.
Same for RFC 3758 (PR-SCTP).

Best regards
Michael

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 27 Jan 2011 21:50:08 +0100

> to the inactive state (no longer offered traffic).  PF as "suggested"
But currently the notifications are defined as such...
> the rate at which it sends messages to that destination.
Why? It used the standards RTO based mechanism and behaves as
aggressive as it would we when using DATA chunks instead of
HEARTBEAT chunks.
> Is that all the advantages of this "suggestion"?
The ID does not describe anything complicated. But it is useful.

CMT is *much* more complex.

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 14:00:57 -0700

Michael,

Michael Tüxen wrote:                                               (Thu, 27 Jan 2011 21:50:08)

> But currently the notifications are defined as such...

Yes, they are defined in RFC4960 as:

   When a destination transport address is marked inactive (e.g., when
   SCTP detects a failure) or marked active (e.g., when SCTP detects a
   recovery), SCTP shall invoke this notification on the ULP.

So, it does just indicate whether the destination is active or inactive.
(Offered traffic or not).  Under RFC4960 an application can be sure that
if the primary is active, it is the only destination receiving traffic.
PF breaks that contract.  Setting PMR = 0 does not.  I view this as
an advantage of setting PMR = 0 over PF, not visa versa.

> aggressive as it would we when using DATA chunks instead of

But you are doing it while sending DATA aggressively to an alternate
destination.  You have the initial cwnd overgrowth of the changeover
(possibly through a shared bottleneck) plus the increase in HEARTBEAT
traffic.  This in fact increases the traffic sent into a network
suspected of experiencing congestion.

This is unnecessary.  Sending _one_ HEARTBEAT immediately serves to
restore destinations quickly that are experiencing very brief congestion
without significant traffic increase.

> CMT is *much* more complex.

I meant advantages of PF over PMR = 0.  I suppose there are no others.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 14:03:34 -0700

Michael,

Michael Tüxen wrote:                            (Thu, 27 Jan 2011 21:47:13)
> Same for RFC 3758 (PR-SCTP).

Yes, but of course, all of those change the on-the-wire protocol and
require IANA assignments and so must be Standards Track.  The PF ID
does neither.

So, what is the reason that you believe that the PF ID needs to be
standards track?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 27 Jan 2011 13:40:44 -0800

Hello Brain,

In the quick-failover draft, we addressed setting PMR to 0/1. (please
see section 4.1 if you're interested)
As we wrote in the draft, setting PMR to 0/1 would have some potential
issues. You'll need to think about AMR value,
dormant status behavior, keep-alive..I mean, you'll need to ignore
some part of RFC4960 and need to think about undocumented functions in
the spec to get good performance.

You might say smart implementers can properly address these issues and
solve them by their own way. That might be true.
But, I'm wondering why SCTP has to rely on the wisdom of the
implementors and why IETF cannot produce
a simple and easy-to-understand solution even though we already know the issue.

I think you could standardize PMR=0/1 solution if you want, but just
setting 0/1 would not be enough.

Thanks,
---
Yoshifumi Nishida
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 15:07:05 -0700

Yoshifumi,

Yoshifumi Nishida wrote:                   (Thu, 27 Jan 2011 13:40:44)
> the spec to get good performance.

I read the ID.  It discusses issues associated with setting PMR=0, but it
does not identify how PF is superior in any of those regards.  And, in
particular, as the same underlying mechanisms are used (just a different
named counter and a questionable each-RTO HEARBEAT procedure) it cannot
provide better performance than PMR = 0.

> a simple and easy-to-understand solution even though we already know the issue.

No, but I will say that Applications can solve these problems easily
by tuning the existing RFC4960 protocol parameters.

> setting 0/1 would not be enough.

Other protocol parameters can be tuned as well to ahieve even better
performance than PF; however, it does not need to be standardized because
it already is: RFC 4960.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Janardhan Iyengar <[email-masked]>
Date: Thu, 27 Jan 2011 18:02:15 -0500

Brian,

>> So why is PF necessary at all?

In addition to what Michael just said about the semantic difference between a PF state and a Failed state to both SCTP and to the application, a subtle point of difference follows.

The PF state drives heartbeats using the data-timer to the PF destination, which is exponentially backing off, as against driven by the HB-timer, which is very slow.  This matters because
simply setting PMR to 0 is aggressive, and on paths that experience timeouts periodically, setting PMR to 0 results in recovery using the slow HB-timer.

- jana

---

From: Janardhan Iyengar <[email-masked]>
Date: Thu, 27 Jan 2011 18:22:31 -0500

Brian,

> (Offered traffic or not).

But that is precisely the point:  with PF, the path is NOT marked inactive; traffic may be offered soon.  A single timeout is not a big deal on many Internet-like paths, and it is definitely not an indication of path failure.  In such a case, the notification to the ULP about path failure becomes too noisy.

There are two things going on here:  what should an SCTP sender do on timeout, and when should a path failure be indicated.  The first one is a performance question, and the second one is a semantic question.

>  Under RFC4960 an application can be sure that

There is no such contract under RFC4960, since retransmissions can still go to alternate destinations.

> suspected of experiencing congestion.

Cwnd overgrowth happens only if there was a fairly large window oustanding on the old path when the "changeover" is initiated.  No cwnd overgrowth will happen here because there was probably a long pause in sending due to the timeout on the old path.

> without significant traffic increase.

Yes, sending one HB immediately is important.  All PF does is use the RTO timer to send more HBs following this first HB.

- jana

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 16:47:14 -0700

Janardhan,

Janardhan Iyengar wrote:                  (Thu, 27 Jan 2011 18:02:15)
> In addition to what Michael just said about the semantic difference
> between a PF state and a Failed state to both SCTP and to the
> application, a subtle point of difference follows.
>
> The PF state drives heartbeats using the data-timer to the PF
> destination, which is exponentially backing off, as against driven by
> the HB-timer, which is very slow.  This matters because simply setting
> PMR to 0 is aggressive, and on paths that experience timeouts
> periodically, setting PMR to 0 results in recovery using the slow
> HB-timer.

With PMR=0 this is simple:  When the application receives the NETWORK
STATUS CHANGE indication that the destination is inactive, it simply
uses the "Change Heartbeat" request (RFC4960/10(I)) to change
HB.interval for that destination to 0.  It can also use the "Request
HeartBeat" request (RFC4960/10(J)) to request an immediate heartbeat.
When it receives the NETWORK STATUS CHANGE indication that the
destination is active, it simply uses the "Change Heartbeat" request
(RFC4960/10(I)) to set the heartbeat interval back to its previous
value.

Anything else?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 16:49:40 -0700

Michael,

> done by Solaris, I think. But it helps.

But you will have to notify PFMR events anyway.

Simply use those pesky NETWORK STATUS CHANGE indications to set
the HB.interval to zero when the destination is inactive.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 27 Jan 2011 17:03:47 -0700

Janardhan,

Please see comments below...

Janardhan Iyengar wrote:                   (Thu, 27 Jan 2011 18:22:31)
> But that is precisely the point:  with PF, the path is NOT marked
> inactive; traffic may be offered soon.

There is no difference with PMR = 0, the destination is marked inactive,
but traffic may still be offerred soon (on the next ack).

> A single timeout is not a big deal on many Internet-like paths, and it
> is definitely not an indication of path failure.

No amount of timeouts is definitive indication of a path failure.
Timeouts can always be caused simply by delay.

> In such a case, the notification to the ULP about path failure becomes
> too noisy.

Subjective.  The notifications can be used to set the HB.interval to
what PF would, even issue an immediate HB.  PF the way it is proposed
provides for a ULP indication on PFMR anyway.  PMR = 0 is less noisy
because the PFMR indications are not followed up by PMR indications.

> There are two things going on here:  what should an SCTP sender do on
> timeout, and when should a path failure be indicated.  The first one
> is a performance question, and the second one is a semantic question.

What an SCTP sender does on a timeout is really no different for PMR=0
than PF.

The semantic question is purely local and API related.

> still go to alternate destinations.

When PMR is set to 0 there is: because all RTO retransmissions cause
the primary to go inactive.

> Cwnd overgrowth happens only if there was a fairly large window
> oustanding on the old path when the "changeover" is initiated.  No
> cwnd overgrowth will happen here because there was probably a long
> pause in sending due to the timeout on the old path.

Yes.  I may have used the alternate path recently and its usability
is over 30 seconds old because my HB.interval is set to 30 seconds.
The path to all destinations is a shared bottleneck and there will
be a cwnd overgrowth.  Under PF as it stands, I also increase HB
traffic to the old destination (on the same path) just to kick the
network while its down.

> Yes, sending one HB immediately is important.  All PF does is use the
> RTO timer to send more HBs following this first HB.

With PMR = 0, the application can use NETWORK STATUS CHANGE indication
to issue a "Request Heartbeat".

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 27 Jan 2011 16:53:12 -0800

Hi,

> performance than PF; however, it does not need to be standardized because

I personally prefer to say "this is vanilla SCTP and it works well"
rather than "Please don't forget tuning before use it"
(also, I'm not very sure just tuning parameters can get good
performance...but, we can discuss later.)

Thanks,
--
Yoshifumi Nishida
[email-masked]