Subject: Re: [Tsvwg] SCTP applicability statement

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 03:58:20 -0600

Coene,

The applicability statement has the following text:

    3 Multihoming Issues

	...

	It should be noted that SCTP multihoming support only deals with
	communication between two endpoints of which one or both is assigned
	with multiple IP addresses on possibly multiple network interfaces.
	It does NOT deal with communication ends that contain multiple
	endpoints (i.e., clustered endpoints) that can switch over to an
	alternate endpoint in case of failure of the original endpoint.

As it is fundamental to the design of SIGTRAN UAs that such redundancy
between hosts be supported, this statement appears to indicate that
either SCTP is unsuitable for SIGTRAN UAs, or SIGTRAN UAs must include
their own support for handling clustered endpoints (e.g., UA queueing
and acknowledgements).

This is a significant issue with all the SS7 SIGTRAN UAs.  Is the
applicability statement saying that SIGTRAN must layer acknowledgements
over SCTP to accomplish this because no support is provided by SCTP?

--brian

On Wed, 24 Oct 2001, Coene Lode wrote:

> > Folks,
> > 
> > The SCTP applicability statement has been reissued incorporating all the
> > comments coming from the AD's
> > (Thank you Allison and Scott). As you will notice, it looks a lot
> > different from the previous release. 
> > So have a look at:
> > http://www.ietf.org/internet-drafts/draft-ietf-sigtran-sctp-applicability-
> > 07.txt
> > 
> > Both the SIGTRAN and TSV mailing lists are included as both have workitems
> > dealing with SCTP.
> > If you have comments, you can send them to me. 
> > 
> > yours sincerly,
> > Lode Coene
> > 
> > Siemens atea ICN D CS D
> > Software development
> > Atealaan 34
> > B-2200   Herentals    Belgium
> > Tel: +32-14-2-52081   Fax: +32-14-2-53212
> > E-mail: [email-masked]
> > 
> 
> _______________________________________________
> tsvwg mailing list
> [email-masked]
> http://www1.ietf.org/mailman/listinfo/tsvwg

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 10:21:43 -0500

>over SCTP to accomplish this because no support is provided by SCTP?

        The Internet Architecture does not have a concept of
"clustered endpoints".  So if one is building such a beast, one
ought to include appropriate *internal* mechanisms (i.e. not visible
to the outside, as changes to protocols would be) to make it work 
properly.

        This isn't necessarily rocket science.  However, just as the 
methods for clustering vary widely from one implementation to another, 
the internal methods for ensuring that everything works properly 
in event of failover will need to vary.  Thus, such mechanisms
do not seem ripe for standardisation in IETF.  If one wanted to
standardise clustering methods, IEEE would seem like a more suitable
venue as they have ongoing operating system standards efforts
(and it looks like an OS problem).

        In any event, a properly designed and implemented clustered
system ought to look like a single system to the outside.  That is,
there *ought not* be special protocol changes that would apply only
to clustered systems.

IMHO,

Ran
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 13:31:16 -0600

RJ,

> to the outside, as changes to protocols would be) to make it work 

What do you mean by *internal*?   *Internal* to what?

> (and it looks like an OS problem).

Are you suggesting that the SIGTRAN WG be dechartered and sent to IEEE?

> there *ought not* be special protocol changes that would apply only

They why does SCTP not support this if special protocol changes *ought
not* be required?

--brian

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 15:36:56 -0500

>What do you mean by *internal*?   *Internal* to what?

        Internal to the "Cluster Operating System".

>Are you suggesting that the SIGTRAN WG be dechartered and sent to IEEE?

        No.  I gather you either didn't understand or didn't read 
what I wrote.  I'll try again with an effort to use simpler language.

        I'm suggesting that the details of how a cluster operating system 
is built are not IETF work and that those details should be kept hidden 
under the hood of the given cluster (i.e. not be visible on-the-wire).
SIGTRAN isn't chartered to design a cluster operating system, nor
is SIGTRAN chartered to modify transport protocols to solve clustering.
Clustering computer systems is an OS issue not a protocol issue.

>They why does SCTP not support this if special protocol changes *ought

        I did not say that the OS would not need modification.  The
difference between old VMS and clustered VMS was that the OS got
modified to support clustering.  DEC did not change on-the-wire TCP 
when they added cluster support to VMS.  Similarly, on-the-wire SCTP
ought not be changed when adding cluster support to some other OS.

        By definition, if SCTP explicitly supported this, then the 
SCTP protocol would require "special protocol changes that apply only 
to clustered systems".  

        It is up to whichever folks are designing the given clustered
system and its OS to make the cluster look like a single system to
the outside world.  Nothing in the SCTP specs prevents one from doing
that in a myriad of ways (I can think of several approaches that could
work fine).  The SCTP specs simply say that building a cluster is not
within the scope of SCTP -- quite right, building a cluster is an OS
and computer systems problem rather than a protocol problem.

Ran
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 14:58:19 -0600

RJ,

RJ Atkinson wrote:                                             (Sun, 04 Nov 2001 15:36:56)
> Clustering computer systems is an OS issue not a protocol issue.

But this is what SIGTRAN *has* done by havinng multiple ASPs and SGPs
with loadsharing, broadcast and active/standby between ASPs and SGPs
as members of the cluster.  This is also what rserpool does.  Yet it
appears that you disagree with this approach.

What is required to support ASPs and SGPs in the model presented by
SIGTRAN is, indeed, the ability to fail over *between* associations.
Support for that is lacking in SCTP and the AS appears to side-step
the issue with it's statement that SCTP only supports failover within
a multi-homed association.

> SCTP protocol would require "special protocol changes that apply only 

Does SCTP support failover between multi-homed hosts?

> and computer systems problem rather than a protocol problem.

Is failover between multi-homed hosts within the scope of SCTP?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Allison Mankin <[email-masked]>
Date: Sun, 04 Nov 2001 16:22:14 -0500

> Is failover between multi-homed hosts within the scope of SCTP?

Brian,

SCTP's multi-homing has no relation to anything between*
hosts, only between interfaces of one host.  There might be
some kind of cluster that appears to its SCTP peer be one host 
with multiple interfaces, but it would not be within the scope
of SCTP to know about the cluster. 

In draft-ietf-aaa-transport-04.txt, section 3.2. there's a 
good discussion of failover/failback, which distinguishes
among:

- what a cluster does in its cluster OS or cluster communication
  system 
- what application-level watchdogs do, where the failover is among
  multiple potential peers (here AAA servers)
- what SCTP does

If all interfaces of an SCTP host are down, that's the second case.

Allison

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 15:33:57 -0600

Allison,

So, is your answer no?

Is it fair to say that SCTP does not support failover between
multi-homed hosts, between associations, your second case?

Are you saying that applications such as SIGTRAN, RSERPOOL, AAA,
must implement their own sequence numbers and acknowledgements
and retransmission queues between associations?

--brian

Allison Mankin wrote:                                     (Sun, 04 Nov 2001 16:22:14)
> ---
> You are currently subscribed to sigtran as: [email-masked]
> To unsubscribe send a blank email to [email-masked]

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 16:32:40 -0500

At 15:58 04/11/01, Brian F. G. Bidulock wrote:
>as members of the cluster.  This is also what rserpool does.  

I disagree with your assertion that defining how to build
a cluster computer is what has been done by those efforts.

Ran
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 15:53:14 -0600

RJ,

Is it the intention that  SCTP support what is being done by
those efforts without requiring them to institute application
-level acks, queues and retransmission for failover between
associations?

Should not SCTP's support (or lack of support) for SIGTRAN
protocols be mentioned in the AS?

As it stands it is not clear whether SCTP is even suitable to
transport the protocols that it was developed to transport...

--brian

RJ Atkinson wrote:                                             (Sun, 04 Nov 2001 16:32:40)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 16:17:57 -0600

Coene,

The applicability draft says:

    If possible the endpoint should always select the source
    address of the packet to correspond to the IP address of
    the Network interface where the packet will be emitted.

In general this is a bad idea.  An SCTP message must have a source
address that was provided in the INIT/INIT Ack exchange or which was
added using the ADDIP approach otherwise the receiver may treat
the message as not belonging to the association (e.g., OOTB).

What happens in the general case where the NIC address is not part
of the source address list for the association?  What is meant by
"if possible"?

This situation is better addressed in the RFC and this sentence
should be struck from the AS because it is misleading.

--brian

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 17:48:49 -0500

>protocols be mentioned in the AS?

        I do not see that such would be strictly required.
SCTP is a general-purpose transport-layer protocol.

>transport the protocols that it was developed to transport...

        Disagree with your unstated assumption that SCTP exists 
only for SS7 purposes.  SCTP is designed as a general-purpose 
transport-layer protocol.  Its potential applicability is wide
and general.  For example, SCTP is being mentioned in some proposals 
in the multi6 WG as a potential way to support multi-homed hosts
more effectively in the IPv6 community.

Ran
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 17:06:55 -0600

RJ,

RJ Atkinson wrote:                                             (Sun, 04 Nov 2001 17:48:49)
> more effectively in the IPv6 community.

I think you need a history lesson.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 18:02:30 -0500

At 17:17 04/11/01, Brian F. G. Bidulock wrote:
>In general this is a bad idea.  

        Disagree.  One generally wants the recipient to know the 
address actually used, so that return packets have the best chance of
reaching a currently valid network interface.  This is particularly
important when there are two network interfaces and one is temporarily
down (e.g. first hop Ethernet switch failed, which is something 
detectable by the originating host's Ethernet interface)).

>An SCTP message must have a source

        It had better be the case that the source address used in 
actual fact is BOTH (1) the actual interface the packet is send out and 
(2) part of the source address list for the SCTP session association.

Ran

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 18:07:23 -0500

>I think you need a history lesson.

        Nope.  I know the history.  And I know that there was an
explicit decision made to design a general-purpose protocol (with
SCTP as the result).  I also know that some folks (perhaps you included)
wished that SCTP were ONLY for use with SS7.  That proposal for SCTP 
to be used ONLY for SS7 did NOT fly with the IETF community at large.  
In practice, I think SCTP will be used as much or more for applications 
other than SS7.  One of the Transport ADs has indicated that the
general-purpose nature of SCTP is part of the reason that SCTP 
was explicitly moved out of SIGTRAN and into TSVWG.

        Now one might not like that history, but dislike doesn't change
the facts on the ground...

Ran

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 17:31:17 -0600

RJ,

> (2) part of the source address list for the SCTP session association.

      192.168.0.10      __  /\  __      192.168.0.12     
        ______         /  \/  \/  \        ______        
       |      |       /            \      |      |       
       | NIC1 |-------\            /------| NIC3 |       
       |______|        \          /       |______|       
                       /          \                      
      192.168.0.11    /            \    192.168.0.13     
        ______        \            /       ______        
       |      |        \          /       |      |       
       | NIC1 |--XXXX---\___/\___/--------| NIC4 |       
       |______|                           |______|       

If I have and association between 192.168.0.11 and 192.168.0.13
as per the above, there is absolutely no reason to use source
address 192.168.0.10 for packets sent to 192.168.0.13 when NIC1
fails.  In fact, using source address 192.168.0.11 will allow
the association to continue.

In this (one of many) examples, using the source addres of
the NI upon which the packet is emitted is counter-recommended.

In fact, the implementation may use whatever source address it
chooses.  If it wishes the association to stay alive it will use
one of the source addresses that corresponds to the SCTP endpoint
that the message is being sent for, regardless of what addresses
are asigned to which network interfaces.

Stike the sentence.

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 17:33:42 -0600

RJ,

I would be happy if it *could* be used for SS7 as well, but it
has some problems, and its applicability appears to exclude SIGTRAN.

--brian

RJ Atkinson wrote:                                             (Sun, 04 Nov 2001 18:07:23)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 21:22:33 -0500

>has some problems, and its applicability appears to exclude SIGTRAN.

        I'm open to being persuaded that SCTP has specific problems, 
IFF someone would construct a specific detailed valid general scenario 
where SCTP is "broken".  So far, such a case has NOT been presented 
here.

Ran
[email-masked]

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 21:19:48 -0500

At 18:31 04/11/01, Brian F. G. Bidulock wrote:
>fails. 

        Wrong.  Your example is broken by not being sufficiently
general (e.g. assuming that all the interfaces are on the same 
IP subnet -- which is generally not the case for an SCTP session).

        The reason is that in the general case NIC1 and NIC2 are 
on the same non-clustered host but are also on diverse subnets 
(which is desirable for routing robustness) AND NIC3 and NIC4 are
on the same non-clustered host but are also on diverse subnets
(for the same reason).  In this general case, packets destined 
to NIC1 will not be delivered generally (due to the way the ARP 
standards, IPv6 Neighbor Discovery standards, IP routing, 
and so forth work).  

        So switching to use the still-valid address as the source
address is desirable in the general case.  Note well that it is
also obviously the case that the SCTP session ought to be configured
to list all valid IP addresses for each SCTP endpoint.

> In fact, using source address 192.168.0.11 will allow

        Not so, considering that NIC1 and NIC2 are on different
subnets in the general case.  So the host connected to NIC1 
will not be able to respond to ARPs for IP address of NIC1
(because NIC1 is down and NIC2 is generally on a different IP subnet
hence can't proxy for the first interface).

        After timeout seconds, the ARP entry associated with 
IP address of NIC1 will get flushed by that last-hop router and 
then packets destined to that IP address of NIC1 will be dropped 
or returned (e.g. via ICMP Dest Unreachable) by the last-hop router.  

        Thus your proposed change impedes robustness in the
general case.

>the NI upon which the packet is emitted is counter-recommended.

s/counter-recommended/recommended/

See above analysis for why.

Cheers,

Ran
[email-masked]

---

From: Allison Mankin <[email-masked]>
Date: Sun, 04 Nov 2001 21:41:38 -0500

Brian,

> So, is your answer no?

Yes, my answer is no :) - SCTP is not intended to fail over between
different associations - the protocol reliability is within
the association.  With the endpoints being on different boxes,
that is, in arbitrary network locations, it would be a different
protocol to do what you request.

But multiple SCTP associations can still be used as failovers
for each other, through use of an application layer watchdog and
a short list of duplicate transactions that might need to be sent.
The section on aaa transport that I recommended (3.2 in
draft-ietf-aaa-transport-04.txt) is about a robust way to do this.

> multi-homed hosts, between associations, your second case?
Yes, that's fair.  But to repeat, those different associations have
arbitrary endpoints.   How do SIGTRAN applications know what to make
of a set of message that start arriving suddenly because failover has
occurred - there must be application support of backup server use.

> and retransmission queues between associations?
No, I think SIGTRAN and RSERPOOL apps (and aaa, by way of the draft I
recommended you look at) can use multiple associations with an application
layer algorithm that is different (and simpler) than sequence numbers,
acks and transport retransmission queues.  

There needs to be duplicate detection of transactions, but failing over
within the SCTP association can result in duplicates too.  

I think you should look at the algorithm I've recommended and compare it
with the specific requirements.  

Allison

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 21:12:41 -0600

Allison,

A couble of remarks on the watchdog timer from AAA and its
(lack) of applicability to SS7 UAs:

1)  The timer is 30 to 60 seconds.  SS7 UAs must normally
    failover with 0.1-0.5 seconds.

2)  The watchdog timer is an application transaction timer.
    SS7 UAs have no implicit acknowledgement or transaction
    structure within the message flow that is usable by M2PA,
    M2UA, M3UA or SUA in determining when to cancel the
    watchdog timer.  Data in these UAs is unstructured (or
    at least its structure is not known to the UA).  A sequence
    number would need to be added to messages and acknolwedged
    separately to cancel the timer.

Because there is no structure know by the UA (no separate
transaction and response as there is in AAA), sequence numbers
and acknowledgements would be required for each message.
Then the watchdog timer reduces to a retransmission timer.

So, it appears that you are saying that SIGTRAN protocols must
implement sequence numbers, acknowledgements, queueing and
retransmission after all.

Are you sure this is what you wish to recommend rather than
fixing several problems in SCTP?

--brian

Allison Mankin wrote:                                     (Sun, 04 Nov 2001 21:41:38)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 21:21:41 -0600

RJ,

It has been presented here (TSVWG and SIGTRAN) and will be presented
again:

For failover between associations the ULP needs to be able to
do the following things:

1)  Withhold acknolwedgements at ULP control for intervals longer
    than several seconds without aborting or otherwise degrading
    an SCTP association.  RFC 2960 says that all new DATA messages
    MUST be acknowledged and sets upper bounds (usually 200ms) on
    any acknowledgement.

2)  Cause the remote peer to stop transmitting new data.  RFC 2960
    permits an association to send new DATA regardless of whether
    it has receive a_rwnd = 0.

3)  Have sent messages acknowledged or have the ability to retrieve
    unsent and unacknowledged messages after an ABORT.  This is
    currently possible and is largely the responsibility of the
    implementation.

Failure to provide (1) and (2) means that an ULP cannot stop the
movement or clearing from the peer's retransmit buffer messages
that cannot be processed locally.

So, please fix those two problems with SCTP so that it can be
used for SIGTRAN UAs.  Solutions have even been proposed: a solution
to (1) is the continguous Gap Ack approach that was dicussed on this
list several months ago.  The solution to (2) is to enforce that
the sender MUST NOT send new data when rwnd = 0 and can only use old
data (or an acked TSN) to "probe" for lost SACKs.

Have any problems with that?

--brian

RJ Atkinson wrote:                                             (Sun, 04 Nov 2001 21:22:33)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Allison Mankin <[email-masked]>
Date: Sun, 04 Nov 2001 22:33:51 -0500

>     failover with 0.1-0.5 seconds.
You can't get this timing across arbitrarily placed boxes doing
the backup for each other - network failures that reliable
transports deal with don't lend themselves to that kind
of recovery in the Internet.  

Must-Ask Question: 
Why does this app want to use multiple associations (on multiple
boxes) rather than one association, lots of interfaces, lots
of redundancy, on (conceptually) one box.  (Conceptually because
it can be a cluster that looks like one box to its network peer).

AAA has mostly idempotent messages and values having a mesh
of redundant, unsynchronized servers.

>     separately to cancel the timer.

You could cancel the timer just by any* data coming back from
the peer - it does not depend on transaction acks - the SCTP
reliability means that the messages are getting through.

> retransmission after all.
No.   Please answer the Must-Ask Question.  

> fixing several problems in SCTP?

SCTP is far from perfect...but I don't think it set out to or
should provide a reliable protocol for Internet-wide fault tolerance.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 21:59:19 -0600

Allison,

Allison Mankin wrote:                                     (Sun, 04 Nov 2001 22:33:51)
> of recovery in the Internet.  

SCTP's application is also for the LAN and WAN.

> it can be a cluster that looks like one box to its network peer).

Because the SS7 network that it must interwork with is
a dual-plane network with redundant transfer/relay nodes
placed at a geographic distance.  Short answer: because
it must interface to two distinct and distant boxes.

> of redundant, unsynchronized servers.

SIGTRAN has telephone switches, MGCs, SCPs and real-time
control of MGs.

> the peer

During these failures message flow in one direction is ok.
Cancelling on any data coming back would not provide any
reliability for sent messages *across* associations.

> - it does not depend on transaction acks - the SCTP

But they do not get past the other end of SCTP.  That is
the application at the other end of SCTP can no longer
process messages via this association.  If the sender places
the messages on an alternate then things will be ok.  The
problem is in the switchover.  ABORTing the association
is not an options, becuase brief processor congestion is
one of the causes.

> No.   Please answer the Must-Ask Question.  

What is your solution?

> should provide a reliable protocol for Internet-wide fault tolerance.

It was certainly set out to handle SS7 changeover, but does not.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 22:02:28 -0600

RJ,

My example shows a case where the source address should
not be used.  The recommendation should be removed because
its applies in some configurations and does not in others.

Coede?

--brian

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 04 Nov 2001 23:04:12 -0500

>its applies in some configurations and does not in others.

        Even in your example, availability is enhanced by
using the actual interface IP address, because of the way that
ARP, IP routing, IPv6 Neighbor Discovery, and such work.

        Your proposed edit has no sensible example case
that I can identify.  And you haven't shown a valid rationale
for that edit yet here.  If you can actually find and document
one (I doubt it exists), I'm open to changing my mind.

        Allison's sundry comments are also on-point here.

Ran
[email-masked]

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 4 Nov 2001 23:08:23 -0600

RJ,

> ARP, IP routing, IPv6 Neighbor Discovery, and such work.

No, in the example, the association will fail if the
actual interface IP address is used.

> one (I doubt it exists), I'm open to changing my mind.

Case already presented.  You just missed it somehow.

>         Allison's sundry comments are also on-point here.

Boy, your adversarial: Allison did not comment on this.  I think
you should let her speak for herself.

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 05 Nov 2001 07:40:54 -0500

>actual interface IP address is used.

        Wrong.  Your example is flawed.

>Case already presented.  

        No.  You presented different arguments, which didn't hold
water.  Emphasis above on the words "sensible" and "valid"...

Ran

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 06:52:35 -0600

Ran:

I agree 100%.. On top of this the SIGTRAN UA's (as I remember them
at least from a while ago) had a concept of ASP's where one
had multiple of them and they would handle this kind of thing
without necessarily being clusters.

This is also where the RSERPOOL working group got started
and is working towards...

...a very cheap session layer fail-over mechanism...

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 07:03:58 -0600

Brain:

Comments below..

> a multi-homed association.

Ahh.. so the ASP's still do exist in the architecture.. then this
can solve your problem with small help from the transport layer 
giving you what was sent and queued... when a failure occurs..

> > ought not be changed when adding cluster support to some other OS.

It should not have to be. see below..

> Does SCTP support failover between multi-homed hosts?

I think there is fundamentally a problem in this dicussion on
what a "cluster" is.

A)Is a cluster a group of computers that give 
an appearance to the outside internet has a
single computer.

or

B)Is a cluster a logical group of processes scattered around
the internet providing a service.

In the latter case I think this is what the ASP concept is
about and it is ALSO what RSERPOOL is all about.

In the former case you can roll whatever you want to do clustering.
I don't see you needing to standardize how you build up multiple
computers that looks like a single machine to the outside world.
You can do what you like i.e. you can take the Tandem triplex
voting mechanism or anything else you dream up. If it truely
looks like a single computer then SCTP and TCP should work
just fine across all the components as well.

> Is failover between multi-homed hosts within the scope of SCTP?

If you are defining the cluster as in (A) above, then SCTP will
work just fine.

If you are defining the cluster as in (B) above, then this is
what the ASP and RSERPOOL concept is for. You break the problem
in to pieces. SCTP cannot be easily written to keep transport
state synchronized across the Internet between two machines. Instead
you put in your PE/PU concept (or ASP concept for SIGTRAN) that
divides the problem to make it more managable..

R

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 07:08:40 -0600

Brian:

I think the if possible is trying to state that
IF the address of the interface is bound to the association
(i.e. the peer knows about it). I don't think we should
strike the sentence.. but clearify it.

R

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 07:21:12 -0600

>     MUST be acknowledged and sets upper bounds (usually 200ms) on

Yes, and has Tom George pointed out one can do as mentioned by
NOT advancing the cum-ack and putting new chunks in GAP acks.
This of course penalizes our cwnd growth since the cwnd is
not allowed to grow unless the cum-ack advances but this
should not be a problem for sigtran. And there may be some
implementations that abort when this occurs.. but it is not
wrong so you should NOT abort really..

>     it has receive a_rwnd = 0.

As I mentioned before this is a arguable point but I will gladly
add the clearification to the implementors guide ..

>     currently possible and is largely the responsibility of the

The socket api that is a wg item allows this.. I do need to
get the latest version in though... 

So I don't see any of your problems as problems.. they all
have been addressed..

R

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 07:26:34 -0600

> to list all valid IP addresses for each SCTP endpoint.

I agree 100% here Ran... the problem with the applicability statement
is its ambiguity on the "if possible" in the statement in question. We
need to clearify this to state that the possible is when the address
is bound to the association. We actually discuss this in the book
Qiaobing and I have written on SCTP that is finally out... and give
examples :>

>         Thus your proposed change impedes robustness in the

And if all addresses are bound on both sides using .10 as a
source address is fine for both cases (same subnet or not) since
they are all valid addresses for the association.

> s/counter-recommended/recommended/

I agree :-)

R

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 07:28:06 -0600

Ran:

The only time I can see Brian's comment even remotely
applying is when the NIC is not bound to the association which
is where the "if possible" is vague and needs clearification...

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 5 Nov 2001 08:12:53 -0600

Randall,

That was the example (one NIC not "bound" but available).

Please clarify the "If possible" as "When the interface
IP address is also one of the addresses that are bound
to the association,"

--brian

---

From: Chip Sharp <[email-masked]>
Date: Mon, 05 Nov 2001 09:57:04 -0500

> >In general this is a bad idea.
...snip...

>(2) part of the source address list for the SCTP session association.
...snip...

Unless (for (1)) of course, you are using a virtual interface on a router.
:-)

---

From: Chip Sharp <[email-masked]>
Date: Mon, 05 Nov 2001 10:40:28 -0500

Sorry if I take a somewhat different view and agree with some of what Brian 
says.
:-0

...snip...

There is at least one real-world case in which I will not use the IP 
address of the transmitting NIC as the source address of the SCTP 
packet.  I've modified Brian's diagram below.  The difference is that the 
left-hand side is a router:
                                 __  __  __
               192.168.0.10     /  \/  \/  \      192.168.0.12
                +------+       /            \      +------+
            +-- | NIC1 |-------\            /------| NIC3 |
x.10.10.1  |   +------+        \          /       +------+
+------+   |                   /          \
|  VI  +---+                  /            \
+------+   |   192.168.0.11   \            /    192.168.0.13
            |   +------+        \          /       +------+
            +-- | NIC2 |--XXXX---\___/\___/--------| NIC4 |
                +------+                           +------+

In this case, a router has two NICs (NIC1 & NIC2).  It also has a Virtual 
Interface (x.10.10.1) representing an SCTP endpoint in the router.  The 
router announces the address x.10.10.1 on its source address list. When it 
transmits SCTP packets from the VI, it uses the VI's IP address.  The 
routing algorithm selects the correct NIC for transmission based on 
destination IP address.  If NIC2 goes down, and the destination is 
reachable over NIC1, then the router would send the packet out NIC1.  Of 
course, the VI address has to be reachable from NIC3 & NIC4.

I'm not saying that in general this is a good idea.  These days I'm feeling 
that there are very few good ideas in general (generalize a good idea 
enough and you get a bad idea).   There is nothing I know of in the 
applicability statement or RFC that would preclude this.

Chip
No amount of tweaks to the spec will fix a bad network design.

---

From: Randall Stewart <[email-masked]>
Date: Mon, 05 Nov 2001 10:04:30 -0600

Chip:

But you miss my earlier point.. in the case you
illustrate neither the address of NIC1 or NIC2 are
bound to the association. Instead VI is the ONLY
address in the association. This is the "if possible"
clause that Lode had in the document. I have sent Lode
a proposed text change that would cleaify things.

Current wording is:
"
   end-points, the multihoming feature needs more than one IP network
   interface for each endpoint. The number of paths used is  the minimum
   of network interfaces used by any of the endpoints. When  an endpoint
   selects its source address, careful consideration must be taken.  If
   the same source address is always used, then it is possible that the
   endpoint will be subject to the same single point of failure. If
   possible the endpoint should always select the source address of the
   packet to correspond to the IP address of the Network interface
   where the packet will be emitted.
"

Maybe something like:

"
    end-points, the multihoming feature needs more than one IP network
    interface for each endpoint. The number of paths used is the minimum
    of network interfaces used by any of the endpoints. When  an
endpoint
    selects its source address, careful consideration must be taken.  If
    the same source address is always used, then it is possible that the
    endpoint will be subject to the same single point of failure. When
the
    endpoint chooses a source address it should always select the 
    source address of the packet to correspond to the IP address 
    of the Network interface where the packet will be emitted subject
    to the binding address constraint.  The binding address constraint
    is put simply that the endpoint must never choose a source address
    that is not part of the association i.e. the peer endpoint must
    recognize any source address used as being part of the association.
 "

Polishing still being needed :>

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 5 Nov 2001 15:09:38 -0600

But, Randall,

how is this different from the RFC and why does it need
to be repeated in the AS?

--brian

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 5 Nov 2001 17:55:14 -0600

Qiaobing,

As I said (although more long-winded) in my previous note, please
set SIGTRAN straight on this matter and write in no uncertain terms
that these are the responsibility of the ULP.  Then we just add
acks, retrans and ULP flow control to M2PA and it will run like a top.

You are actually preaching to choir.

--brian

Qiaobing Xie wrote:                                      (Mon, 05 Nov 2001 17:35:05)
> This is the kind of common misconception flying around often, that if
> you use a reliable transport (TCP or SCTP), then you no longer need to
> worry about app-level timers, acks, retrans, heartbeats, integrity
> checks, etc., since all those will be done by the transport.
> Unfortunately, this is false.
> 
> But this is the first time I've heard anyone saying you can even get rid
> of app-level queues by using a reliable transport.
> 
> > 
> SCTP never intends to support doing EVERYTHING for SIGTRAN. Instead, it
> just provides the "SIG" layer for SIGTRAN as stated in RFC2719. ULPs
> will need to do their part as well.
> 
> -Qiaobing
> 
> > 

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 05 Nov 2001 17:35:05 -0600

Brian,

-Qiaobing