Subject: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network equipment failure scenario

From: "Schantz, John" <[email-masked]>
Date: Thu, 27 Jun 2002 15:14:47 -0500

We are pretty mush finished with our development of M3UA and we are in
the process of trying to find ways to improve or fix some of the holes.
I have a proposal below for fixing a network equipment failure scenario
that I would like to get added to the IG.  It will also clarify the use
of ASP IDs.  I also think the proposal could be extended to perform a
"change-over" if we are willing to get some transmission info from SCTP,
but that can wait.

The failure of IP network equipment (HUBs, routers, etc.) has a major
impact on the performance of M3UA (and other UAs).  If the failure
occurs at an adjacent network interface, the connecting ASP or SGP can
detect this failure immediately (association failure) and can move the
traffic to another association.  The problem is that the non-adjacent
ASP or SGP can not detect the failure immediately since it must wait for
missed SCTP SACKs or Heartbeat Acks.  Section 4.3.4.6 states that if a
Heartbeat Ack (or any other message) is not received within 2*T(beat),
then the remote M3UA is considered unavailable.  There are a couple of
problems with this:  1) the non-adjacent side is still sending messages
during this 2*T(beat) time period, this period could be several seconds
during which message buffers could overflow and/or messages could be
lost, and 2) only the ASP/SGP connected to the failure is unavailable
while the other ASP/SGPs at the node could still be available and should
have the traffic moved over to them as soon as possible.. 

Below is a proposal that should solve most of this problem and limit
(eliminate?) the possible message loss.    The M3UA RFC allows for the
assignment of IDs to ASPs but they are only transported in the ASP UP
message and there is no discussion of what they are used for.  Some
changes and clarifications are needed.  First, for the ASP IDs to be
useful, they must be unique per association from an AS.  Second, the ASP
Down and ASP Down Ack must also have the ASP ID parameter as an optional
parameter (adding it to the ASP Inactive [Ack] should also be
considered).  Third, an ASP/SGP that still has an available association
to the remote node should be able to send (forward) an ASP Down/ASP Down
Ack that contains the ASP ID of the unavailable ASP (association).  Each
of these items are addressed below.

ASP ID uniqueness

It is fairly easy to show that assigning an ASP ID to the process that
handles all associations for a network interface is of very little use.
Suppose that you have an ASP that has an association to 2 (or more) SGPs
at an SG.  Suppose the ASP only wants to stop traffic on one of the
associations.  It would send an ASP Inactive or ASP Down.  There is no
ASP IDs present but there is an implied ASP ID for the ASP.  Will the SG
assume that all associations to this ASP are going out of service or
should it only assume that the association that carried the message is
going out of service?  It is the latter case that is assumed.  Hence,
the generic ASP Up, ASP Inactive, ASP Down, etc. only apply to the
association (ASP and ASP ID) that the messages are received on.
Therefore, the implicit and explicit ASP IDs must be unique per
association between the AS and SG.

ASP ID Optional Parameter in ASP Down and ASP Down Ack

The need for the ASP ID to be an optional parameter in the ASP Down and
ASP Down Ack will become clear in the next section.  It will allow the
ASP/SGP to be explicit about which association is being impacted.
Making the ASP ID optional in the ASP Inactive and ASP Inactive Ack
should also be considered.

Forwarding of Locally Detected Association Failure.

When a node detects an association failure, its M3UA should notify the
affected adjacent node of the failure via an available association to
the adjacent node.  When the AS/SG detects the association failure, it
should send an ASP Down [Ack] with the "failed" ASP ID (association) on
an association that is still available to the adjacent SG/AS.  When the
M3UA at the adjacent SG/AS receives the ASP Down [Ack] with an ASP ID
that is different from the known ASP ID for the association, it should
handle message as though the ASP Down [Ack] was received by the SGP/ASP
(association) corresponding with the ASP ID.  This will cause the SG/AS
transition the ASP/association to the DOWN state and to move the traffic
away from the failed ASP/association (that it has not yet detected).

---

From: "Tolga Asveren" <[email-masked]>
Date: Thu, 27 Jun 2002 16:48:14 -0400

John,

> We are pretty mush finished with our development of M3UA and we are in the
> process of trying to find ways to improve or fix some of the
> holes.  I have
> a proposal below for fixing a network equipment failure scenario that I
> would like to get added to the IG.  It will also clarify the use
> of ASP IDs.
> I also think the proposal could be extended to perform a
> "change-over" if we
> are willing to get some transmission info from SCTP, but that can wait.
> The failure of IP network equipment (HUBs, routers, etc.) has a
> major impact
> on the performance of M3UA (and other UAs).  If the failure occurs at an
> adjacent network interface, the connecting ASP or SGP can detect this
> failure immediately (association failure) and can move the traffic to
> another association.  The problem is that the non-adjacent ASP or SGP can
> not detect the failure immediately since it must wait for missed
> SCTP SACKs
> or Heartbeat Acks.  Section 4.3.4.6 states that if a Heartbeat Ack (or any
> other message) is not received within 2*T(beat), then the remote M3UA is
> considered unavailable.  There are a couple of problems with this:  1) the
> non-adjacent side is still sending messages during this 2*T(beat) time
> period, this period could be several seconds during which message buffers
> could overflow and/or messages could be lost, and 2) only the ASP/SGP
> connected to the failure is unavailable while the other ASP/SGPs
> at the node
> could still be available and should have the traffic moved over to them as
> soon as possible..
> message and there is no discussion of what they are used for.
> Some changes
> and clarifications are needed.  First, for the ASP IDs to be useful, they
> must be unique per association from an AS.  Second, the ASP Down and ASP
> Down Ack must also have the ASP ID parameter as an optional parameter
> (adding it to the ASP Inactive [Ack] should also be considered).
> Third, an
> ASP/SGP that still has an available association to the remote
> node should be
> able to send (forward) an ASP Down/ASP Down Ack that contains the
> ASP ID of
> the unavailable ASP (association).  Each of these items are
> addressed below.
> ASP ID uniqueness
> Suppose that you have an ASP that has an association to 2 (or
> more) SGPs at
> an SG.  Suppose the ASP only wants to stop traffic on one of the
> associations.  It would send an ASP Inactive or ASP Down.  There is no ASP
> IDs present but there is an implied ASP ID for the ASP.  Will the
> SG assume
> that all associations to this ASP are going out of service or
> should it only
> assume that the association that carried the message is going out of
> service?  It is the latter case that is assumed.  Hence, the
> generic ASP Up,
> ASP Inactive, ASP Down, etc. only apply to the association (ASP
> and ASP ID)
> that the messages are received on.  Therefore, the implicit and
> explicit ASP
> IDs must be unique per association between the AS and SG.
> The need for the ASP ID to be an optional parameter in the ASP
> Down and ASP
> Down Ack will become clear in the next section.  It will allow the ASP/SGP
> to be explicit about which association is being impacted.  Making
> the ASP ID
> optional in the ASP Inactive and ASP Inactive Ack should also be
> considered.
> affected adjacent node of the failure via an available association to the
> adjacent node.  When the AS/SG detects the association failure, it should
> send an ASP Down [Ack] with the "failed" ASP ID (association) on an
> association that is still available to the adjacent SG/AS.  When
[TOLGA] If there is still a healthy association between those two nodes, why
initially not to have a multihomed SCTP association, which also includes
this interface? If number of interfaces is high, IMO this would increase
detection time for host failures, but is this the reason, why you didn't
prefer to use multihoming?
> the M3UA at
> the adjacent SG/AS receives the ASP Down [Ack] with an ASP ID that is
> different from the known ASP ID for the association, it should handle
> message as though the ASP Down [Ack] was received by the SGP/ASP

_______________________________________________
Sigtran mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/sigtran

---

From: [email-masked]
Date: Thu, 27 Jun 2002 17:06:38 -0400

John,

The idea behind multi-homing is that there should be no single point of 
failure common to the two paths. If this rule is followed when 
provisioning the network, then a multi-homed association will survive the 
failure of a single hub, router, etc.

Barry Nagelberg

"Schantz, John" <[email-masked]>
Sent by: [email-masked]
06/27/2002 04:42 PM

        To:     "Tolga Asveren" <[email-masked]>, <[email-masked]>
        cc: 
        Subject:        RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network equipment 
failure scenario

Tolga, if the HUB or router that the NIC/Ethernet is connected to fails,
it will take ALL association with it.  SCTP multi-homing will not help
you!

-----Original Message-----
From: Tolga Asveren [mailto:[email-masked]] 
Sent: Thursday, June 27, 2002 3:48 PM
To: [email-masked]
Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network
equipment failure scenario

John,

> the process of trying to find ways to improve or fix some of the 
> holes.  I have a proposal below for fixing a network equipment failure

> scenario that I would like to get added to the IG.  It will also 
> clarify the use of ASP IDs.
ASP
> affected adjacent node of the failure via an available association to
the
> association that is still available to the adjacent SG/AS.  When
[TOLGA] If there is still a healthy association between those two nodes,
why initially not to have a multihomed SCTP association, which also
includes this interface? If number of interfaces is high, IMO this would
increase detection time for host failures, but is this the reason, why
you didn't prefer to use multihoming?
> (association) corresponding with the ASP ID.  This will cause the 
> SG/AS transition the ASP/association to the DOWN state and to move the

> traffic away from the failed ASP/association (that it has not yet 
> detected).

---

From: "Schantz, John" <[email-masked]>
Date: Thu, 27 Jun 2002 17:32:13 -0500

Tolga, we don't support it and the effort to support it would not be
worth it.  I'm from the SS7 side of the world so I believe in true
multi-homing via SS7 like methods.

-----Original Message-----
From: Tolga Asveren [mailto:[email-masked]] 
Sent: Thursday, June 27, 2002 4:13 PM
To: [email-masked]
Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network

John,

> From: Schantz, John [mailto:[email-masked]]
> Sent: Thursday, June 27, 2002 4:42 PM
> To: Tolga Asveren; [email-masked]
> Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a 
> network equipment failure scenario
>
>
> Tolga, if the HUB or router that the NIC/Ethernet is connected to 
> fails, it will take ALL association with it.  SCTP multi-homing will 
> not help you!
[TOLGA] But then how can the other association, from which you propose
to send ASPDOWN[ACK] survives? Probably it uses another NIC/Ethernet,
but then why not make this interface also part of the multihomed
association?
> Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a 
> network equipment failure scenario
>
>
> John,
>
> > clarify the use of ASP IDs. I also think the proposal could be 
> > extended to perform a "change-over" if we
> > major impact on the performance of M3UA (and other UAs).  If the 
> > failure occurs at
> an
> > adjacent network interface, the connecting ASP or SGP can detect 
> > this failure immediately (association failure) and can move the 
> > traffic to another association.  The problem is that the 
> > non-adjacent ASP or SGP
> can
> > SCTP SACKs or Heartbeat Acks.  Section 4.3.4.6 states that if a 
> > Heartbeat Ack (or
> any
> > other message) is not received within 2*T(beat), then the remote 
> > M3UA
> is
> > considered unavailable.  There are a couple of problems with this:  
> > 1)
> the
> > non-adjacent side is still sending messages during this 2*T(beat) 
> > time period, this period could be several seconds during which 
> > message
> buffers
> > could overflow and/or messages could be lost, and 2) only the 
> > ASP/SGP connected to the failure is unavailable while the other 
> > ASP/SGPs at the node could still be available and should have the 
> > traffic moved over to
> them as
> > soon as possible..
> > assignment of IDs to ASPs but they are only transported in the ASP 
> > UP message and there is no discussion of what they are used for. 
> > Some changes and clarifications are needed.  First, for the ASP IDs 
> > to be useful,
> they
> > Down Ack must also have the ASP ID parameter as an optional 
> > considered). Third, an ASP/SGP that still has an available 
> > association to the remote node should be
> > IDs present but there is an implied ASP ID for the ASP.  Will the SG

> > the ASP ID optional in the ASP Inactive and ASP Inactive Ack should 
> > also be considered.
> [TOLGA] If there is still a healthy association between those two 
> nodes, why initially not to have a multihomed SCTP association, which 
> also includes this interface? If number of interfaces is high, IMO 
> this would increase detection time for host failures, but is this the 
> reason, why you didn't prefer to use multihoming?
> > the M3UA at
> > the adjacent SG/AS receives the ASP Down [Ack] with an ASP ID that 
> > is different from the known ASP ID for the association, it should 
> [email-masked] https://www1.ietf.org/mailman/listinfo/sigtran

---

From: "Schantz, John" <[email-masked]>
Date: Fri, 28 Jun 2002 10:55:36 -0500

Hi Javier.

You should not assume that multi-homing is always there.  Not all
architectures support it and especially if the NIC only has a single
Ethernet interface and the NICs are handled independently and are only
linked at the M3UA level.  Also NTFY is a very weak message and you can
probably get by by ignoring them.  Also aren't they only sent by SGs and
how would indicate the real problem?

We have done a lot of testing and found that if you pull an Ethernet
cable (equal to a cable cut, hub or router failure, etc.) the connecting
ASP/SGP detects the failure immediately and moves its outbound traffic
to other associations.  The other side of the association is left
hanging and does not detect the failure very quickly.

I felt that it would be a simple matter to inform the remote side of the
failure since that would be quicker than waiting for it to detect the
failure.  If you guys are not interested in solving issues like this, I
will not waste my time on them.

John

-----Original Message-----
From: Javier Pastor-Balbas (ECE)
[mailto:[email-masked]] 
Sent: Friday, June 28, 2002 5:57 AM
To: Schantz, John; Tolga Asveren; [email-masked]
Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network
equipment failure scenario

Hi John, 

You are talking about a hub/router failure:

Router failure: ideally if you have a router failure you will have
alternatives paths if the network configuration it's well set up. This
means that the routing protocols will update the routes and just a small
delay will happen. But everything keeps working. Multihoming is not
crucial here.

Hub failure. For the recommended configuration with multihoming, this
should not be a major problem. Multihoming allows you to have multi-hub
configuration. And then, if one hub fails you will still have someothers
to use with different local NICs.

What it seems to be clear to me is that multihoming has its advantages.
You are removing one of the main reliability features included for the
transport of SS7oIP if you're not using it.

Regarding your text, I don't understand very well your proposal. ASPID
is just used by the NTFY message and it is sent with ASPUP to inform the
SG of its value.

More comments below....

> Sent: Friday, June 28, 2002 12:32 AM

> > > We are pretty mush finished with our development of M3UA
> and we are
> > > in
> >
> > > must be unique per association from an AS.  Second, the
> ASP Down and
> > ASP
> > > associations.  It would send an ASP Inactive or ASP Down. 
>  There is
> > > no
> > ASP
> > > service?  It is the latter case that is assumed.

I would say here that is the ASP who sent the message the one who moves
to DOWN (out of service). But assocation could be still used by other
ASes

> > > Hence, the
> > > and ASP ID) that the messages are received on.  Therefore, the 
> > > implicit and explicit ASP

I would say ASPID is unique per ASP and AS

> > > ASP ID Optional Parameter in ASP Down and ASP Down Ack The need 
> > > for the ASP ID to be an optional parameter in the ASP Down and ASP
> > > to be explicit about which association is being impacted.

I don't understand the real need for this...

> > > Making
> > > adjacent node.

If you just have one association and it is broken....

It seems to me that you have two association between an SGP and an ASP.
Is that true? And you'd like to inform using the second assoc that the
first assoc failed, right?

I'm afraid this is not the standard use for ASPID. ASPID should identify
an ASP.

Another possibility is that you consider your ASP as two ASPs and set an
ASPID for each one. In that case you could sent NOTIFY(ASPID) with the
ASPID that is failing through the working association.

Cheers // Javier.

> > >When the AS/SG detects the association failure, it
> > should

---

From: [email-masked]
Date: Fri, 28 Jun 2002 17:14:49 -0400

John,

You should probably take a hard look at the SCTP parameter values that you 
are using. After pulling the primary IP cable of a multi-homed association 
that passes through a LAN, we are able to switch 16 links over to the 
secondary IP path in less than 1 second.

Barry Nagelberg

06/28/2002 02:32 PM

        To:     "Javier Pastor-Balbas (ECE)" <[email-masked]>, 
"Tolga Asveren" <[email-masked]>, <[email-masked]>
        cc: 
        Subject:        RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network equipment 
failure scenario

Javier,

I think you are giving multi-homing more credit than it is due.  The
scenario happens whether multi-homing is implemented or not.  And it is
not just my problem, it's a general problem.  The issue is that the AS
or SG that is not next to the failure can not detect it via M3UA or SCTP
until several seconds (and lots of traffic) has passed.  Why not let the
side that can immediately detect the problem inform the side that can't
so the other associations or multi-home IP addresses can be used to go
around the problem.  Test it in your lab, connect 2 systems via a LAN
and pull the Ethernet Cable from one of the interfaces on one system.
How long did it take the other system to detect the problem and move the
traffic?

John

-----Original Message-----
From: Javier Pastor-Balbas (ECE)
[mailto:[email-masked]] 
Sent: Friday, June 28, 2002 11:30 AM
To: Schantz, John; Javier Pastor-Balbas (ECE); Tolga Asveren;
[email-masked]
Subject: RE: [Sigtran] M3UA: IG: ASP IDs & a proposal to solve a network
equipment failure scenario

Hi John, 

See below...

> Sent: Friday, June 28, 2002 5:56 PM
> To: Javier Pastor-Balbas (ECE); Tolga Asveren; [email-masked]

> linked at the M3UA level.  Also NTFY is a very weak message and you 
> can probably get by by ignoring them.  Also aren't they only sent
> by SGs and
Yes.

> cable (equal to a cable cut, hub or router failure, etc.) the 
> connecting ASP/SGP detects the failure immediately and moves its 
> outbound traffic to other associations.  The other side of the 
> association is left hanging and does not detect the failure very 
> quickly.
> 

John, it is not a matter of not being interested in it. In fact, I am.
It is a matter of not having enough information.  :-/

For the normal case (IMHO) anyone will use a multihoming association
with hardware redundancy to get the desired behaviour. But I'm lacking
the knowledge of what are you using in your backbone and how do you use
SCTP and M3UA. At least I didn't undertand it enough....

Basic questions for me are:
- How is your networks set up?
- Do you have two associations between two nodes (SGP/ASP)?

(just getting more info to find out a solution to your problems)

Cheers // Javier.

> > > > are willing to get some transmission info from SCTP, 
> but that can
> > > wait.
> > > > (eliminate?) the possible message loss.    The M3UA RFC 
> allows for
> > the
> > > > It is fairly easy to show that assigning an ASP ID to 
> the process
> > that
> > > > handles all associations for a network interface is of 
> very little
> > > use.
> > > > Down Ack will become clear in the next section.  It 
> will allow the
> > > ASP/SGP
> > > > When a node detects an association failure, its M3UA 
> should notify
> > the
> > > > affected adjacent node of the failure via an available 
> association
> > to
> > > the
> > > > adjacent node.
> 
> > > > SG/AS transition the ASP/association to the DOWN state 
> and to move
> > the
> > >