Subject: RE: [Megaco] MGC-MG dialogue failure detection

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 11:17:24 -0400

Hi Tom and list,

         Yes, you have a perfect wording, this is exactly the intention:
"that no handover to secondary occurs without either the permission
of the primary MGC or determination that it is unreachable."
And it makes it even more sense to add these properties to the inactivity
timer package since it is the one that helps to detect "unreachable" case.

Does anybody else have comments and/or agree/disagree with it?
I'm sure that everybody kind of sick of this thread :) so I'd like to
bring it to some kind of consensus/agreement.

Thanks.
         Aleks

At 11:48 PM 7/15/2002, Tom-PT Taylor wrote:

>Speaking as a referee on this thread (which deals with a recurring theme), I
>believe your package approach would be legitimate.  It clearly involves new
>behaviour, but no new protocol elements, and it is not a threat to
>interoperability: MGs not supporting the package would have well-defined and
>appropriate behaviour and the network would survive, though perhaps with
>additional lost calls.  If I were designing the package, I would postulate
>the following requirement: that no handover to secondary occurs without
>either the permission of the primary MGC or determination that it is
>unreachable.
>
>Whether you can get other vendors to adopt the package is another question.
>
>
>-----Original Message-----
>From: Aleksandr Ryabin [mailto:[email-masked]]
>Sent: Friday, July 12, 2002 1:01 PM
>To: Christian Groves; [email-masked]; Boyle, Kevin [NCRTP:3R90:EXCH];
>Carl Rutter
>Subject: Re: [Megaco] MGC-MG dialogue failure detection
>
>
>Hi Christian,
>
>          Now I'm confused, just before your words were:
> > > >If the main aim of the addition is to stop calls being dropped then
>fine.
>
>And so I agreed, but now you are saying non-issue. I guess I was
>reading it wrong in the first place, right? Or not?
>It seem that it is turning into a word game :)
>
>So to give it another spin, lets follow the other thread direction and set
>the
>requirements (again, some are given, some optional, or even questionable):
>
>1. on primary MGC unexpected failure provide mechanism to minimize
>service outage due to failure detection delay
>2. maintain security requirements mandated by protocol
>3. maintain consistently the rule of "one primary/active MGC"
>4. provide backward compatibility
>5. provide unified solution for all versions of protocol
>6. provide protocol based solution for MGCs with different IP interfaces
>
>So far the proposals are:
>
>1. Christian G.: use outside protocol solution (e.g. "IP/port hot swap") to
>mask primary MGC failure thus making it transparent to MG;
>
>2. Aleks R: add "establishSecondaryLinks" and "acceptSecondaryHandoff"
>ROOT properties to inactivity package to direct MG to establish secondary
>control connections on startup and to allow handoff to be initiated from
>secondary MGC .
>
>Analysis of solutions:
>
>1. "hot swap":
>#1 - #5 - conforms, since MGC failover is transparent to MG;
>#6 - does not conform, solution is out of scope of the protocol,
>requires hardware support for MGC capable of hot swap of IP interface.
>
>2. "secondary handoff":
>#1 - conforms, no delay;
>#2 - conforms, physical connection is established by MG on startup
>depending on "establishSecondaryLinks" property setting;
>#3 - conforms, there is only one primary/active MGC at any point of time,
>if case of "incidental" handoff initiation from secondary MGC, on detection
>primary MGC can regain control by initiating handoff;
>#4 - conforms, MGs supporting the package which has new additional
>ROOT properties can be configured to use the mechanism, the other
>MGs can follow regular procedures with no changes, or functionality loss;
>#5 - conforms, new properties defined in a package which can be
>implemented for any version of the protocol;
>#6. - conforms, solution realized as package in scope of the protocol,
>MGCs can have different IP interface.
>
>Analysis summary:
>
>1. "hot swap" - conforms all requirements, but #6.
>2. "secondary handoff" conforms all requirements.
>
>Comments please?
>
>Thanks
>          Aleks
>
>I addressed (or at least that what I think) all issues with the solution
>
>At 11:12 PM 7/11/2002, Christian Groves wrote:
> >G'Day Aleks,
> >
> >No, I still believe this is a non issue for the reasons and questions that
> >have
> >been
> >raised before.
> >
> >Christian
> >
> >Aleksandr Ryabin wrote:
> > >
> > > Hi Christian,
> > >
> > >          Yes it is to stop calls being dropped as well as calls already
> > > in progress to be processed.
> > >
> > > Does it mean we identified the problem to be an issue in the protocol?
> > >
> > > Thanks
> > >          Aleks
> > >
> > > At 02:35 AM 7/11/2002, Christian Groves wrote:
> > > >G'Day Aleks,
> > > >
> > > >Please see my responses below.
> > > >
> > > >Regards, Christian
> > > >
> > > >Aleksandr Ryabin wrote:
> > > > >
> > > > > At 03:37 PM 7/10/2002, Kevin Boyle wrote:
> > > > > >If you are already talking about building in state synchronization,
> > > > then why
> > > > > >don't you build it hot-swap capability too?  Then this isn't a
> > problem.
> > > > >
> > > > > State synchronization and hot-swap are two different animals.
> > > > > The first can depend only on software support, while the second
> > > > > heavily depends on hardware support.
> > > > >
> > > > > I'd like to stress that "why not this or that" is not a question
>here.
> > > > > We are not trying to solve MGC implementation issue, but rather
> > > > > MEGACO protocol limitation issue.
> > > >
> > > >[CHG] Its not as easy as that. For this issue implementation and
> > protocol are
> > > >linked. Some are saying there is no problem. You (I assume) because of
>an
> > > >implementation
> > > >choice are saying that it is a protocol limitation issue. Before we put
> > > >something into the main protocol we have to understand why it is
> > needed and in
> > > >what situations. Carl has given a good explanation of his MGC/MG system
> > > >and what
> > > >call load he is managing and the fact that he has gone for a "hot swap"
> > > >system.
> > > >If the main aim of the addition is to stop calls being dropped then
> > fine. But
> > > >if the fact that call control redirection means we can't get new calls
> > in that
> > > >time then the problem is a non-issue.
> > > >
> > > >...snip..
>
>
>_______________________________________________
>Megaco mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/megaco

_______________________________________________
Megaco mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/megaco

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 12:56:37 -0400

I am not sure how this helps.  If the primary is dead (but not yet 
detected by the MG) and the MG receives some sort of indication from the 
secondary that it wants to take over, the suggested language would still 
require the MG to determing that the primary is "unreachable".  This 
would involve sending a message and waiting for it to timeout (or 
possibly getting an ICMP error).  If it must wait for a timeout then we 
have a delay very similar to that currently required in the Inactivity 
Timer package.  The only difference would be to stimulate something 
similar to the it/ito event immediately rather than waiting the normal 
mit time.  But is that really enough faster to be worth the effort? 
 "unreachable" would still be determined only after T-MAX (see D.1) or 
about 6 sec.  In a high rate MG, I would assume many would set the mit 
value to 1 sec or so and so we are only talking about 6 vs 7 sec or some 
similar small difference.  The only exception is where the connection to 
the primary has failed in such a way the the network will give a more 
immediate ICMP error (host unreachable) so that one does not have to 
wait for retransmisison timeout, but in practice I believe that this is 
not often the case.

-- 

------------------------------------------------------------
Terry L Anderson              mailto:[email-masked]
Tel:732.949.5628   Fax:732.949.7016
Lucent Technologies/INS/Voice Over IP Access Networks
Rm 2G-219A, 101 Crawfords Corner Rd, Holmdel, NJ 07733-3030
http://its.lucent.com/~tla (Lucent internal) http://www.gti.net/tla

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 12:56:37 -0400

--

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 13:27:51 -0400

Oh.  I didn't catch that.  So that still does not solve the issue of 
whether the secondary's detection of primary's failure is correct.  We 
still have the issue that the primary may still think it is in control 
and send transactions.  I assume that primary should not send 
acceptSecondaryHandoff unless it knows that it and the secondary can 
reliably agree about failure (through some means that I cannot imagine 
unless they are colocated) or is willing to tolerate false handoffs. 
 Does the MG reject transactions from the primary (if they arrive) after 
such a handoff?

Aleksandr Ryabin wrote:

> "Permission of the primary" happens in advance by setting 
> acceptSecondaryHandoff
> property from primary MGC on its startup, that is upon its initial 
> registration with MG.
> Thus when later handoff is initiated by secondary MGC it is ether 
> allowed, or not
> depending on current value of acceptSecondaryHandoff.
>
>         Aleks
>
> At 12:56 PM 7/16/2002, Terry L Anderson wrote:
>
>> detected by the MG) and the MG receives some sort of indication from 
>> the secondary that it wants to take over, the suggested language 
>> would still require the MG to determing that the primary is 
>> "unreachable".  This would involve sending a message and waiting for 
>> it to timeout (or possibly getting an ICMP error).  If it must wait 
>> for a timeout then we have a delay very similar to that currently 
>> required in the Inactivity Timer package.  The only difference would 
>> be to stimulate something similar to the it/ito event immediately 
>> rather than waiting the normal mit time.  But is that really enough 
>> faster to be worth the effort? "unreachable" would still be 
>> determined only after T-MAX (see D.1) or about 6 sec.  In a high rate 
>> MG, I would assume many would set the mit value to 1 sec or so and so 
>> we are only talking about 6 vs 7 sec or some similar small 
>> difference.  The only exception is where the connection to the 
>> primary has failed in such a way the the network will give a more 
>> wait for retransmisison timeout, but in practice I believe that this 
>> is not often the case.
>>
>> Aleksandr Ryabin wrote:
>>
>>> Hi Tom and list,
>>>

--

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 13:27:51 -0400

--

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 13:06:53 -0400

"Permission of the primary" happens in advance by setting 
acceptSecondaryHandoff
property from primary MGC on its startup, that is upon its initial 
registration with MG.
Thus when later handoff is initiated by secondary MGC it is ether allowed, 
or not
depending on current value of acceptSecondaryHandoff.

         Aleks

At 12:56 PM 7/16/2002, Terry L Anderson wrote:
>require the MG to determing that the primary is "unreachable".  This would 
>involve sending a message and waiting for it to timeout (or possibly 
>getting an ICMP error).  If it must wait for a timeout then we have a 
>delay very similar to that currently required in the Inactivity Timer 
>package.  The only difference would be to stimulate something similar to 
>the it/ito event immediately rather than waiting the normal mit time.  But 
>is that really enough faster to be worth the effort? "unreachable" would 
>still be determined only after T-MAX (see D.1) or about 6 sec.  In a high 
>rate MG, I would assume many would set the mit value to 1 sec or so and so 
>difference.  The only exception is where the connection to the primary has 
>failed in such a way the the network will give a more immediate ICMP error 
>(host unreachable) so that one does not have to wait for retransmisison 
>timeout, but in practice I believe that this is not often the case.
>
>Aleksandr Ryabin wrote:
>
>>Hi Tom and list,
>>

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 13:06:53 -0400

Aleks

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 13:59:17 -0400

As previously was noted on this thread there is a possibility of "unexpected"
(for primary MGC) handoff from secondary MGC. In this case primary MGC
will loose control (that is MG will reject messages with error 505).

Ideally this would not happen since if the mechanism is used (e.g primary
MGC explicitly sets acceptSecondaryHandoff to yes) then MGCs better
have these "some means" to detect each other run status, otherwise
secondary MGC would not know when to initiate handoff in the first place.
But even if in this case "incidental" handoff is initiated by secondary MGC,
primary MGC can regain control on the very next send since its very next
message to MG will fail with error 505, thus primary now has turned into
secondary and thus it can initiate handoff to regain control (if necessary).
One can say this can then potentially turn into a race. The answer is in
word "incidental": that is normally MGCs will correctly identify each other
run status and if "incidental" handoff detected MGCs will take steps
to report it via some management mechanism and correct as needed.
All this can be documented in "procedures" of the package.

         Aleks

At 01:27 PM 7/16/2002, Terry L Anderson wrote:
>still have the issue that the primary may still think it is in control and 
>send transactions.  I assume that primary should not send 
>unless they are colocated) or is willing to tolerate false handoffs. Does 
>the MG reject transactions from the primary (if they arrive) after such a 
>handoff?
>
>Aleksandr Ryabin wrote:
>

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 13:59:17 -0400

Aleks

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 14:37:01 -0400

I am assuming that primary and secondary probe each other for liveness 
over the network (presumably more often than MG probes), and there is 
the case where MG can reach both MGCprim and MGCsec but due to some 
router failure MGCsec cannot reach MGCprim and so concludes that it is 
dead.  I believe that this case is possible but I suppose that it is not 
real common since a path between MGCprim and MGCsec must exist through 
MG's local router except when the paths are over half the max hop count 
limit (and if the hop count is that hight the network may be too poor of 
quality to be acceptible for VoIP anyway).

In this case, however, each MGC believes the other is dead and will yank 
the MG back and forth unless, as you suggest, they detect this rapid 
alternation state and take some other corrective action.  Of course each 
may be very busy auditing its "new MG" to determine the state of all its 
calls.

I just wonder how the MGCs monitor each other reliably significantly 
faster than the MGs can monitor.  Are you just counting on the number of 
monitors being much smaller (list of secondaries rather than all MGs) so 
that the polling frequency can be higher or are you primarily interested 
in the case where MGC are colocated and polling over LAN or even 
backplane connections?

Aleksandr Ryabin wrote:

>> whether the secondary's detection of primary's failure is correct.  
>> We still have the issue that the primary may still think it is in 
>> control and send transactions.  I assume that primary should not send 
>> reliably agree about failure (through some means that I cannot 
>> imagine unless they are colocated) or is willing to tolerate false 
>> handoffs. Does the MG reject transactions from the primary (if they 
>> arrive) after such a handoff?
>>
>> Aleksandr Ryabin wrote:
>>
>>>> detected by the MG) and the MG receives some sort of indication 
>>>> from the secondary that it wants to take over, the suggested 
>>>> language would still require the MG to determing that the primary 
>>>> is "unreachable".  This would involve sending a message and waiting 
>>>> for it to timeout (or possibly getting an ICMP error).  If it must 
>>>> wait for a timeout then we have a delay very similar to that 
>>>> currently required in the Inactivity Timer package.  The only 
>>>> difference would be to stimulate something similar to the it/ito 
>>>> event immediately rather than waiting the normal mit time.  But is 
>>>> that really enough faster to be worth the effort? "unreachable" 
>>>> would still be determined only after T-MAX (see D.1) or about 6 
>>>> sec.  In a high rate MG, I would assume many would set the mit 
>>>> value to 1 sec or so and so we are only talking about 6 vs 7 sec or 
>>>> some similar small difference.  The only exception is where the 
>>>> connection to the primary has failed in such a way the the network 
>>>> will give a more immediate ICMP error (host unreachable) so that 
>>>> one does not have to wait for retransmisison timeout, but in 
>>>> practice I believe that this is not often the case.
>>>>
>>>> Aleksandr Ryabin wrote:
>>>>
>>>>> Hi Tom and list,
>>>>>

--

---

From: Terry L Anderson <[email-masked]>
Date: Tue, 16 Jul 2002 14:37:01 -0400

--

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 15:56:14 -0400

Any "reliable" connection between MGC will do it.
By "reliable" I mean a connection with disconnection detection time less
then the system (e.g. soft switch) allowable service outage/delay value.

This "reliable" connection can be realized number of ways. It can use,
for example as you noted, backplane, or even a shared memory if both
MGC are realized with shared memory access. Or it can use some
reliable transport protocol (e.g SCTP, TCP, etc.) on isolated packet
based network which differ from MG-MGC control network.

The important thing that the means of realization of that "reliable" 
connection
is not limited by application (e.g. MEGACO in this case) protocol limitations
and/or by application transport protocol (e.g. UDP in this case) limitations
potentially imposed by MG (since MG, for example, can chose to implement
UDP only).

Obviously if that "reliable" connection does not offer failure detection time
decrease over the standard MEGACO failure detection time then it does not
make any sense to use this new proposed mechanism.

         Aleks

At 02:37 PM 7/16/2002, Terry L Anderson wrote:
>over the network (presumably more often than MG probes), and there is the 
>case where MG can reach both MGCprim and MGCsec but due to some router 
>failure MGCsec cannot reach MGCprim and so concludes that it is dead.  I 
>believe that this case is possible but I suppose that it is not real 
>common since a path between MGCprim and MGCsec must exist through MG's 
>local router except when the paths are over half the max hop count limit 
>(and if the hop count is that hight the network may be too poor of quality 
>to be acceptible for VoIP anyway).
>
>may be very busy auditing its "new MG" to determine the state of all its calls.
>
>in the case where MGC are colocated and polling over LAN or even backplane 
>connections?
>
>Aleksandr Ryabin wrote:
>
>>>>>the secondary that it wants to take over, the suggested language would 
>>>>>still require the MG to determing that the primary is 
>>>>>faster to be worth the effort? "unreachable" would still be determined 
>>>>>only after T-MAX (see D.1) or about 6 sec.  In a high rate MG, I would 
>>>>>assume many would set the mit value to 1 sec or so and so we are only 
>>>>>talking about 6 vs 7 sec or some similar small difference.  The only 
>>>>>exception is where the connection to the primary has failed in such a 
>>>>>way the the network will give a more immediate ICMP error (host 
>>>>>unreachable) so that one does not have to wait for retransmisison

---

From: Aleksandr Ryabin <[email-masked]>
Date: Tue, 16 Jul 2002 15:56:14 -0400

Aleks