Subject: RE: [Megaco] Support for redundant MG failover

From: Steve Cipolli <[email-masked]>
Date: Thu, 1 May 2003 17:14:54 -0400

My understanding of both MGCP and MEGACO is that neither has a mechanism to
distinguish multihomed interfaces from redundant MGCs/MGs.  In MGCP the list
of IP addresses returned for a given domain name may be a set of
(multihomed) interfaces, a set of redundant MGCs/MGs, or some combination of
both.  In MEGACO the provisioned list of MGCs in an MG has this exact same
property.  

I do see the fact that MEGACO MGCs have no mechanism to contact MGs on
multihomed interfaces as a significant deficiency.  I agree that mapping a
list IP addresses returned from a domain name to the set of interfaces
supported by a MEGACO association would be benefical, but the problem of how
to distinguish multihomed interfaces from redundant MGs/MGCs still remains
unclear to me (for both MEGACO and MGCP).

--Stephen

> -----Original Message-----
> From: Flemming Andreasen [mailto:[email-masked]]
> Sent: Thursday, April 24, 2003 11:04 AM
> To: Adrian Soncodi
> Cc: Megaco
> Subject: Re: [Megaco] Support for redundant MG failover
> 
> 
> 
> 
> Adrian Soncodi wrote:
> 
> > Hello, all,
> >
> > I was wondering what is your oppinion on the following:
> >
> > Another topic that is not addressed in the MEGACO spec is 
> the case of
> > a multi-homed MG, i.e. an MG where:
> > - the primary controller has 2 physical interfaces A1 and A2,
> > potentially on 2 different subnets, working in load-sharing mode
> > (rather than active-standby)
> > - the secondary MG controller can then be either transparrent (it
> > takes over A1 and A2 on switchover) or non-transparent (has 2 more
> > physical addresses, B1 and B2).
> >
> > One may argue that multi-homing is not necessary, for 
> example because
> > you can always assign a single "logical" IP address A that is
> > reachable over both interfaces A1 and A2. Yes, but this schema
> > requires some dynamic IP routing capability in the backbone (e.g. to
> > figure out when A1 is down and packets for A should be 
> routed to A2),
> > which has a number of disadvantages:
> > - Depending on the routing protocol, the latency for dynamic updates
> > of the addresses can be very large (180 seconds in case of RIP),
> > unacceptable for telephony.
> > - Depending on the network topology, interface failures may 
> not become
> > visible in the backbone. For example, if there is a LAN 
> switch between
> > the MG and the first router, then the router does not see when the
> > interface A1 fails (unless the dynamic routing domain is extended to
> > include the MG, which is not always the case).
> > - Some service providers (especially the smaller ones) do not run a
> > dynamic IP routing protocol in their (whole) backbone, or 
> they may not
> > even have a real backbone, for example they may use leased lines, IP
> > over ATM/FR PVCs with no sophisticated routing capabilities.
> >
> > On the other hand, making the physical addresses A1 and A2 visible
> > allows for redundant signaling paths to the MG using only static IP
> > routing in the network, which in certain situations is desirable.
> >
> > The MGC-MG redundancy schema gets even more complicated if you
> > consider that the MGC can be multi-homed too. Effectively, 
> it can have
> > the same A1, A2, B1, B2 schema as the MG. In this case, 
> what is really
> > useful is an SCTP association (MGC-A1, MGC-A2, MG-A1, 
> MG-A2), because
> > this takes care of all the combinations. But if the MG supports only
> > UDP, it is not clear from the standard how to emulate this SCTP
> > behavior. For example:
> > - Some MGs do a DNS query for the MGC name, get 2 IP addresses in
> > response, but they just ignore the second IP address, thus ignoring
> > that the MGC is multi-homed.
> > - Other MGs consider the IP addresses as being 2 different MGCs,
> > rather than one multi-homed MG. The problem is that they 
> only talk to
> > one MGC at a time, and then they need a "handover" to 
> switch. So while
> > talking on A1-A1, the MGC cannot for example audit on A2-A2, because
> > this is immediately considered a failover (causing for example the
> > pending notifications to be lost). The MGCP spec, for example,
> > explicitly states that an MG should not change its 
> controlling entity
> > when being audited from another IP address (and it doesn't 
> say whether
> > this is a different MGC or the same multi-homed MGC).
> >
> > I think what needs to be clarified here is:
> > - Whether an MG/MGC can be multi-homed and how to handle 
> its multiple
> > IP addresses.
> > - Whether an MG/MGC with active-standby controllers, transparent or
> > non-transparent switchover, are considered one MG/MGC or 2 MG/MGCs
> > - Whether a domain name for an MG/MGC which yields more than one IP
> > address through DNS, is associated with one controller, or with an
> > active-standby pair, or possibly even with more 
> active-standby pairs.
> >
> 
> I agree - these are rather big holes in the current specs IMO.
> 
> 
> >
> > The case with non-transparent MG controller switchover, as Flemming
> > mentioned, poses additional problems. For example, if the active MG
> > controller loses connectivity over A1 and A2, it may not 
> even notice,
> > because not all MGs do heartbeats. So if the MGC determines it can
> > actually talk to B1 and B2, it cannot force an MG controller
> > switchover, or at least the standard does not define the 
> procedure for
> > this. I would note that, in practice, however, most of the MGs and
> > MGCs have transparent switchover (there is no B1, B2, they reuse A1,
> > A2 ...).
> >
> > The MGCP spec is a little more explicit on how the IP addresses are
> > handled. So I don't understand why this topic is outside 
> the scope of
> > MEGACO.
> >
> 
> Right. I have always thought it was unfortunate, that Megaco did not
> specify anything about use of multiple IP addresses and/or interfaces
> (apart from the static provisioning of backup MGCs) as this limits the
> robustness of the protocol when using UDP/TCP. I was hoping 
> that Annex F
> would rectifiy some of that, but it doesn't seem like it.
> 
> >
> > I raised these issues because, from my experience, every 
> single vendor
> > interprets them differently, and MGC-MG redundancy is a delicate
> > subject. Any suggestions?
> 
> 1. Continued discussion on the mailing list (preferably with 
> more input
> from some of the traditional Megaco gurus).
> 2. Write a contribution for SG16 addressing this.
> 3. Forget about TCP and UDP and just use SCTP.
> 
> -- Flemming
> 
> 
> _______________________________________________
> Megaco mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/megaco
> 
_______________________________________________
Megaco mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/megaco

---

From: Steve Cipolli <[email-masked]>
Date: Tue, 13 May 2003 18:37:52 -0400

The topics I've been able to gleen from the email exchanges are:
1. How to distinguish redundant entities from multiple interfaces
(multihoming) on a single entity.

2. Issues regarding item #1 with respect to SCTP, which one would expect to
use with multiple IP addresses (to a single entity) for link redundancy.

3. Issues regarding item #1 with respect to UDP, which would allow similar
advantages to that of SCTP at the expense of some Megaco transport
complexity.

4. MGC-to-MG communication via a DNS domain name that yields multiple IP
addresses is not specified in Megaco.

I'm sure there were more topics...

It's interesting to note regarding items #1 & #4 that SIP, MGCP, and soon
H.323 (Annex O) have dealt with this problem.

SIP and soon H.323 use DNS SRV records to specify redundant entities, and
multiple IP addresses in an A or AAAA record to define multiple interfaces
on the same entity.  

MGCP uses multiple IP addresses in an A or AAAA record to indicate multiple
interfaces on the same "logical" entity.  Redundancy/fail-over is handled by
two different schemes that are linked to the specifics of MGCP. 

It seems to me that this problem should be solved consistently across these
(and many other) protocols and Megaco.  The SIP/H.323 direction seems to
make sense, providing for both redundant and multihome support in a
consistent and general way.  

--Stephen

> From: Adrian Soncodi [mailto:[email-masked]]
> Sent: Tuesday, May 13, 2003 4:34 PM
> To: Tom Taylor
> Cc: Fernando Vallejo Lazaro; [email-masked]
> Tom Taylor wrote:
> > It seems to me that the outlines of a solution have been 
> sketched in 
> > Fernando's note.
> 
> 
> Tom,
> 
> I don't quite understand, can you elaborate on this statement? I was 
> trying to make the point that, for multi-homing, we could allow 2 or 
> more IP addresses per MG and/or MGC, which would make it simpler to 
> establish for example one SCTP association using all of them.
> 
> Initially, Fernando said that 1 MEGACO control association = 1 IP 
> address, and implied that 2 IP addresses would be viewed as 2 
> distinct 
> MGCs. But in this case:
> - explaining which MGC pairs with which other MGC as multi-homed 
> entities is more difficult,
> - plus, if an SCTP association uses more than 1 IP address (which it 
> should, in order to take advantage of multi-homing), then it 
> would mean 
> that 1 control association spans 2 MGCs, which seems to 
> contradict the 
> model. Handoff and failover wouldn't work in this case, 
> because MEGACO 
> does not control the selection of the IP addresses withing 
> the SCTP layer.
> 
> Therefore, what do you think about the number of IP addresses per 
> MG/MGC? If SCTP is allowed to use only one IP address per 
> entity, then 
> why would one ever use SCTP instead of TCP? My understanding 
> is SCTP was 
> supposed to be a solution for multi-homing.
> 
> Additionally (I don't know if you followed the whole 
> discussion in this 
> thread), a number of people have expressed interest in supporting 
> multi-homing over UDP too. This would also benefit from allowing 
> multiple IP addresses per MG/MGC, but a few clarification in the 
> procedures would be required. Would you be willing to consider adding 
> this additional functionality?
> 
> Regards,
> 
> Adrian
> 
> 
> > Al the information to support multi-homing and the associated 
> > behaviour can be introduced through a set of ROOT 
> properties that can 
> > be either provisioned on the MG or set there after the 
> initial control 
> > association has been established.
> >
> >
> > Adrian Soncodi wrote:
> >
> >>
> >>
> >> Fernando Vallejo Lazaro wrote:
> >>
> >>> Hi,
> >>>
> >>> I agree with Flemming that Megaco should be based on UDP and TCP.
> >>> SCTP is not widely deployed and can lead to problems of 
> >>> interoperability.
> >>> I would leave it as optional.
> >>>
> >>> For me the control association is clearly defined and it is 1 
> >>> association = 1 IP address.
> >>> Why? Because the assocation is done through a 
> ServiceChange Restart 
> >>> message which involves
> >>> an IP address (in the MGC ID)  and not a domain name. If the DNS 
> >>> returns more than
> >>> one IP address for a given name the MG must choose one of them.
> >>>
> >> This is how one would be tempted to interpret the spec today, 
> >> although it is not explicitly stated. But as I have mentioned in a 
> >> previous e-mail, this is not adequate for SCTP, where one 
> would like 
> >> to know that the 2 IP addresses are on the same machine, 
> to be able 
> >> to use them within a single SCTP association.
> >>
> >> Also, the following quote from section 7.2.8 seems to 
> indicate more 
> >> than 1 IP address is possible:
> >>
> >>  "The ServiceChangeAddress provides an address to be used 
> within the 
> >> Context of the association currently being negotiated ..."
> >>
> >> Since this is in the reply to ServiceChange, it looks like it can 
> >> actually be a different IP address (which then must be 
> interpreted in 
> >> the context of the current association ...). Otherwise, 
> why would the 
> >> MGC return the same IP address as the destination IP 
> address of the 
> >> ServiceChange message? The MG already knows that one!
> >>
> >>> What I feel is that the gap is in the definition of the secondary 
> >>> MGCs and its provisionning.
> >>> I think there should be a way in which the MGC could 
> inform the MGs 
> >>> about existing secondaries
> >>> MGCs. Now this can only be done using a  HandOff method but 
> >>> sometimes this message cannot
> >>> be sent by the primary MGC (e.g. network failure, crash 
> of the MGC 
> >>> ...).
> >>> So it would be useful to add a new method in the ServiceChange to 
> >>> send this information and add
> >>> a new parameter indicating what type of relationship there is 
> >>> between primary and secondary
> >>> (e.g. mirror, backup, independent, third party.....)
> >>> So in case a MGC is multihomed it could inform the MG at startup 
> >>> that a MGC exists which is a
> >>> secondary mirror MGC (which corresponds to the same MGC but with 
> >>> another IP address).
> >>> This process can be independent on the actual procedures 
> done by the 
> >>> operators to declare
> >>> secondary MGCs.
> >>>
> >> I'm not sure a "mirror MGC" is the same MGC with another 
> IP address. 
> >> Some people would interpret "mirrored" as different machines, with 
> >> some state synchronization between them...
> >>
> >> Anyway, considering 2 or more multi-homed MGCs, i.e. 4 or more IP 
> >> addresses, and if each IP address is a different MGC, then 
> it becomes 
> >> quite cumbersome to signal which one is paired with which one. I 
> >> think it would be simpler and more clear to allow an MG to be 
> >> provisioned with more than one IP address per MGC. Then you would 
> >> immediately know the MGC is multi-homed. This works more elegantly 
> >> with SCTP, and also with DNS, which can return several IP 
> addresses 
> >> for an FQDN.
> >>
> >> Adrian
> >>
> >>> When a problem arises the MG can act in two ways:
> >>> 1) If the new MGC is a mirror (maybe other types can be included 
> >>> here) it can resend the transactions
> >>> which have failed in the old one.
> >>> 2) If the new MGC is not a mirror it should try to establish a 
> >>> connection with a ServiceChange as defined
> >>> in the megaco protocol. Then the MGC is in charge of 
> recovering the 
> >>> state of the MG ASAP and the service.
> >>>
> >>> In this way not too much work has to be added and we can 
> accomplish 
> >>> a solution.
> >>>
> >>> Regards
> >>> Fernando
> >>>  
> >>>
> >>> Flemming Andreasen wrote:
> >>>
> >>>> Qiaobing
> >>>>
> >>>> Nobody is denying that SCTP is an existing spec that solves the 
> >>>> multihoming problem and it
> >>>> is quite understandable that you in particular advocate it. 
> >>>> However, you seem to have lost
> >>>> sight of the original question in this thread, which was about 
> >>>> native Megaco protocol
> >>>> support for redundancy and failover, and as part of 
> that, support 
> >>>> for multihoming. Initial
> >>>> claims was that Megaco fully supports this, but we seem to have 
> >>>> converged on the opinion
> >>>> that it does not, unless you use SCTP, which is not part of the 
> >>>> base Megaco protocol.
> >>>> Furthermore, despite everything, the fact of the matter 
> is that for 
> >>>> various reasons, not
> >>>> everybody will be implementing SCTP at least for now (NATs, 
> >>>> firewalls and security do not
> >>>> help SCTP here) and since UDP and TCP are the basic 
> requirements, 
> >>>> it would have been nice to
> >>>> have better mechanisms for these; it can be done very 
> easily (see 
> >>>> MGCP for example).
> >>>> Sometimes, it's good to be a bit pragmatic.
> >>>>
> >>>> -- Flemming
> >>>>
> >>>> Qiaobing Xie wrote:
> >>>>
> >>>> > (Try to respond to similar remarks from two mails)
> >>>> >
> >>>> > Flemming Andreasen wrote:
> >>>> > ...
> >>>> > > In any case, SCTP is there as an option today for those
> >>>> > > that wish to use it. For those that don't, we've already 
> >>>> concluded that
> >>>> > > multihoming/redundancy/failover is not very well 
> supported, and 
> >>>> that there doesn't
> >>>> > > seem to be much desire to change that. I personally think 
> >>>> that's a bit of a shame, but
> >>>> > > so be it.
> >>>> > ...
> >>>> >
> >>>> > Adrian Soncodi wrote:
> >>>> > ...
> >>>> > > You seem to say "if you want multi-homing, it must 
> be only over 
> >>>> SCTP".
> >>>> > > Why? Someone could similarly say "if you want 
> MEGACO, it must 
> >>>> be over
> >>>> > > UDP". But we don't, we allow all 3 protocols, for 
> flexibility.
> >>>> > >
> >>>> >
> >>>> > I didn't mean to say "you must". I was under the 
> impression that 
> >>>> some of
> >>>> > us were saying that we had a technical issue to support 
> >>>> multi-homing in
> >>>> > MEGACO, and my response was essentially "this is not an issue, 
> >>>> there are
> >>>> > protocol specs existing in IETF to do just that, which is
> >>>> > MEGACO/H.248-Annex-H plus SCTP." Now, I think what I 
> am hearing 
> >>>> is, it
> >>>> > is not really about the lack of a solution, it's more 
> about that 
> >>>> some of
> >>>> > us just don't want to use the existing IETF solution 
> (for whatever
> >>>> > non-tech reasons, or simply just don't wish to use it) 
> and prefer to
> >>>> > define another solution in IETF. Well, if that is the case, I 
> >>>> don't have
> >>>> > much else to say...
> >>>> >
> >>>> > Of cause, if there are tech 
> issues/shortcomings/drawbacks/advantages
> >>>> > related to using SCTP for MEGACO multi-homing support, I'd be 
> >>>> more than
> >>>> > glad to help answer.
> >>>> >
> >>>> > -Qiaobing
> >>>>
> >>> -- 
> >>> Fernando Vallejo Lazaro
> >>> Phone:  +34 91 330 5614
> >>> SW  Wireline ASD-Madrid
> >>>       ALCATEL
> >>>
> >>>  
> >>
> >>
> >>
> >>
> >>

---

From: Steve Cipolli <[email-masked]>
Date: Tue, 13 May 2003 18:37:52 -0400

--Stephen