Subject: Re: updated text on HA section

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Thu, 13 Oct 2005 22:30:02 -0700

Fair enough, I'll add that note.

Thanks
Hormuzd

-----Original Message-----
From: Forwarding and Control Element Separation
[mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Thursday, October 13, 2005 7:55 PM
To: [email-masked]
Subject: Re: [FORCES] updated text on HA section

I think the main point we are making here is that while we are putting
in 
hooks to support full redundancy, we are not claiming to have built all
the 
pieces that are needed.  We have not even built all the hooks probably.
We need to be clear about that.

Yours,
Joel

At 06:50 PM 10/13/2005, Khosravi, Hormuzd M wrote:
>Good point Furquan.
>
>I am not sure we can solve this in an elegant way or any way without
some 
>sort of CE-to-CE communication mechanism.
>I'll send out the text with the addition of the CE Multicast ID case as

>Jamal suggested, but that might not be directly applicable for this
case.
>
>We can definitely mention this as an issue in the draft and propose
some 
>solutions based on CE-to-CE mechanisms, which are not yet defined.
>
>Thanks for reviewing this!
>Hormuzd
>
>
>----------
>From: Forwarding and Control Element Separation 
>[mailto:[email-masked]] On Behalf Of Furquan A. Ansari
>Sent: Thursday, October 13, 2005 9:03 AM
>To: [email-masked]
>Subject: Re: [FORCES] updated text on HA section
>
>Hello Hormuzd/all,
>     Haven't been closely following the discussion recently  on the 
> mailing list since I was quite
>busy with a few things. However, after quickly going through the HA 
>support section, I would be
>hesitant to put this in the current draft. I feel that the HA situation
is 
>more complicated than
>spelled out here and in fact could/should come as a separate draft in
its 
>own right.
>
>Here are a few things that complicate the situation:
>1) In an NE with multiple CEs (Active and backups) and FEs, it is 
>mentioned that a communication
>failure between a CE and an FE - due to either link reasons or CE
failure 
>- it would trigger the FE to
>switch over to the backup (using PL/TML) CE. This can lead to a "very
bad 
>situation" in case of loss
>of communication due to link failure. We do not want to be in a
situation 
>where only "one" of the FEs is
>now being managed/controlled by the backup CE while all other FEs are 
>being controlled by the active
>CE (since the active CE is still functional and the loss of association

>between the active CE and one of
>the FEs in question occured either due to link failure or TML channel 
>failure etc.). This not only
>destroys the notion of the having a single active CE within an NE
(since 
>we now have the original active
>CE still controlling the other FEs along with the backup CE now acting
as 
>an active CE for "one" FE), but
>making the whole collection of CE and FEs function together as a single
NE 
>would be close to impossible,
>since higher layer applications such as routing protocols etc. will now

>have to be run from both the CEs.
>
>So, essentially, I would say that there are a few choices we have:
>1) If one of the FEs is switching over to the backup CE, then all other

>FEs will have to switch over the
>backup CE as well
>     - Don't like this choice very much, because this would cause great

> disruption at the network level since applications will have to be 
> restarted at the backup (e.g. routing protocols)
>2) When a majority/all of the FEs are reporting loss of the
connectivity 
>to the active CE (reported to the backup CE), only then should the
backup 
>CE take over the functionality as the active CE.
>    - Better choice than before, but still not the best one - since
what 
> happens to the FEs that are not
>reachable by the active CE? How do we establish control over them since

>the communication is lost?
>Should the backup CE somehow help this FE in trying to establish the 
>connection/association back to the active CE?
>
>I can go on with many scenarios, but will stop here for now.
>
>Regards,
>
>
>Khosravi, Hormuzd M wrote:
>>The text is attached as per the protocol team discussions.
>>
>>regards
>>Hormuzd
>>
>>
>>
>>
>>8. High Availability Support
>>
>>The ForCES protocol provides mechanisms for CE redundancy and
failover,
>>in order to support High Availability as defined in [RFC3654].  FE 
>>redundancy
>>and FE to FE interaction is currently out of scope of this draft.
There
>>can be multiple redundant CEs and FEs in a ForCES NE.  However, at any
>>time there can only be one Primary CE controlling the FEs and there
can
>>be multiple secondary CEs.  The FE and the CE PL are aware of the
primary
>>and secondary CEs.  This information (primary, secondary CEs) is
configured
>>in the FE, CE PLs during pre-association by FEM, CEM respectively.
Only
>>the primary CE sends Control messages to the FEs.
>>
>>There are 2 modes of HA defined in the ForCES protocol, Report Primary
Mode
>>and Report All Mode. The Report Primary Mode is the default mode of
the 
>>protocol,
>>in which the FEs only associate with one CE (primary) at a time. The 
>>Report All
>>mode is for future study and not part of the current protocol version.
In 
>>this mode,
>>the FE would establish association with multiple CEs (primary & 
>>secondary) and
>>report events, packets, Heart Beats to all the CEs. However, only the 
>>primary CE
>>would configure/control the FE in this mode as well. This helps with 
>>keeping state
>>between CEs synchronized, although it does not guarantee
synchronization.
>>
>>The HA Modes are configured during Association setup phase, currently 
>>only Report
>>Primary Mode is configured.  A CE-to-CE synchronization protocol will
be 
>>needed
>>to support fast failover, however this will not be defined by the
ForCES 
>>protocol.
>>
>>During a communication failure between the FE and CE (which is caused
due 
>>to CE
>>or link reasons, i.e. not FE related), the TML on the FE will trigger
the 
>>FE PL
>>regarding this failure.  This can also be detected using the HB
messages 
>>between
>>FEs and CEs.  The communication failure (detected via either of the
two 
>>means
>>described before) is considered as a loss of association between the
CE and
>>corresponding FE and the FE PL must establish association with the 
>>secondary CE at
>>this point.  During this phase, if the original primary CE comes alive

>>and starts
>>sending any commands to the FE, the FE should ignore those messages
and 
>>send an
>>Event to all CEs indicating its change in Primary CE.  Thus the FE
only 
>>has one
>>primary CE at a time.
>>
>>An explicit message (Config message setting Primary CE attribute in 
>>ForCES Protocol
>>object) from the primary CE, can also be used to change the Primary CE

>>for an FE
>>during normal protocol operation.
>>
>
>
>--
>Furquan Ansari
>Bell Labs Research                      Ph : 732-949-5249
>Holmdel, NJ                             Fax: 732-949-4556

---

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Sun, 16 Oct 2005 18:58:46 -0700

Here is the updated figure for HA section.

regards
Hormuzd

         FE                   CE Primary        CE Secondary
         |                       |                    |
         |  Asso Estb,Caps exchg |                    |
       1 |<--------------------->|                    |
         |                       |                    |
         |       All msgs        |                    |
       2 |<--------------------->|                    |
         |                       |                    |
         |                       |                    |
         |                   FAILURE                  |
         |                                            |
         |         Asso Estb,Caps exchange            |
       3 |<------------------------------------------>|
         |                                            |
         |              Event Report (pri CE down)    |
       4 |------------------------------------------->|
         |                                            |
         |                   All Msgs                 |
       5 |------------------------------------------->|

   Figure 33: CE Failover for Report Primary Mode