Subject: Re: [issue 7] HB msg

From: Robert Haas <[email-masked]>
Date: Tue, 11 Jan 2005 11:48:01 +0100

On the Heartbeat exception:

First we all agree that, at least from a logical point of view, the HB 
originates from the FE Protocol LFB to confirm 
ForCES-protocol-processing availability. The question is in what format 
should this message be sent from the FE to the CE. I try to list them 
all here:

1) give HB its own PL-message header type, the PL-message body is empty.

2) give HB the format of an event: LFB-Selector selects the FE Protocol 
LFB, command is EVENT-REPORT and path-data identifies the heartbeat 
event attribute in the FE Protocol LFB. Data is empty.

3) give HB its own command type, and hence no path-data needs to be 
transmitted, just an LFB-selector selecting the FE Protocol LFB and a 
special HB Command.

Most importantly, one may want to extend the use of HB to other LFBs, 
i.e., beyond only verifying ForCES-protocol-processing availability. For 
instance, a HB coming from the FE LFB to indicate liveliness of the 
overall FE. For that purpose only formats 2) and 3) are suitable.

_If we think it is worth extending the use of HB beyond verifying 
__ForCES-protocol-processing availability_, then we can reconsider its 
format, for the sake of remaining consistent.

Regards,
-Robert

Khosravi, Hormuzd M wrote:

>-----Original Message-----
>From: Jamal Hadi Salim
>  
>
>>You definitely have a different opinion on this from what we have in
>>    
>>
>the
>  
>
>>draft
>>    
>>
>
>Why would we be discussing it otherwise???
>my recollection is _you_ put it in the draft with intent we will revisit
>and disscuss.
>I sent the attached document back in October to accelerate the changes
>to the original draft. Look at section 6.7 - I gave you a clear
>opportunity to discuss this and didnt impose. Now its in the draft, is
>the implication that it no longer holds?? Sounds almost like a lawyer
>sneaking in fine print.
>
>[HK] Jamal, lets not get personal here. This issue has been discussed by
>the protocol team in context with the attached document that you sent
>out in October. The emails that I forwarded previously (also exchanged
>in October) were pertaining to that discussion. Based on that email
>discussion, I was under the impression that this issue is resolved >> go
>with the old approach for HB msg that we had in the draft. Anyways, you
>are free to revisit issues...hopefully we will eventually reach some
>resolution :)
>
>There hasn't been much response yet, expect from Joel.
>
>Regards
>Hormuzd
>
>
>  
>

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 11 Jan 2005 12:27:21 -0500

On Tue, 2005-01-11 at 05:48, Robert Haas wrote:
> First we all agree that, at least from a logical point of view, theHB
> ForCES-protocol-processing availability.

Nod. 

>  The question is in what format should this message be sent from the
> FE to the CE. I try to list them all here:
> 
> 2) give HB the format of an event: LFB-Selector selects the FE
> Protocol LFB, command is EVENT-REPORT and path-data identifies the
> heartbeat event attribute in the FE Protocol LFB. Data is empty.
> 

I think this is what i am leaning towards ...
A heartbeat is a synchronous _event_. A CE needs to kick the 
FE::LFBofinterest to start emiting the events (I suppose after
setting the proper parameters such as clock interval and dead interval).
One could say a basic heartbeat will have a zero sized message.
OTOH, an synchronous event such as the one emited by the topology LFB
will have > zero sized data (if configured to periodically send updates
to the CE).

The source of this message is the LFB object if going from CE to FE. 
Thats what has been configured to emit these synchronous events.
What i am having problems with so far is the restriction of this message
to only the FE protocol LFB and then taking further shortcuts to say
we dont even need to mention the FE protocol LFB in the addressing.

>3) give HB its own command type, and hence no path-data needs to be >
> i.e., beyond only verifying ForCES-protocol-processing availability.
> For instance, a HB coming from the FE LFB to indicate liveliness of
> the overall FE. For that purpose only formats 2) and 3) are suitable.
> 
> If we think it is worth extending the use of HB beyond verifying
> ForCES-protocol-processing availability, then we can reconsider its

Thanks for pointing this Robert.
I think its worth it to attach it to the LFB reference. I really see no
harm whatsoever in adding the extra bits to reference the FE protocol
LFB in the case of the FE heartbeat. The example of the synchronous
topology LFB updates is one example. I cant lay my hands on whether an
MPLS ping will fit in such a scenario as well. Regardless, I think
restricting the addressing to _not_ include LFB reference is an issue
that needs to be addressed.

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 11 Jan 2005 13:19:44 -0500

If the heartbeat goes only from the FE to the CE, treating it as an Event 
makes sense to me.  That seems to be consistent re-use of mechanisms that 
have general utility.

The problem I see with that is the CE->FE direction.  Except for this case, 
the FE logic has no reason to be able to receive event messages.
One alternative is to redefine heartbeat slightly.
Define it as simply that the CE and FE must hear from each other at the 
configured interval.
The FE generates his side by generate heartbeat notifications if he has 
nothing else to say.
The CE can deal with his side by sending null protocol messages (messages 
with no operations).

Does that work for people?

Thank you,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 11 Jan 2005 15:01:50 -0500

On Tue, 2005-01-11 at 13:19, Joel M. Halpern wrote:
> The problem I see with that is the CE->FE direction.  

Sorry, a typo on my part when i typed that in. Meant FE->CE.

> Except for this case, 

Scratch the CE->FE direction; its FE->CE

> Does that work for people?

Sounds neat and will reduce the amount of noise. 
Other adaptive heartbeats scheme i have seen (can be used in conjunction
with what you propose) have the CE adjust the FE timers depending on the
number of FEs present. Example, its 1 sec for upto 100 FEs and 2 seconds
for [101,200] 4 seconds interval for > 200 nodes etc ... Some adaptation
of this is used by mmusic mbus protocol.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 12 Jan 2005 10:05:47 -0500

>  makes sense to me.  That seems to be consistent re-use of mechanisms that 

Attached sample FE Protocol Object definition. We havent settled on the
events definition/subscription (so this could be way off). A
subscription to an event in this case is equivalent to just kicking the
LFB to start generating events/HBs.

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 12 Jan 2005 11:01:52 -0500

Thanks.  That is quite readable.  (I hope it is as readable when we have 
the final format and all the other information.)

I presume HBI (Heartbeat Interval) is the rate at which the FE will 
generate Heartbeat events.
HBDI (Heartbeat Dead Interval) is presumably a period at which the FE would 
declare the CE dead?  After all, the FE does not care when the CE will give 
up if it does not get heartbeats.  But if it is an FE timer, what resets 
it?  Any messages from the CE?  Including the null message I hypothesised 
in an earlier reply?  Or something else?

Thank you,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 12 Jan 2005 17:11:59 -0500

> the final format and all the other information.)

BTW, I used u32 instead of uint32 as the type - not sure why the doc
validated.

> it?  

Good question.
The original goal was for HA purposes. This hasnt been discussed in
details yet; however, the general view at the time was for the FE to
associate (failover) to another CE after that deadinterval period of not
hearing from the router - as defined by policy which resides in that
same LFB definition. Two such failover methodologies are mentiooned
briefly infact in the protocol draft.
I left that blank for now since i wasnt sure what to put in.

> Any messages from the CE?

We need to discuss it. Suggestion: The lack of any messages from the CE
after a period of HBI will result in a FE firing a HB message.
The lack of any message within DeadInterval will force the FE to ask
for an ACK for its HB message (by setting the ACK flag in the header).

>  Including the null message I hypothesised  in an earlier reply?  
> Or something else?

Hasnt been discussed in details. This is an opportunity to do so.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 16 Jan 2005 18:51:59 +0800

Joel,

We actually have had quite a long discussion on the need for a specific HB msg
or just use events for HBs.I think my thought complies  with yours well. More
detailedly, my thought is:

1. There are two entities related to HB, one is the HB attribute, the other is
HB event.
2. For FE, HB event and HB attribute belongs to FE object attributes. For CE,
there is only HB event that belongs to CE events.
3. HB attribute at FE can be set by CE to decide its parameters like the
interval. ForCES has no access to HB parameters generated by CE.
4. To start HBs in FE is at the time to subscirbe it. There is no access for
ForCES to decide when to start HB from CE.
5. PDU of HB events can fully be the same as general Event notification message.

I think we have not met the issue of how the Event Notification message
generated by CE is formatted. I'm sure when this is resolved, the message format
for HB from CE to FE can also be resolved.

Now, I think, we come to the discussion again on if we need a specific HB
message for ForCES.

Thank you.
Weiming

---

From: Jon Maloy <[email-masked]>
Date: Tue, 25 Jan 2005 15:10:04 -0500

I think it it time for me to reiterate what I said during the last meeting
in DC.
As I see it, it is not the task of the PL layer to prescribe how
connection or peer failure should be performed, i.e, whether it should
be by heartbeats or any other mechanism. The PL layer could state
a _maximum failure detection time_ for the connection, register an event
for this type of failure in the PL/TML API, and then leave the details to
the TML.
It is true that for TCP and for most other  TMLs we will probably
implement this as a heartbeat supervision, but in some cases (e.g.
when a inteface board can detect and report carrier loss), failure can
be detected much earlier. This is not an possibility we should preclude
by making unnecessary assumptions in the PL layer.
Also, there are many ways to implement a HB protocol, and I see no
reason for the PL to impose a rigid and costly scheme of  failure
detection which the TML potentially (I know an example) can make 
more efficient than the PL layer ever can do.

If there is no agreement on this, we should as a minimum make the
heartbeat scheme PL-to-PL an optional feature.

Regards /jon