Subject: Re: Multiple CEs talking to the same FE

From: "Putzolu, David" <[email-masked]>
Date: Tue, 15 Feb 2005 00:17:53 -0800

(hat off...)

From a technical point of view, wouldn't it be sufficient
for the process on the CE that is aggregating the multiple
processes to just demux responses based on LFB ID?  I.e.
any replies regarding the firewalling LFB go to the CE
firewall process, any replies regarding the routing table
be directed to the CE routing process?

-David

> -----Original Message-----
> From: Forwarding and Control Element Separation 
> [mailto:[email-masked]] On Behalf Of Robert Haas
> Sent: Monday, February 14, 2005 10:09 AM
> To: [email-masked]
> Subject: Re: [FORCES] Multiple CEs talking to the same FE
> 
> Coming back to implementation issues that fire up in the protocol:
> 
> In the case of a single active CE, there can be multiple processes 
> "hiding" behind the same CE ID, i.e., a process updating firewalling 
> rules and another process updating the routing table, in parallel. We 
> must be able to:
> 
> 1) correlate at the CE the responses from the FE with the 
> commands that 
> were originated by each process.
> 2)  forward at the CE FE-event notifications to the appropriate 
> processes that registered for it.
> 
> So if the CE ID is the same for all these processes, something is 
> missing in the protocol  to support 1) and 2) ... Unless the 
> ForCES PL 
> decides to use different TCP/UDP ports for each process (I wouldn't 
> recommend this), but this will anyway not solve 2).
> 
> BTW, the multi-CE problem is something one should still think 
> of while 
> defining the model. Although the requirements draft only talks of a 
> single active CE for each FE, it is a scalability issue if I 
> can't run 
> the firewalling and routing processes on two different boxes 
> (i.e. two 
> CEs). So let's at least consider the impact of the 
> multi-manager problem 
> on the model so we don't close all the doors for future support.
> 
> Regards,
> -Robert
> 
> Joel M. Halpern wrote:
> 
> > I thought we had agreed that for now we would focus on a 
> single active 
> > CE per FE>  An FE that wants to be managed by several CEs is (I 
> > thought) to pretend it is several FEs, probably with 
> distinct resources.
> >
> > The multi-manager problem is very very messy.
> >
> > The demultiplexing is a valid issue anyway.  I think that 
> the answer 
> > is probably that one should use different ports if one is 
> going to run 
> > several CEs (or FEs) on the same machine / interface.  I am 
> not sure 
> > we can / should introduce an extra demux layer for the "native 
> > ethernet transport" case.
> >
> > Yours,
> > Joel
> >
> > At 11:25 AM 1/12/2005, Robert Haas wrote:
> >
> >> Here are some considerations around many CEs talking to 
> the same FE, 
> >> based on early implementation experience:
> >>
> >> Associations: Multiple CEs may access the same FE, for instance to 
> >> accomplish different tasks, such as routing, security, 
> QoS. These CEs 
> >> may each run in a separate machine, or they may share the same 
> >> machine. So there is an association for each (CE-FE) pair. 
> Or do we 
> >> consider associations to include more than two members ? Note that 
> >> multiple associations may share the same TCP/SCTP/... connection.
> >>
> >> Demultiplexing: Note that in the case where multiple CEs 
> run on the 
> >> same machine, the CE ID can be viewed as a demultiplexor. 
> The ForCES 
> >> protocol does not otherwise include correlators that would enable 
> >> callbacks to the appropriate CE on the response to a 
> ForCES command 
> >> (or reception of an event).
> >>
> >> Access conflicts: The framework draft specifies that CE 
> >> synchronization is beyond the scope of ForCES. This means 
> that, for 
> >> instance, two CEs changing the same LFB attribute value, is not 
> >> viewed at the ForCES level as a conflict. But if two CEs try to 
> >> perform interleaved 2-phase commit operations on the same 
> attribute, 
> >> then ForCES should of course prevent it and report it.
> >>
> >> Event reporting: If multiple CEs subscribe to different 
> events in an 
> >> LFB, then the LFB must somehow know to which CE(s) events must be 
> >> fired. I propose that a list of interested CE IDs is 
> associated with 
> >> each event attribute in the LFB.
> >>
> >> Regards,
> >>
> >> -- 
> >> Dr. Robert R. Haas
> >> IBM Zurich Research Laboratory
> >> Säumerstrasse 4
> >> CH-8803 Rüschlikon/Switzerland
> >> phone +41-1-724-8698  fax +41-1-724-8578  
> http://www.zurich.ibm.com/~rha
> >
> >
> >
> 
> -- 
> Dr. Robert R. Haas
> IBM Zurich Research Laboratory
> Säumerstrasse 4
> CH-8803 Rüschlikon/Switzerland
> phone +41-1-724-8698  fax +41-1-724-8578  
> http://www.zurich.ibm.com/~rha
>

---

From: Robert Haas <[email-masked]>
Date: Tue, 15 Feb 2005 16:16:52 +0100

>(hat off...)
That's a very polite (although fallen into disuse) way to greet someone ;-)

>be directed to the CE routing process?
This would not always work: both firewalling and routing processes may 
want to query information from the same LFB (say, the port LFB).

Regards,
-Robert

-- 
Dr. Robert R. Haas
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
phone +41-1-724-8698  fax +41-1-724-8578  http://www.zurich.ibm.com/~rha

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 15 Feb 2005 10:29:02 -0500

It would seem to me that from the design point we are working, if two 
components want to work with the same LFB then they will need close 
coordination anyway.  So a dispatch mechanism, while convenient, is not a 
large benefit to the application.  And it adds complexity to the protocol 
just for that special case.

Yours,
Joel

At 10:16 AM 2/15/2005, Robert Haas wrote:
>>>1) correlate at the CE the responses from the FE with the commands that 
>>>missing in the protocol  to support 1) and 2) ... Unless the ForCES PL 
>>>BTW, the multi-CE problem is something one should still think of while 
>>>single active CE for each FE, it is a scalability issue if I can't run 
>>>the firewalling and routing processes on two different boxes (i.e. two 
>>>CEs). So let's at least consider the impact of the multi-manager problem 
>>>>we can / should introduce an extra demux layer for the "native ethernet 
>>>>transport" case.
>>>>
>>>>Yours,
>>>>Joel
>>>>
>>>>>instance, two CEs changing the same LFB attribute value, is not viewed 
>>>>>at the ForCES level as a conflict. But if two CEs try to perform 
>>>>>interleaved 2-phase commit operations on the same
>>>attribute,
>>>
>>>>>-- Dr. Robert R. Haas
>>>-- Dr. Robert R. Haas

---

From: Robert Haas <[email-masked]>
Date: Tue, 15 Feb 2005 17:12:16 +0100

Joel M. Halpern wrote:

> coordination anyway.

How this coordination is implemented (such as a layer above ForCES that 
ensures coordination among actions performed by various 
processes/applications, or sth else) is beyond the ForCES protocol 
specification. An FE will obey to all commands issued by a given CE ID, 
even if it means contradicting operations issued by applications that 
are badly synchronized/configured/coded.

I do not want to hide that in my mind, in the case of multi-CEs (but not 
in the case of multi-processes behind the same CE), the FE could 
implement simple mechanisms to avoid such contradicting operations. For 
instance, SET operations to any given LFB would only be allowed from one 
CE. But all CEs could query all LFBs or subscribe to all events, as this 
would not cause inconsistencies.

>   So a dispatch mechanism, while convenient, is not a large benefit to 
> the application.  And it adds complexity to the protocol just for that 
> special case.

I argue that correlators would be very useful even to the "coordination 
layer" too, if not directly to the applications/processes running in the CE.
That's because the coordination layer anyway has to decide which 
application/process should it send an event upon receing a notification 
from the FE. And there could be more than one application interested in 
notifications when security events occur.

Regards,
-Robert

>> This would not always work: both firewalling and routing processes 
>> may want to query information from the same LFB (say, the port LFB).
>>
>>
>> Regards,
>> -Robert
>>
>>
>>> -David
>>>
>>>
>>>
>>>> "hiding" behind the same CE ID, i.e., a process updating 
>>>> firewalling rules and another process updating the routing table, 
>>>> in parallel. We must be able to:
>>>>
>>>> 1) correlate at the CE the responses from the FE with the commands 
>>>> that were originated by each process.
>>>> missing in the protocol  to support 1) and 2) ... Unless the ForCES 
>>>> PL decides to use different TCP/UDP ports for each process (I 
>>>> wouldn't recommend this), but this will anyway not solve 2).
>>>>
>>>> BTW, the multi-CE problem is something one should still think of 
>>>> while defining the model. Although the requirements draft only 
>>>> talks of a single active CE for each FE, it is a scalability issue 
>>>> if I can't run the firewalling and routing processes on two 
>>>> different boxes (i.e. two CEs). So let's at least consider the 
>>>> impact of the multi-manager problem on the model so we don't close 
>>>> all the doors for future support.
>>>>
>>>> Regards,
>>>> -Robert
>>>>
>>>> Joel M. Halpern wrote:
>>>>
>>>>
>>>>
>>>>>> Associations: Multiple CEs may access the same FE, for instance 
>>>>>> to accomplish different tasks, such as routing, security,
>>>>>
>>>> QoS. These CEs
>>>>
>>>>>> consider associations to include more than two members ? Note 
>>>>>> that multiple associations may share the same TCP/SCTP/... 
>>>>>> connection.
>>>>>>