Subject: Re: [vnfpool] RSerPool protocol experience

From: "Susan Hares" <[email-masked]>
Date: Mon, 16 Dec 2013 11:18:57 -0500

Melinda:

I'm trying to provide input for problem statement 2.  This post is to dig
into the issues you mentioned on the RSerPool protocol experience. 

Focus in the draft-zong-vnfpool-problem-statement-01: section 3.3 service
state synchronization plus impact on 3.1, 3.4, & 3.5 (see below for the
comments). 
  
-------------

Can you provide details on the comment: 
  "The principal authors had a hammer (SCTP) and wanted to pound things.
They failed to
   adequately address core problems around state sharing,  synchronization,
etc. - distributed computing 
   problems"

What exactly did the principal authors "hammer" with STCP?   And how to
impact it: 

Why am I curious: 

I convinced that the distribute of state has time-dependencies
(critical/urgent, quick, configuration state). 
Different algorithm have different strengths. 

State sharing for sub-second of massive amounts of  network-dependent data
has a study of routing for 30+ years.  A few algorithms there are extremely
useful to do state sharing.  One lesson for the SDN TE/Link state is that
the work horses of the Internet routing (ISIS/OSPF, BGP, VRRP) or name
distribution (DNS, DHCP) do a good job for the distribution with/or without
policy. 

Combining the routing state distribution with augmented TCP handling makes
sense for rapid congestion avoidance.  The state distribution (L1-L7) tells
me where the bottle necks are.   Link states algorithms and coding have
improved to be able to handle L1-L7 state distribution rapidly. 

Explicit congestion notification can be passed sub-second in order to tune
the TCP bandwidth and avoid the TCP saw-tooth.  If you distribute the levels
of the network congestion, you can avoid sending the whole system into the
repeat performance of TCP. 

So.. can we go a level deeper on this discussion ?

Sue Hares 

=====

Focus in draft-zong-vnfpool-problem-statement-01:

3.3) Service state synchronization:  
    a) Who is responsible and how to collect/keep service state of instance?

   b) How to synchronize service state with back-up instsance. 

Questions implied but not in document: 

1) Does the work need a cluster concept within a pool manager? 
    
To clarify this new question:  Do the verticals of mobile istores, medical
devices, and financial devices require a the concepts of a cluster to make
vertical NFV connections easy to make? 
 
 [There may be machine-2-machine devices that require the same things, but
since m2m is consider a bit off I have quoted the current applications.]  

2)  How does the VNF clusters interact with instance selection/status (3.1),
backup selection/status (3.2), transition handling (3.4), and policy
enforcement (3.5). 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Melinda Shore
Sent: Tuesday, September 17, 2013 4:21 PM
To: [email-masked]
Subject: Re: [vnfpool] RSerPool protocol experience

On 9/17/13 12:05 PM, Thomas Narten wrote:
> I believe they were published as experimental protocols. Have they 
> been implemented in any products? Do we have any deployment experience 
> with them?

I co-authored a few of those documents and I know of no deployments.

I was concerned when I first saw rserpool in the vnfpool documents, but
here's my take on it:

1) rserpool was not a good answer to the problem that
   it purported to address.  The principal authors had a hammer
   (SCTP) and wanted to pound things.  They failed to
   adequately address core problems around state sharing,
   synchronization, etc. - distributed computing problems

2) It's fine for providing a pooling mechanism, but an
   actual application built around it will require significant
   effort

3) the VNF reliability problem is probably a pretty good
   use of rserpool, as long as the work is properly scoped
   and the charter is very, very clear about what would
   need to be done.  This is not really a distributed
   computing problem.

To summarize, I think that the core problems with rserpool have to do with
its nearly complete inadequacy as a telephony gateway and/or switch pooling
solution.  That doesn't mean that it's inapplicable to other problems.

--
Melinda Shore
No Mountain Software
[email-masked]

"Software longa, hardware brevis."
_______________________________________________
vnfpool mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vnfpool

---

From: Melinda Shore <[email-masked]>
Date: Mon, 16 Dec 2013 07:35:40 -0900

On 12/16/13 7:18 AM, Susan Hares wrote:
> What exactly did the principal authors "hammer" with STCP?   

Sure: what SCTP provided, relevant to this question, was
multihomed endpoints.  There were other attractive transport
features, of course, but it was if they said "What can we do
with multihoming?" rather than asking "How can we do server
pooling/provide reliability/whatever?"

I tend to think that the further you move up the stack the
harder it is to do the kind of state distribution you're
asking for and keep it in the sub-second range.  For example,
firewalling and load balancing tend to have or need awareness
of session or application protocol state.  I really don't
think this devolves to a routing problem.

Melinda

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 16 Dec 2013 13:06:26 -0500

Melinda:

Thank you for the fast response.  

The multi-homing endpoints is a critical new feature for many types of
applications that we struggle with in IPv6 and many cases -- so I can image
why SCTP authors asked the question. 

Can we dig down on what you thought was valuable on the multi-homed
endpoints for the RSerPool work? 

My question are: 
------------------------
1) Since the beginning of web service on financial institutions the HA
demands required multiple interfaces on web-server, and fail-over between
web servers.  If we redefine the web server as a cluster of
web-applications, does the multiple-end points work? 

2) If we have the mega-web server broken down into secure functions (e.g.
financial transactions that a broker needs a secure network) versus queries
that are open (e.g. Details on what broker services), does the connection
coming in need to be split or multi-homed?  

3) Can you give an example of what you meant by multi-homed endpoints? 

Example Scenario for discussion 
-----------------------------------------------
Let's focus on the broker scenario to chat about this.  For example, the
financial transactions may require the broker to be run through firewall,
AAA server, load-balancing (to give every broker the same service response
time), and then into a cluster of  NFV VMs that provide the brokerage
services or web interfaces.  Within the Web services, there may be web
authorization or security. 

All of these brokerage services VMs may be connected to multiple places
within the Data Center or multiple data centers to obtain information.  The
financial VMs need compute, network, and data storage. Connecting all three
according to business policy is critical to balancing the Financial houses
budget. 

Some of the connections may need to be secure (due to transfer of money or
privacy issues). 
--------

Thanks !

Sue 

-----Original Message-----
From: vnfpool [mailto:[email-masked]] On Behalf Of Melinda Shore
Sent: Monday, December 16, 2013 11:36 AM
To: [email-masked]
Subject: Re: [vnfpool] RSerPool protocol experience

> What exactly did the principal authors "hammer" with STCP?   

Sure: what SCTP provided, relevant to this question, was multihomed
endpoints.  There were other attractive transport features, of course, but
it was if they said "What can we do with multihoming?" rather than asking
"How can we do server pooling/provide reliability/whatever?"

I tend to think that the further you move up the stack the harder it is to
do the kind of state distribution you're asking for and keep it in the
sub-second range.  For example, firewalling and load balancing tend to have
or need awareness of session or application protocol state.  I really don't
think this devolves to a routing problem.

Melinda

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Dec 2013 19:25:17 +0100

> state synchronization plus impact on 3.1, 3.4, & 3.5 (see below for the
Hi Susan,

a very high level view:
If you want to build a fault tolerant system, SCTP deals with network failures
by using multiple path (multi-homing) and RSerPool deals with end-point failures
by using multiple end-points (server pools).

Best regards
Michael

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 16 Dec 2013 13:53:15 -0500

Michael: 

<Snip> 
>> What exactly did the principal authors "hammer" with STCP?   And how to
impact it: 
<snip> 
>> 
>Hi Susan,
>
>a very high level view:
>If you want to build a fault tolerant system, SCTP deals with network
failures by using multiple path (multi->homing) and RSerPool deals with
end-point failures by using multiple end-points (server pools).
>Michael

Michael:

This matches my understanding.  Both are necessary to build a product.  

Three resources needed for VMs: computer, network, storage.  

If the VNF Pools needs end-point reliability: see RSerPools or others such
pools.   
If the VNF Pools  network pathway reliability: see SCTP (transport) +
network pathways 
If the VNF Pools needs storage check-offs, see another protocol. 

Sue 

<snip>

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Dec 2013 20:04:50 +0100

> If the VNF Pools needs storage check-offs, see another protocol. 
You could use RSerPool to handle a pool of file servers. The syncing
between the file servers has to be done somehow. But you could use
RSerPool to say "I want to connect to a file server" and get back
the IP address / port number / protocol for your NFS server, for example.
The service provided by the server pool is generic and not limited in
any way by RSerPool.

Best regards
Michael

---

From: Melinda Shore <[email-masked]>
Date: Mon, 16 Dec 2013 13:28:12 -0900

> web-applications, does the multiple-end points work? 

I hope to holler that application services are out of
scope for this particular piece of work.  I am not
particularly a fan of rigid layering and I think that
layering discussions in the IETF have tended to rathole
(see: nvo3) but some clarity about it can help prevent
unfortunate architectural mistakes.  For example, the
nsis working group had a difficult time conceptualizing
lines between layers and ended up with a clunky,
expensive security mechanism, while I think we got that
one right in NLS by having a clearer model of what's
in and what's out and by focusing on protecting the
protocol components we can expect to have "knowledge"
of.

In this case, I think we need to be clear about what
a virtualized network function can reasonably be
expected to have knowledge of, in terms of protocol
and network state.  A firewall may track certain
aspects of application protocol state and we may want
to provide reliable firewall modularization.  But
doing that does not necessitate putting a virtualized
web service into the NFV graph, any more than a web server
runs directly on a firewall (leaving aside for the
moment specialized use-specific web services, like a
management interface).

I think that aspects of this are going to be difficult
enough that we probably want to avoid vertical applications
and keep a tight focus on basically decomposing a
network device.

Melinda

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 16 Dec 2013 18:47:05 -0500

Melinda:

While you are holler that application services are out of scope, can we
refocus on the question and use case I asked about?   Why do you want to
avoid consider vertical applications as a use case for deployment?  The
medical, financial, insurance, business SAP/Order transactions, mobile store
applications actually consume DC and network resources.  Are you looking for
only home-based NFV (firewall, load balancers, AAA, email spam reduction)?  

If you start by answering that question, it would help me understand your
market.  I gave a use cases in a set of markets for you to solve. If some of
the use case below are out, just let me know and why.  If a use case is in,
tell me where you draw the lines for the NFV graph and why. 

The Pool Managers will come in different sizes and shapes - just as routing
PCEs, SDN controllers, and NSC calculation devices.  Allowing a protocol
with options allows vendors to innovate.  

By the way, there are reasons to shorten the web-server distance to the
firewall.  Pop-up ads run in the delay it take to get your web page. 

Inquiring minds want to know, 
Cheerily, 

Sue 

-----Original Message-----
From: vnfpool [mailto:[email-masked]] On Behalf Of Melinda Shore
Sent: Monday, December 16, 2013 5:28 PM
To: [email-masked]
Subject: Re: [vnfpool] RSerPool protocol experience

On 12/16/13 9:06 AM, Susan Hares wrote:
> demands required multiple interfaces on web-server, and fail-over 
> between web servers.  If we redefine the web server as a cluster of 

I hope to holler that application services are out of scope for this
particular piece of work.  I am not particularly a fan of rigid layering and
I think that layering discussions in the IETF have tended to rathole
(see: nvo3) but some clarity about it can help prevent unfortunate
architectural mistakes.  For example, the nsis working group had a difficult
time conceptualizing lines between layers and ended up with a clunky,
expensive security mechanism, while I think we got that one right in NLS by
having a clearer model of what's in and what's out and by focusing on
protecting the protocol components we can expect to have "knowledge"
of.

In this case, I think we need to be clear about what a virtualized network
function can reasonably be expected to have knowledge of, in terms of
protocol and network state.  A firewall may track certain aspects of
application protocol state and we may want to provide reliable firewall
modularization.  But doing that does not necessitate putting a virtualized
web service into the NFV graph, any more than a web server runs directly on
a firewall (leaving aside for the moment specialized use-specific web
services, like a management interface).

I think that aspects of this are going to be difficult enough that we
probably want to avoid vertical applications and keep a tight focus on
basically decomposing a network device.

Melinda