Subject: Re: [Anima] [ANIMA] Self-Managed Networks

From: John Strassner <[email-masked]>
Date: Sat, 17 Oct 2015 12:02:08 -0700

Hi,

sorry for the late reply, I've been buried in writing I-Ds.

In response to the original train of emails (not the latest from
Mehmet), Michael made a key point:

> The ACP is self-managing. It needs to do self-healing, and
> automatically adapt to new situations. But to me, this isn’t
> fault management or performance management as an
> operator understands it.

The last phrase ("as an operator understands it) is what is so
important. We need to differentiate between functions that
the Autonomic System provides and functions that are visible
to the operator.

When an autonomic system self-heals, why would the operator
need to know? This is comparable to routing doing its job and
changing the path. There is no reason that the operator has to
be alerted on every change in the system - there are already
WAY TOO MANY of these for the operator to handle.

IMHO, there are core functions provided by the ACP, and functions
that sit on top of the ACP and use these core functions. FM and PM
should sit on top of the ACP. Otherwise, the ACP doesn't scale,
because why is FM or PM more important than config mgmt.,
security, accounting, analytics, ...?

Finally, I don't see any difference between a) and b). You have an
ACP, and you have a set of Autonomic Functions that use the ACP.
(You probably have different layers of Autonomic Functions, where
a core set of Autonomic Functions are reused by other Autonomic
Functions to provide more powerful Autonomic Functions, but this
hasn't been discussed yet (I think)). The point is that you must start
with a small set of Autonomic Functions. Otherwise, the ACP becomes
a big, bloated, Ball of Mud and will end up useless.

regards,
John

-- 
regards,
John

---

From: "Michael Behringer (mbehring)" <[email-masked]>
Date: Sun, 18 Oct 2015 09:11:17 +0000

John wrote: 

I think this is a key point that many folks that start in ANIMA don't understand. The ACP is a tool kit; autonomic functions are the true value add. Of course, the tool kit has some value add as well, but the focus of the ACP is simplicity, and providing value to autonomic functions. 

We know that we've succeeded when folks *outside* the autonomic work space (IETF and elsewhere) come to us and ask how they can leverage the features of the ACP. Because it should make their job a lot easier. 

Michael

---

From: Michael Richardson <[email-masked]>
Date: Mon, 19 Oct 2015 13:39:43 -0400

John Strassner <[email-masked]> wrote:
    > When an autonomic system self-heals, why would the operator need to
    > know? This is comparable to routing doing its job and changing the
    > path. There is no reason that the operator has to be alerted on every
    > change in the system - there are already WAY TOO MANY of these for the
    > operator to handle.

Yes, but just the same, the healing activity needs to be an audited event.
The ACP should be involved in capturing these events.

Sometimes things don't heal perfectly; often moving traffic from
one system to another causes a load spike, and while things continue to
function, the performance is different.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Reddy Pallavali <[email-masked]>
Date: Mon, 19 Oct 2015 19:58:41 +0100

Michael,

Yes, for every small issue there is no need to contact operator except, if
it is a complex one.
Even for self-healing we can use Swarm Intelligence or Artificial
Intelligence techniques, but for learning may take some time. However,
these are scalable techniques.
Here, also there is a need of consensus, if the system is fully distributed
and asynchronous need to know which object is crashed, how to schedule the
loads and coverage issues as well.

On Mon, Oct 19, 2015 at 6:39 PM, Michael Richardson <[email-masked]>
wrote:

> _______________________________________________
> Anima mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/anima
>
>

-- 
Thank you very much for your time and consideration.

Yours sincerely,

P Radha Krishna Reddy, M.Sc, M.Tech,

Junior Researcher/Ph.D Student,

COPE LABS, Universidade Lusofona, Lisboa - Portugal.
Author of: Security Issues of Cloud Computing over General & IT Sector
Mobile: +351923095671
www.prkreddy.webs.com
http://pt.linkedin.com/in/reddypallavali/

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 19 Oct 2015 15:04:45 -0400

There are at least two different factors to consider in terms of what 
events are communicated to the operator.

First, for initial deployment of autonomic technologies, operators will 
want ways to observe all the events so they can gain confidence in the 
technology.  (We received these requests with existing self-organizing 
network technologies.)

Secondly, different operators have different degrees of information 
harvesting and display relative to the state of the network.  Even if 
the network is taking care of itself perfectly, some operators will want 
visibility to any internal state change.  These operators currently 
monitor their routing protocol state for similar effects.

Yours,
Joel

On 10/19/15 2:58 PM, Reddy Pallavali wrote:
> Michael,
>
> Yes, for every small issue there is no need to contact operator except,
> if it is a complex one.
> Here, also there is a need of consensus, if the system is fully
> distributed and asynchronous need to know which object is crashed, how
> to schedule the loads and coverage issues as well.
>
> On Mon, Oct 19, 2015 at 6:39 PM, Michael Richardson
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>
>     John Strassner <[email-masked] <mailto:[email-masked]>> wrote:
>     need to
>     Michael Richardson <[email-masked]
>     <mailto:[email-masked]>>, Sandelman Software Works
>     [email-masked] <mailto:[email-masked]>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 20 Oct 2015 08:25:05 +1300

> Yes, but just the same, the healing activity needs to be an audited event.

Yes, but logging for later analysis please, not for annoying the operator.
Also, in the ideal world, the system should be capable of alerting the
operators  when a large number of healing events have occurred in the same
geographical area, or when a particular component is involved in multiple
events.

> The ACP should be involved in capturing these events.

The ACP? It's just a communication layer. I'm thinking we would model fault
monitoring as an ASA that collects fault data for logging and analysis.
We can model fault notifications as GRASP synchronization objectives that
look like ["notify",..,<some data structure describing the fault>].
Any ASA that wanted to report a fault would simply synch it to any ASA
that wanted to log it.

(OK, I am trying to verify that the GRASP hammer can be used for this
particular nail.)

> function, the performance is different.

Indeed. But analysing that is a separate function from healing and reporting.

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 20 Oct 2015 08:35:24 +1300

On 20/10/2015 08:04, Joel M. Halpern wrote:
> There are at least two different factors to consider in terms of what events are communicated to the operator.
> 
> First, for initial deployment of autonomic technologies, operators will want ways to observe all the events so they can gain
> confidence in the technology.  (We received these requests with existing self-organizing network technologies.)
> 
> Secondly, different operators have different degrees of information harvesting and display relative to the state of the
> network.  Even if the network is taking care of itself perfectly, some operators will want visibility to any internal state
> change.  These operators currently monitor their routing protocol state for similar effects.

What I get from this is a requirement that all ASAs are capable of notifying events,
but whether and how those events are reported live to operators is a separate issue.

(However, I have never forgotten that at Three Mile Island, the system was so busy
reporting minor alarms that it took hours before the underlying alarm message
saying what was really wrong got printed. TMI, indeed.)

   Brian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 19 Oct 2015 15:44:02 -0400

Agreed that being able to turn off notifications is as important as 
being able to turn them on.

Yours,
Joel

---

From: Michael Richardson <[email-masked]>
Date: Mon, 19 Oct 2015 15:55:16 -0400

> monitoring as an ASA that collects fault data for logging and analysis.

The audit events need to use the ACP addresses.
The ACP is replacing "craft" console terminals which normally would be
plugged into serial console servers that capture all the traffic to a log
file.

The ACP has to be used because it is self-healing and is "out-of-band", and
the event may well be about a failure of the primary communication layer.
Note that I said to use the ACP *addresses* --- given primary communication
layer routing of those addresses, and things like MPTCP, it could well be
that the traffic travels over the primary links.  I also take your Three-Mile
Island comment seriously:  the ACP may well have to provide some different
qualities of service, some way to reserve bandwidth, and if implemented as
tunnels, may have to request/reserve bandwidth for itself.

    > (OK, I am trying to verify that the GRASP hammer can be used for this

I don't really care what the communication protocol is, and neither should
the ACP.  The ACP is, as you said, a communication layer.

GRASP is a protocol, and other than finding the right end-point for the
auditable event logger (and perhaps the format/protocol to use), I don't
think GRASP should be involved.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 20 Oct 2015 13:30:54 +1300

> The audit events need to use the ACP addresses.

Yes, but it also needs device IDs I think. Remember that ACP addresses
are constructed autonomically and don't necessarily map to physical
topology in the same way as data plane addresses. And during a major
outage they might be ephemeral.

> think GRASP should be involved.

I am concerned that a lowest common denominator ASA written by a junior
programmer in a high level language supports event logging. So I do see a
case for making GRASP a possible carrier for event logging. This certainly
doesn't exclude using existing event logging for any case where it's
applicable.

  Brian

---

From: John Strassner <[email-masked]>
Date: Mon, 19 Oct 2015 19:35:44 -0700

Hi Michael,

agreed. See my answer to Jason. Auditing is critical. I'd
just like to see this done more intelligently than the
current approach of flooding everything.

regards,
John

-- 
regards,
John

---

From: Duzongpeng <[email-masked]>
Date: Tue, 20 Oct 2015 03:26:17 +0000

Hi Mehmet：

         IMO, service agents in autonomic network can get network information from many sources. ACP is just one of the sources. As ACP is now working at the IP layer, it can of course provide information at this level.

We should not assume the service agents can not get information from other sources. ACP is a control plane which has priority to be used if possible in the autonomic network. But if it can be proved reasonable, some service agents can also use other information sources to help make decision without depending on ACP.

For example, if FM or PM action in a lower layer is implemented, perhaps the IP layer even does not notice it. However, related service agents in the autonomic network should be involved in this situation.

Whether it must be reported to the administrator is uncertain, but if the administrator wants to know, there is must be some means to access the log of the event.

Best Regards
Zongpeng Du

From: Anima [mailto:[email-masked]] On Behalf Of John Strassner
Sent: Tuesday, October 20, 2015 10:36 AM
To: Michael Richardson
Cc: Anima WG
Subject: Re: [Anima] [ANIMA] Self-Managed Networks

Hi Michael,

regards,
John

On Mon, Oct 19, 2015 at 10:39 AM, Michael Richardson <[email-masked]<mailto:[email-masked]>> wrote:

--
Michael Richardson <[email-masked]<mailto:[email-masked]>>, Sandelman Software Works
 -= IPv6 IoT consulting =-

_______________________________________________
Anima mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/anima

--
regards,
John

---

From: John Strassner <[email-masked]>
Date: Tue, 20 Oct 2015 08:48:05 -0700

Hi all,

Sorry, I'm a bit confused.

To me, this thread is starting to discuss generic messaging system
capabilities (e.g., at the minimum, a message broker, at the other
end of the scale, a full-blown ESB). I didn't think that the purpose
of the ACP included such capabilities.

In other words, there is a huge difference between GRASP
generating a notification and GRASP functioning as a full
messaging system, complete with message transformation and
routing, queuing, etc.

I was assuming the former, not the latter.

Please correct me if I am wrong.

(Note that FOCALE used an ESB for inter-agent communication.)

regards,
John

On Mon, Oct 19, 2015 at 12:35 PM, Brian E Carpenter <
[email-masked]> wrote:

> want ways to observe all the events so they can gain
> harvesting and display relative to the state of the
> > network.  Even if the network is taking care of itself perfectly, some
> operators will want visibility to any internal state

-- 
regards,
John

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 20 Oct 2015 08:53:28 -0700

What do folks on the list think would be typical/important fault events and
healing actions ? Especially actions of "self-healing" that today are not
done, but could be done with better fault management ?

Me:

Link failure. self-healing: re-routing. But no good idea of additional self-healing
that would be enabled by autonomic. Fuzzy feeling that there can be a lot of
services whose parameters might need adjustment under reduced network capacity
though. Just no simple example workflow right now.

Is overloading capacity a fault ? Eg: Too many BGP routes injected, too many
subscribres attaching to wireless ? Are actions like admission control (stop
after threshold) fault management ? Eg: i am not sure about the scope of the
terminology "fault management". 

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 20 Oct 2015 09:17:11 -0700

The way i see it:

Before anima, every distributed function we designed in the IETF
used its own reinvented communication paradigm. The only common
mechanism was use of IP packets, but not even that was the lowest common
denominator (IS-IS anybody).  

So the question really is what are the pragmatic common communication
mechanisms we can come up with to make future distributed functions
easier to implement, deploy and operate. With focus on "deploy" because
thats what we want to be autonomic. And start with what our own "anima"
distributed functions realized via "agents" need.

We know p2p with common presentation layer and set of session transaction
types, and some limited discovery, eg: mp2mp transactions.

The discussion now is what's logically next. Once we have an idea, then
we can figure whether it makes sense to call/put-into GRASP and/or if/how
to deal with it in ANIMA.

-- 
---
Toerless Eckert, [email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 20 Oct 2015 13:09:48 -0400

While i think we should be cautious about attepting to boil the ocean, I 
certainly prefer GRASP plus good distribution design as an information 
distribution mechanism rather than BGP.

Yours,
Joel

---

From: "Jason Coleman (colemaj)" <[email-masked]>
Date: Tue, 20 Oct 2015 18:25:43 +0000

Taking a step back; why are we defining what messages AN will or will not send?
Is there a reason that we simply leave this to the implementation of the code itself?

My thought is that at most we could right what SHOULD, COULD, or MAY but not MUST when it comes to messages.

-- 
Jason Coleman

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 21 Oct 2015 07:56:25 +1300

> Please correct me if I am wrong.

No, you're right. However, one property of the ACP itself is that
it will provide a secure VPN amongst all autonomic nodes and that
means that in some sense it can be used for anything at all.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 21 Oct 2015 08:05:42 +1300

> Is there a reason that we simply leave this to the implementation of the code itself?

Well, there are some general considerations like loop prevention, and functionality
that all ASAs must support. The actual semantics for managing a given parameter
of course have to be specific.

> My thought is that at most we could right what SHOULD, COULD, or MAY but not MUST when it comes to messages.

The devil is in the details. For example GRASP currently says that every objective
MUST carry a loop count and that when it hits zero the negotiation MUST end.
On the other hand it says that the data format of the objective MAY be whatever
you want.

   Brian

---

From: "Jason Coleman (colemaj)" <[email-masked]>
Date: Tue, 20 Oct 2015 19:45:50 +0000

>>JMC>> I agree that we need to define the messaging that is required for GRASP and ACP do function properly.
My comments are more directed toward the conversation about messaging that we plan to send forward to a operator or management platform/center.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 30 Oct 2015 10:12:48 -0400

Joel M. Halpern <[email-masked]> wrote:
    > While i think we should be cautious about attepting to boil the ocean,
    > I certainly prefer GRASP plus good distribution design as an
    > information distribution mechanism rather than BGP.

Is "BGP" a typo? Did you mean ACP?

If so, are you suggesting that we shouldn't have an IP-based ACP, but
should rather run things on top of GRASP?

If GRASP is going to do all these things, such as prioritzation of events,
then wouldn't we be better off to start some place like MMTQ or that other
protocol that starts with D, which I can't recall, sitting here on the
airplane.  (But I'll reply once I look it up)

-- 
Michael Richardson
-on the road-

---

From: Michael Richardson <[email-masked]>
Date: Fri, 30 Oct 2015 09:59:03 -0400

Toerless Eckert <[email-masked]> wrote:
    > Before anima, every distributed function we designed in the IETF used
    > its own reinvented communication paradigm. The only common mechanism
    > was use of IP packets, but not even that was the lowest common

Yes, but at this point, we have a huge number of already invented (and coded,
and deployed) protocols which run across IP.  So, all we need to do is make
sure that we can move IP packets during an outage and during initial
deployment... that's why we have an ACP that routes IP packets, not GRASP.

If we run everything across GRASP, then we don't need routing, numbering or
connectivity.  We can run everything over link-local, or even across LLDP or
ethernet. 

I think that the ASA and Intent negotiation and subsequent implementation is
going to be difficult enough as it is.  I don't see the value in making GRASP
into a kitchen sink.

-- 
Michael Richardson
-on the road-

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 31 Oct 2015 08:16:21 +1300

> deployment... that's why we have an ACP that routes IP packets, not GRASP.

Sure. GRASP provides formats and a state machine for certain types of
interactions that we think will be generally useful to support autonomic
processes. But when a particular interaction doesn't map well into GRASP,
the ACP will provide a secure IP service among trusted nodes, so you can do
anything else that you need to.

> going to be difficult enough as it is.  I don't see the value in making GRASP

No. But on the other hand, when we identify a class of interaction that seems to
be needed by all or most ASAs, we might choose to support it.

   Brian