Subject: [Anima] Object store and PubSub model

From: Artur Hecker <[email-masked]>
Date: Wed, 28 Jun 2017 15:45:42 +0000

Dear ANIMA community,

Please allow these two questions, which popped up while following recent group discussions and I-Ds:

1. Would the group consider an ANIMA-integrated data storage to be relevant in the context of the current charter, or, otherwise, does it look like a relevant topic for the planned re-chartering?

Background: The idea for an integrated distributed storage in ANIMA could be motivated e.g. through network-wide configuration, where data objects would become available to all individual nodes and ASAs without relying on any means external to ANIMA (like specialized nodes with specialized ASAs, or similar). As another example, it seems ANIMA nodes might need to store data in the case of e.g. intent distribution, as intents could be conditional and shifted in time, like in "Do this, once this and that have materialized" (see e.g. ECA in SUPA).

2. In a similar vein, does a publish/subscribe information distribution model, built upon an underlying storage, look promising and relevant to the group? Can GRASP support that or will it require extensions? The pub/sub is, in our opinion, a more general information distribution model than what we saw in the current work stream. Yet, before submitting a draft that details such a model, we wanted to kindly ask for your opinion on that. 

Kind regards
artur

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 29 Jun 2017 08:35:09 +1200

> Background: The idea for an integrated distributed storage in ANIMA could be motivated e.g. through network-wide configuration, where data objects would become available to all individual nodes and…

So far, I don't believe we've discussed distributed storage as such. It would be a natural part of the discussion of intent. We have tended to assume that intent originates centrally, so it needs a replication model rather than distributed storage in the general sense. The simplest replication model is of course flooding, which we already have for relatively small data objects. For example, GRASP as it exists today could flood an objective whose value is the URL of the latest intent for the whole network.

I would not advocate developing a distributed storage model specific to Anima. Rather, if we decided it was necessary, IMHO we should adopt an existing solution, since this is not a trivial problem.

> 2. In a similar vein, does a publish/subscribe information distribution model, built upon an underlying storage, look promising and relevant to the group? Can GRASP support that or will it require…

Have you studied draft-liu-anima-grasp-distribution?

But first, what sort of autonomic use cases would require distributed storage or a pub/sub solution?

Regards
    Brian

---

From: Artur Hecker <[email-masked]>
Date: Thu, 29 Jun 2017 15:28:08 +0000

Dear Brian, all

Thanks a lot for the explanations. Please find my answers below:

-----Original Message-----
From: Brian E Carpenter [mailto:[email-masked]] 
Sent: 28 June 2017 22:35
To: Artur Hecker; [email-masked]
Subject: Re: [Anima] Object store and PubSub model

> So far, I don't believe we've discussed distributed storage as such. It would be a natural part of the discussion of 
> intent. We have tended to assume that intent originates centrally, so it needs a replication model rather than 
> distributed storage in the general sense. The simplest replication model is of course flooding, which we 
> already have for relatively small data objects. For example, GRASP as it exists today could flood an objective whose
> value is the URL of the latest intent for the whole network.

[[AH] ] Ok, very clear, thanks. Would you welcome such a discussion e.g. for the re-chartering of ANIMA?

We believe that network-wide storage could be of high value for autonomous networking, just as routing is. The reasons for this are, among others:

c) given the existing ANIMA descriptions, I believe that limited storage is presumed available on all nodes (i.e. at least to store that URL in your example);
b) the APIs for data object storage are well-established and compact (put(), get());
a) the information a node might need to store is not necessarily limited to the scope of that single node. Therefore, an efficient capability to access network-wide information (e.g. configuration data, network-wide intents, network-wide averages or aggregates of data) would really be of a value. If every application has to redo it, we run the risk of repetitive wheel reinventions - or flooding. Essentially, as you just hinted, if we do not provide these means (either by a mandatory ASA or over the ANIMA API), then every developer will have to revert to flooding at least for the first message. While this might be assumed rare for intents, I am not sure we can claim this for every individual application scenario.

> I would not advocate developing a distributed storage model specific to Anima. Rather, if we decided it was
 > necessary, IMHO we should adopt an existing solution, since this is not a trivial problem.

[[AH] ] Agreed, and it was not intended. If there is a perceived consensus of the value of this, we would rather see how to integrate/adopt a suitable solution.

>> 2. In a similar vein, does a publish/subscribe information distribution model, built upon an underlying storage, 
>> look promising and relevant to the group? Can GRASP support that or will it require extensions? The pub/sub is, 
>> in our opinion, a more general information distribution model than what we saw in the current work stream. Yet,
>> before submitting a draft that details such a model, we wanted to kindly ask for your opinion on that.

> Have you studied draft-liu-anima-grasp-distribution?
[[AH] ] Yes, that was where our motivation came from. It seems to be based on the flooding/replication model. Which is fine for some scenarios, but which also has known limitations. The idea was to allow for a more general model, considered to be better for scalability, etc.

> But first, what sort of autonomic use cases would require distributed storage or a pub/sub solution?

[[AH] ] Any example under a) above or our original example of ECA type of intent, which would require to "ripen" before being executed. The exact condition could be something that an individual node might not know, e.g. "close the blinds when the temperature in the building exceeds 25 C (=77F)".

Regards
artur