Subject: [Supa] FW: [netmod] "00" Notification for draft-voit-netmod-peer-mount-requirements

From: "Eric Voit (evoit)" <[email-masked]>
Date: Fri, 26 Sep 2014 13:31:31 +0000

I just learned about SUPA, and based on the evolving charter it seems quite relevant to an informational draft several of us submitted to Netmod yesterday.
http://www.ietf.org/internet-drafts/draft-voit-netmod-peer-mount-requirements-00.txt

Our draft aggregates requirements we have uncovered as we have built several new Data Center edge services upon an Open Daylight controller.  We already have a working YANG service model for two of the two services in the draft.  If there is interest here, I could publish that model as well.

Thanks,
Eric

-----Original Message-----
From: netmod [mailto:[email-masked]] On Behalf Of Eric Voit, September 25, 2014 6:38 PM

We have been developing SDN applications which require quick, iterative access to objects representing network and device abstractions.  These objects include ones authoritatively owned by a local system, as well as ones provided from remote datastores.  Two such applications we have prototyped are “Cloud Policer” and “DDoS Thresholding”.  (See links below for more on these apps.)

The good news is that we found YANG models and syntax to be a nice fit for these applications.   However we have been trying to cover several missing elements:

Missing Element #1: Developer Simplicity We wanted applications to access remote objects from a single datastore, without having to have the applications peer directly with remote devices.  We also wanted clients and applications to access datastores in the same way independent of whether the applications were hosted on a Network Element or Controller.

Missing Element #2: Who owns an Object?
Applications can't tell which Network Element across a set of devices is the authoritative owner of a particular replicated object.  Since replicated data gets out of synch, resolution / convergence is a problem.

Missing Element #3: Maintaining caches of remote information Applications need timely access to remote information, which can necessitate the introduction cache infrastructure.  Netconf provides little support to facilitate this.  For example, notifications are not a good match to subscribe to push-updates.  

To solve these problems, we need technology solutions which work equally well for both Controllers and Network Elements.  These solutions must expose local datastores which provide authoritative, consolidated views of distributed network objects.  This must be done in a way which allows the applications to sit anywhere in the network, on various abstractions, while maintaining domain consistency. And this must be done while maximizing application developer simplicity.  

This draft is intended to start aggregating requirements in this space.  The draft is not complete, so it would be great to hear from others who are interested in helping so that we end up with a more comprehensive context.

Solutions do exist which can solve a subset of these requirements.  One particular technology we have built is known as “peer mount”.  Peer mount gives the ability to reference YANG information into a local datastore from remote datastores.  This provides client applications with local access to information which might be transparently federated across multiple systems and datastores.  It also retains clear object ownership to minimize synchronization and inter-system dependency issues.   There are already implementations of this technology, for example, in Open Daylight.  However, we believe requirements and use cases for this technology apply equally well to network devices.  

Please let us know if either the requirements or technology specifications are problem you are interested in exploring.

Thanks!
Eric, Alex, Prabhakara, Ambika, Shashi

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Thursday, September 25, 2014 8:41 AM
To: Prabhakara Yellai (pyellai); Shashi Kumar Bansal (shabansa); Ambika Prasad Tripathy (ambtripa); Alexander Clemm (alex); Eric Voit (evoit); Ambika Prasad Tripathy (ambtripa); Alexander Clemm (alex); Prabhakara Yellai (pyellai); Eric Voit (evoit); Shashi Kumar Bansal (shabansa)
Subject: New Version Notification for draft-voit-netmod-peer-mount-requirements-00.txt

A new version of I-D, draft-voit-netmod-peer-mount-requirements-00.txt
has been successfully submitted by Eric Voit and posted to the IETF repository.

Name:                  draft-voit-netmod-peer-mount-requirements
Revision:              00
Title:                     Requirements for Peer Mounting of YANG subtrees from Remote Datastores
Document date: 2014-09-25
Group:                  Individual Submission
Pages:                  22
URL:            http://www.ietf.org/internet-drafts/draft-voit-netmod-peer-mount-requirements-00.txt
Status:         https://datatracker.ietf.org/doc/draft-voit-netmod-peer-mount-requirements/
Htmlized:       http://tools.ietf.org/html/draft-voit-netmod-peer-mount-requirements-00

Abstract:
   Network integrated applications want simple ways to access YANG
   objects and subtrees which might be distributed across network.
   Performance requirements may dictate that it is unaffordable for a
   subset of these applications to go through existing centralized
   management brokers.  For such applications, development complexity
   must be minimized.  Specific aspects of complexity developers want to
   ignore include:

   o  whether authoritative information is actually sourced from remote
      datastores (as well as how to get to those datastores),

   o  whether such information has been locally cached or not,

   o  whether there are zero, one, or more controllers asserting
      ownership of information, and

   o  whether there are interactions with other applications
      concurrently running elsewhere

   The solution requirements described in this document detail what is
   needed to support application access to authoritative network YANG
   objects from controllers (star) or peering network devices (mesh) in
   such a way to meet these goals.

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Georgios Karagiannis <[email-masked]>
Date: Fri, 26 Sep 2014 13:34:52 +0000

Hi Eric,

Thanks for the information.
I have not read your draft in detail, but from your explanation, it might be relevant to SUPA.

Best regards,
Georgios

> From: Supa [mailto:[email-masked]] On Behalf Of Eric Voit (evoit)
> Sent: Friday, September 26, 2014 2:32 PM
> To: [email-masked]
> Cc: Alexander Clemm (alex)
> Subject: [Supa] FW: [netmod] "00" Notification for draft-voit-netmod-peer-
> mount-requirements
> 
> I just learned about SUPA, and based on the evolving charter it seems quite
> relevant to an informational draft several of us submitted to Netmod yesterday.
> Our draft aggregates requirements we have uncovered as we have built several
> new Data Center edge services upon an Open Daylight controller.  We already
> have a working YANG service model for two of the two services in the draft.
> If there is interest here, I could publish that model as well.
> 
> Thanks,
> Eric
> 
> We have been developing SDN applications which require quick, iterative access
> to objects representing network and device abstractions.  These objects
> include ones authoritatively owned by a local system, as well as ones provided
> from remote datastores.  Two such applications we have prototyped are “Cloud
> Policer” and “DDoS Thresholding”.  (See links below for more on these apps.)
> 
> The good news is that we found YANG models and syntax to be a nice fit for
> these applications.   However we have been trying to cover several missing
> elements:
> 
> Missing Element #1: Developer Simplicity We wanted applications to access
> remote objects from a single datastore, without having to have the
> applications peer directly with remote devices.  We also wanted clients and
> applications to access datastores in the same way independent of whether the
> applications were hosted on a Network Element or Controller.
> 
> Applications can't tell which Network Element across a set of devices is the
> authoritative owner of a particular replicated object.  Since replicated data
> gets out of synch, resolution / convergence is a problem.
> 
> Missing Element #3: Maintaining caches of remote information Applications need
> timely access to remote information, which can necessitate the introduction
> cache infrastructure.  Netconf provides little support to facilitate
> this.  For example, notifications are not a good match to subscribe to push-
> updates.
> 
> To solve these problems, we need technology solutions which work equally well
> for both Controllers and Network Elements.  These solutions must expose local
> datastores which provide authoritative, consolidated views of distributed
> network objects.  This must be done in a way which allows the applications to
> sit anywhere in the network, on various abstractions, while maintaining domain
> consistency. And this must be done while maximizing application developer
> simplicity.
> 
> This draft is intended to start aggregating requirements in this space.  The
> draft is not complete, so it would be great to hear from others who are
> interested in helping so that we end up with a more comprehensive context.
> 
> Solutions do exist which can solve a subset of these requirements.  One
> particular technology we have built is known as “peer mount”.  Peer mount
> gives the ability to reference YANG information into a local datastore from
> remote datastores.  This provides client applications with local access to
> information which might be transparently federated across multiple systems and
> datastores.  It also retains clear object ownership to minimize
> synchronization and inter-system dependency issues.   There are already
> implementations of this technology, for example, in Open Daylight.  However,
> we believe requirements and use cases for this technology apply equally well
> to network devices.
> 
> To: Prabhakara Yellai (pyellai); Shashi Kumar Bansal (shabansa); Ambika Prasad
> Tripathy (ambtripa); Alexander Clemm (alex); Eric Voit (evoit); Ambika Prasad
> Tripathy (ambtripa); Alexander Clemm (alex); Prabhakara Yellai (pyellai); Eric
> Voit (evoit); Shashi Kumar Bansal (shabansa)
> Title:                     Requirements for Peer Mounting of YANG subtrees
> from Remote Datastores Document date: 2014-09-25
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
> 
> Supa mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/supa

---

From: Tina TSOU <[email-masked]>
Date: Sun, 28 Sep 2014 01:38:04 +0000

Dear Eric et al,

Distributed Data Center services are typical case for SUPA.

Your submission of drafts are very welcome.

You may be interested to look at these.
draft-cheng-supa-ddc-use-cases-00<http://datatracker.ietf.org/doc/draft-cheng-supa-ddc-use-cases/>

Use Cases for Distributed Data Center Applications in SUPA

2014-09-17
new<http://www.ietf.org/rfcdiff?url2=draft-cheng-supa-ddc-use-cases-00>

I-D Exists

draft-contreras-supa-yang-network-topo-00<http://datatracker.ietf.org/doc/draft-contreras-supa-yang-network-topo/>

A YANG Data Model for Network Topologies

2014-09-18
new<http://www.ietf.org/rfcdiff?url2=draft-contreras-supa-yang-network-topo-00>

I-D Exists

draft-zaalouk-supa-configuration-model-00<http://datatracker.ietf.org/doc/draft-zaalouk-supa-configuration-model/>

YANG Data Model for Configuration of Shared Unified Policy Automation (SUPA)

2014-09-22
new<http://www.ietf.org/rfcdiff?url2=draft-zaalouk-supa-configuration-model-00>

I-D Exists

draft-pentikousis-supa-mapping-00<http://datatracker.ietf.org/doc/draft-pentikousis-supa-mapping/>

SUPA Configuration and Policy Mapping

2014-09-23
new<http://www.ietf.org/rfcdiff?url2=draft-pentikousis-supa-mapping-00>

I-D Exists

Thank you,

Tina

-----Original Message-----
From: Supa [mailto:[email-masked]] On Behalf Of Georgios Karagiannis
Sent: Friday, September 26, 2014 9:35 PM
To: Eric Voit (evoit); [email-masked]
Cc: Alexander Clemm (alex)
Subject: Re: [Supa] [netmod] "00" Notification for draft-voit-netmod-peer-mount-requirements

Hi Eric,

Best regards,

Georgios

> From: Supa [mailto:[email-masked]] On Behalf Of Eric Voit

> (evoit)

> To: [email-masked]<mailto:[email-masked]>

> Subject: [Supa] FW: [netmod] "00" Notification for

> draft-voit-netmod-peer- mount-requirements

> I just learned about SUPA, and based on the evolving charter it seems

> quite relevant to an informational draft several of us submitted to Netmod yesterday.

> http://www.ietf.org/internet-drafts/draft-voit-netmod-peer-mount-requi

> rements-

> 00.txt

> Our draft aggregates requirements we have uncovered as we have built

> several new Data Center edge services upon an Open Daylight

> controller.  We already have a working YANG service model for two of the two services in the draft.

> From: netmod [mailto:[email-masked]] On Behalf Of Eric Voit,

> September 25, 2014 6:38 PM

> We have been developing SDN applications which require quick,

> iterative access to objects representing network and device

> abstractions.  These objects include ones authoritatively owned by a

> local system, as well as ones provided from remote datastores.  Two

> such applications we have prototyped are “Cloud Policer” and “DDoS

> Thresholding”.  (See links below for more on these apps.)

> The good news is that we found YANG models and syntax to be a nice fit

> for these applications.   However we have been trying to cover several

> missing

> Missing Element #1: Developer Simplicity We wanted applications to

> access remote objects from a single datastore, without having to have

> the applications peer directly with remote devices.  We also wanted

> clients and applications to access datastores in the same way

> independent of whether the applications were hosted on a Network Element or Controller.

> Applications can't tell which Network Element across a set of devices

> is the authoritative owner of a particular replicated object.  Since

> replicated data gets out of synch, resolution / convergence is a problem.

> Missing Element #3: Maintaining caches of remote information

> Applications need timely access to remote information, which can

> necessitate the introduction cache infrastructure.  Netconf provides

> little support to facilitate this.  For example, notifications are not

> a good match to subscribe to push- updates.

> To solve these problems, we need technology solutions which work

> equally well for both Controllers and Network Elements.  These

> solutions must expose local datastores which provide authoritative,

> consolidated views of distributed network objects.  This must be done

> in a way which allows the applications to sit anywhere in the network,

> on various abstractions, while maintaining domain consistency. And

> this must be done while maximizing application developer simplicity.

> This draft is intended to start aggregating requirements in this

> space.  The draft is not complete, so it would be great to hear from

> others who are interested in helping so that we end up with a more comprehensive context.

> Solutions do exist which can solve a subset of these requirements.

> One particular technology we have built is known as “peer mount”.

> Peer mount gives the ability to reference YANG information into a

> local datastore from remote datastores.  This provides client

> applications with local access to information which might be

> transparently federated across multiple systems and datastores.  It

> also retains clear object ownership to minimize synchronization and

> inter-system dependency issues.   There are already implementations of

> this technology, for example, in Open Daylight.  However, we believe

> requirements and use cases for this technology apply equally well to network devices.

> Please let us know if either the requirements or technology

> specifications are problem you are interested in exploring.

> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]

> To: Prabhakara Yellai (pyellai); Shashi Kumar Bansal (shabansa);

> Ambika Prasad Tripathy (ambtripa); Alexander Clemm (alex); Eric Voit

> (evoit); Ambika Prasad Tripathy (ambtripa); Alexander Clemm (alex);

> Prabhakara Yellai (pyellai); Eric Voit (evoit); Shashi Kumar Bansal

> (shabansa)

> Subject: New Version Notification for draft-voit-netmod-peer-mount-

> requirements-00.txt

> Title:                     Requirements for Peer Mounting of YANG

> subtrees from Remote Datastores Document date: 2014-09-25

> URL:

> http://www.ietf.org/internet-drafts/draft-voit-netmod-peer-

> mount-requirements-00.txt

> Status:

> https://datatracker.ietf.org/doc/draft-voit-netmod-peer-mount-

> requirements/

> Htmlized:

> http://tools.ietf.org/html/draft-voit-netmod-peer-mount-

> requirements-00

>    must be minimized.  Specific aspects of complexity developers want

> to

>    o  whether authoritative information is actually sourced from

> remote

>    objects from controllers (star) or peering network devices (mesh)

> in

> Please note that it may take a couple of minutes from the time of

> submission until the htmlized version and diff are available at tools.ietf.org.

> [email-masked]<mailto:[email-masked]>

> [email-masked]<mailto:[email-masked]>

Supa mailing list

[email-masked]<mailto:[email-masked]>

---

From: Tina TSOU <[email-masked]>
Date: Mon, 29 Sep 2014 09:49:35 +0000

Dear Eric et al,

Now I read draft-voit-netmod-peer-mount-requirements, it is very good at describing the requirements.

In this draft, two optional architectures were proposed: a centralized one similar to SUPA and a distributed one which is also compatible with SUPA (with a little extension of SUPA charter and scope, would you please review the charter of SUPA and provide some text for modification?).

One key requirement of SUPA is network-wide modeling for standardizing. In your work, is there also a similar requirement? I mean, do you need the model between app and network element be standardized?

Have you considered the mapping mechanism from network-wide model to device level configuration? This is part of the work of SUPA.

I would agree that there are many benefits of using distributed solution, however, there is one key issue: if the decisions or actions from different network elements are difference and incompatible, how to avoid the fluctuation? IMHO, this may be solved by slicing the resource or using multiple instances from virtualization? Or using the consistence check of distributed database?

Some words from this draft may imply the requirement for extending the existing NETCONF. But I’m not sure. Do you really want that?

Hope it helps.

Thank you,
Tina

From: Supa [mailto:[email-masked]] On Behalf Of Tina TSOU
Sent: Sunday, September 28, 2014 9:38 AM
To: Georgios Karagiannis; Eric Voit (evoit); [email-masked]
Cc: Alexander Clemm (alex)
Subject: Re: [Supa] [netmod] "00" Notification for draft-voit-netmod-peer-mount-requirements

Dear Eric et al,

2014-09-17

I-D Exists

2014-09-18

I-D Exists

2014-09-22

I-D Exists

2014-09-23

I-D Exists

Thank you,

Tina

-----Original Message-----
From: Supa [mailto:[email-masked]] On Behalf Of Georgios Karagiannis
Sent: Friday, September 26, 2014 9:35 PM
To: Eric Voit (evoit); [email-masked]<mailto:[email-masked]>
Cc: Alexander Clemm (alex)
Subject: Re: [Supa] [netmod] "00" Notification for draft-voit-netmod-peer-mount-requirements

Hi Eric,

Best regards,

Georgios

Supa mailing list

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Mon, 29 Sep 2014 14:10:32 +0000

>From: Tina TSOU, September 29, 2014 5:50 AM
>
>Dear Eric et al,
>
>In this draft, two optional architectures were proposed: a centralized one 
>similar to SUPA and a distributed one which is also compatible with SUPA 
>(with a little extension of SUPA charter and scope, would you please review 
>the charter of SUPA and provide some text for modification?).

Will do in a separate response.

>One key requirement of SUPA is network-wide modeling for standardizing. 
>In your work, is there also a similar requirement?

From an application standpoint, I expect that there will be many network wide 
service models which match to applications.  Since applications will always
want their minimal management model, the trick will be to instrument the
interfaces for easy meshing with places like OpenStack, OPNFV, OpenDaylight.

IMHO, it is premature to start standardizing many (most?) application models 
Northbound of a controller.  We don't know which services will be successful.  
We don't know enough about key scale or operational considerations.

>I mean, do you need the model between app and network element be 
>standardized?

No.  I want to work with existing device models.  However if a multi-network 
element application sits in an container which is co-resident on/near the 
network element, then this could be standardized.  

>Have you considered the mapping mechanism from network-wide model 
>to device level configuration? This is part of the work of SUPA.

Yes.

>I would agree that there are many benefits of using distributed solution, 
>however, there is one key issue: if the decisions or actions from different 
>network elements are difference and incompatible, how to avoid the 
>fluctuation? 

This is the most interesting part of the problem.   One benefit of our "Mount" 
approach is that authoritative objects are accessible anywhere in a domain.  
- "peer mount" infrastructure is agnostic to where such check occur
- changes in state can be monitored on remote devices via subscription
- distributed objects previous inaccessible can now be compared
- consistency applets (driven by these triggers) can be placed anywhere

From there it is local business logic which determines what to do if 
different authoritative objects end up being incompatible.  Often it is 
possible to mechanize the resolution.  

This is exactly why we are pushing "Eventual Consistency".  Routing 
Protocols and Amazon.com's shopping cart system have addressed these 
problems extremely well.  It is time to apply these solutions to general 
networking policy.  Even multiple controller environments can be supported.

>IMHO, this may be solved by slicing the resource or using 
>multiple instances from virtualization? Or using the consistence check of 
>distributed database?
>
>Some words from this draft may imply the requirement for extending the 
>existing NETCONF. But I’m not sure. Do you really want that?

What I want is the standards track for a new version of 
draft-clemm-netmod-mount.   The version should be out soon.

Eric 

>draft-cheng-supa-ddc-use-cases-00 
>new
I-D Exists 

draft-contreras-supa-yang-network-topo-00 
A YANG Data Model for Network Topologies
2014-09-18 
>new
I-D Exists 

draft-zaalouk-supa-configuration-model-00 
YANG Data Model for Configuration of Shared Unified Policy Automation (SUPA)
2014-09-22 
>new
I-D Exists 
draft-pentikousis-supa-mapping-00 
SUPA Configuration and Policy Mapping
2014-09-23 
>new
I-D Exists

---

From: Tina TSOU <[email-masked]>
Date: Tue, 30 Sep 2014 11:13:16 +0000

Dear Eric,

In line...

Thank you,
Tina

On Sep 29, 2014, at 10:10 PM, "Eric Voit (evoit)" <[email-masked]<mailto:[email-masked]>> wrote:

Dear Eric et al,

Now I read draft-voit-netmod-peer-mount-requirements, it is very good
at describing the requirements.

Got it.

That was very useful as a booster for a converged version of charter.

IMHO, it is premature to start standardizing many (most?) application models
Northbound of a controller.  We don't know which services will be successful.
We don't know enough about key scale or operational considerations.
Agree. Not everything, just a little piece of all the puzzles.

The IESG suggested network management services, especially distributed data center services, before Toronto.

There are a few solid use cases, and you also brought "mount" use case.
https://datatracker.ietf.org/doc/draft-cheng-supa-ddc-use-cases/
https://datatracker.ietf.org/doc/draft-sun-supa-openv6-use-cases/

Openstack Neutron is mostly interested at the L3VPN service NBIs.
https://datatracker.ietf.org/doc/draft-zaalouk-supa-configuration-model/
is a starting point for virtual links.

Look forward to the models you mentioned earlier.

It can be stand alone side by side, or converge with others.

No.  I want to work with existing device models.  However if a multi-network
element application sits in an container which is co-resident on/near the
network element, then this could be standardized.
Thank you for considering the popular scenarios.

Yes.
IMHO, https://datatracker.ietf.org/doc/draft-pentikousis-supa-mapping/ needs to consider "mount" case in later version.

This is exactly why we are pushing "Eventual Consistency".  Routing
Protocols and Amazon.com<http://Amazon.com>'s shopping cart system have addressed these
problems extremely well.  It is time to apply these solutions to general
networking policy.  Even multiple controller environments can be supported.
Exactly, I lean more to the Amazon and Tencent like approaches.

That's what I mean below.

What I want is the standards track for a new version of
draft-clemm-netmod-mount.   The version should be out soon.
Look forward to the new version.

If there is potential change to Netconf, should that part be sent to the Netconf WG?

Eric

Hope it helps.

Thank you,
Tina

To: Georgios Karagiannis; Eric Voit (evoit); [email-masked]<mailto:[email-masked]>

Dear Eric et al,

Thank you,
Tina

-----Original Message-----
From: Supa [mailto:[email-masked]] On Behalf Of Georgios Karagiannis
Sent: Friday, September 26, 2014 9:35 PM
To: Eric Voit (evoit); [email-masked]<mailto:[email-masked]>
Cc: Alexander Clemm (alex)
Subject: Re: [Supa] [netmod] "00" Notification for draft-voit-netmod-peer-mount-requirements

Hi Eric,

Best regards,
Georgios

-----Original Message-----

Thanks,
Eric

-----Original Message-----

Thanks!

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]
Sent: Thursday, September 25, 2014 8:41 AM
To: Prabhakara Yellai (pyellai); Shashi Kumar Bansal (shabansa);

Abstract:

Please note that it may take a couple of minutes from the time of
submission until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

The IETF Secretariat