Subject: [i2rs] WG LC for Topology (10/1 to 10/14)

From: "Susan Hares" <[email-masked]>
Date: Thu, 1 Oct 2015 18:41:34 -0400

This begins a 2 week WG Last call (10/1 to 10/14) 

draft-ietf-yang-network-topo - modeling draft 

http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-network-topo/

draft-ietf-i2rs-yang-l3-topology - L3 topology 

http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-l3-topology/

draft-ietf-i2rs-yang-l2-topology - L2 topology 

http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-l2-network-topology/

Implementation status: 

This an OpenDaylight (ODL) implementation of the L3 topology and general
model.  It is likely the L2 topology model will be supported in future ODL
implementations.   Jeff and I would appreciate anyone who has
implementations of these topology models to provide details on list or
offlist to the chairs. 

Sue Hares

---

From: Xufeng Liu <[email-masked]>
Date: Thu, 8 Oct 2015 20:15:08 +0000

Hi Folks,

The following is an update on the ongoing I2RS<=>TE Topology alignment discussion (between the authors of the TE Topology model and Alex).

The issues / solutions that are currently being discussed are:

I2RS Generic Network Topology Model (draft-ietf-yang-network-topo):

1.      Some groupings in ietf-network.yang and ietf-network-topology.yang cannot be used in augmenting module because of missing name spaces, such as "path "/network/network-id" at line 42 of ietf-network.yang.
Solution: Proposed fixes have been sent to Alex to verify.

2.      "leaf network-ref" in ietf-network-topology.yang:169, 220, is causing pyang validation errors.
Solution: Alex will check the errors.

3.      Can the group of schedule attributes be moved from ietf-te-topology.yang to ietf-network-topology.yang?
Solution: We agreed to keep them in ietf-te-topology.yang.

4.      How to support state branch in augmenting module?
Solution: Keep base model ietf-network-topology.yang as is. In the augmenting module, for each entity like topology, node and link, create a config container for configuration attributes and a state container for operational state attributes.

5.      Should "server-provided" flag be used to block user operation on read-only entities?
Solution: Keep base model ietf-network-topology.yang as is. TE topology will use other means to achieve such a purpose.

6.      Should I2RS Generic Network Topology model have a top-level container? The benefit of doing so is to provide an augmentation target node to define attributes global to all networks.
Solution: I2RS Generic Network Topology module authors will consider making "/networks" as the top level container.

L3 Topology Model (draft-ietf-i2rs-yang-l3-topology)

1.      L3 Topology Model should have references to TE Topology model, so that the related TE information can be properly retrieved, when L3 topology and TE topology are congruent.
Solution: L3 Topology Model authors will need to update the model and draft to include these.

L2 Topology Model (draft-ietf-i2rs-yang-l2-topology)

1.      L2 Topology Model should have references to TE Topology model, so that the related TE information can be properly retrieved, when L2 topology and TE topology are congruent.
Solution: This needs to be discussed with L2 Topology Model authors.

Regards,
- Xufeng (on behalf of the authors of the TE Topology Model)

From: i2rs [mailto:[email-masked]] On Behalf Of Susan Hares
Sent: Thursday, October 01, 2015 6:42 PM
To: [email-masked]
Cc: 'Jeffrey Haas'; 'Alia Atlas'
Subject: [i2rs] WG LC for Topology (10/1 to 10/14)

Implementation status:

This an OpenDaylight (ODL) implementation of the L3 topology and general model.  It is likely the L2 topology model will be supported in future ODL implementations.   Jeff and I would appreciate anyone who has implementations of these topology models to provide details on list or offlist to the chairs.

Sue Hares

---

From: "Zhangxian (Xian)" <[email-masked]>
Date: Fri, 9 Oct 2015 02:20:19 +0000

Hi, Xufeng, Alex,

Thank you for taking the effort to work toward getting aligned as discussed in last meeting.   I have some comments on some of the conclusions.

Could you share a bit more on reasons behind Point 3 conclusion?

For points 4 and 5, I think they are related. My understanding is that for ietf-network, it does not follow the method taken by YANG modules such as ietf-interface and ietf-te-topology, in which each –rw leaf will have a corresponding –ro leaf. Instead, it choose to use the “server-provided” attribute to say whether all the leafs in the module is configurable or state. I am not sure I understand the conclusion of these two points. Do you agree that both methods are acceptable?

One last comment on point 6: is this for future-proofness or you have already have some attributes in mind that can apply to all networks?

Thank you,
Xian

From: i2rs [mailto:[email-masked]] On Behalf Of Xufeng Liu
Sent: 2015年10月9日 4:15
To: Susan Hares; [email-masked]
Cc: 'Jeffrey Haas'; 'Alia Atlas'
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Folks,

1.      Some groupings in ietf-network.yang and ietf-network-topology.yang cannot be used in augmenting module because of missing name spaces, such as “path "/network/network-id" at line 42 of ietf-network.yang.
Solution: Proposed fixes have been sent to Alex to verify.

2.      “leaf network-ref” in ietf-network-topology.yang:169, 220, is causing pyang validation errors.
Solution: Alex will check the errors.

5.      Should “server-provided” flag be used to block user operation on read-only entities?
Solution: Keep base model ietf-network-topology.yang as is. TE topology will use other means to achieve such a purpose.

6.      Should I2RS Generic Network Topology model have a top-level container? The benefit of doing so is to provide an augmentation target node to define attributes global to all networks.
Solution: I2RS Generic Network Topology module authors will consider making “/networks” as the top level container.

Regards,

draft-ietf-yang-network-topo – modeling draft
http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-network-topo/

draft-ietf-i2rs-yang-l3-topology – L3 topology
http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-l3-topology/

draft-ietf-i2rs-yang-l2-topology – L2 topology
http://datatracker.ietf.org/doc/draft-ietf-i2rs-yang-l2-network-topology/

Implementation status:

Sue Hares

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Oct 2015 09:22:07 +0200

Hi,

there is a discussion on the yang doctors mailing list about the data
model design that mixes state data and configuration data into one
subtree and that uses a data model specific runtime object to identify
what is config data and what is state data. One of the main outcomes
of the IAB workshop back in 2002 was the need to clearly separate
configuration from operational state and this has been driven the
design of NETCONF and YANG. YANG data model validation rules, for
example, make a distinction between config true and config false data.
The config true or false property impacts what is returned by
NETCONF's get-config operation. Also note that config data is not
allowed to refer to config false data in constraints.

It is unclear why the document does not follow the typical design
pattern, namely to have a config true subtree

  /networks/network*

and a config false subtree

  /networks-state/network*

Section 3.5 describes this approach in the 3rd paragraph and states
"As most data is defined in those groupings, the amount of additional
definitions required will be limited." and there are no strong reasons
given why this approach has not been followed.

/js

> _______________________________________________
> i2rs mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i2rs

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Fri, 9 Oct 2015 21:55:19 +0000

Hi Juergen,

which specific objects/ subtrees are you referring to?  

Essentially, in ietf-network and ietf-network-topology, we have all configuration information.  The same is true for the Layer 3 topology - all configuration information.  (I cannot comment on L2 topology as I am not involved there.)  

Are you saying that we should add an additional branch as a placeholder (and perhaps an augmentation target) for operational data, even if we have not otherwise defined any operational data?  

The only item in the topology that is read-only concerns the "server-provided" flag, but this concerns a separate issue that was also discussed (I am not sure if this is what you are referring to).  It is analogous to the other discussion concerning distinguishing configuration that has been intended, versus one that is in effect etc .  This concerns the issue that some topologies are populated by the server whereas some topologies can be populated by client applications.  We have had discussions in the past whether to "split this up", having a (rw) branch to populate "intended" topologies and a (ro) branch for topologies "in effect".  We decided against it for various reasons - every piece of information would essentially be duplicated inside the model (this is not your normal config vs oper data distinction, but would essentially reflect a limitation of the framework), leading to unnecessary additional complexity in the model (every augmentation has to be conducted in two places), more complex validation rules, etc.    

--- Alex

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Friday, October 09, 2015 12:22 AM
To: Susan Hares <[email-masked]>
Cc: [email-masked]; Martin Bjorklund <[email-masked]>; Ladislav Lhotka <[email-masked]>; Andy Bierman <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; 'Alia Atlas' <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi,

there is a discussion on the yang doctors mailing list about the data model design that mixes state data and configuration data into one subtree and that uses a data model specific runtime object to identify what is config data and what is state data. One of the main outcomes of the IAB workshop back in 2002 was the need to clearly separate configuration from operational state and this has been driven the design of NETCONF and YANG. YANG data model validation rules, for example, make a distinction between config true and config false data.
The config true or false property impacts what is returned by NETCONF's get-config operation. Also note that config data is not allowed to refer to config false data in constraints.

It is unclear why the document does not follow the typical design pattern, namely to have a config true subtree

  /networks/network*

  /networks-state/network*

Section 3.5 describes this approach in the 3rd paragraph and states "As most data is defined in those groupings, the amount of additional definitions required will be limited." and there are no strong reasons given why this approach has not been followed.

/js

On Thu, Oct 01, 2015 at 06:41:34PM -0400, Susan Hares wrote:
> This an OpenDaylight (ODL) implementation of the L3 topology and 
> general model.  It is likely the L2 topology model will be supported in future ODL

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Sat, 10 Oct 2015 00:20:48 +0000

Hi Xian,

responses inline, <ALEX>

Thanks
--- Alex

From: i2rs [mailto:[email-masked]] On Behalf Of Zhangxian (Xian)
Sent: Thursday, October 08, 2015 7:20 PM
To: Xufeng Liu <[email-masked]>; Susan Hares <[email-masked]>; [email-masked]
Cc: 'Jeffrey Haas' <[email-masked]>; TEAS WG <[email-masked]>; 'Alia Atlas' <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi, Xufeng, Alex,

<ALEX> The reason is that the schedule stuff, while generic / non-specific to any specific topology, is not “common”, i.e. does not apply to all topologies/ every instantiation of the model.  The goal is to have ietf-network-topology refer to the stuff that is truly common.
</ALEX>

For points 4 and 5, I think they are related. My understanding is that for ietf-network, it does not follow the method taken by YANG modules such as ietf-interface and ietf-te-topology, in which each -rw leaf will have a corresponding -ro leaf. Instead, it choose to use the “server-provided” attribute to say whether all the leafs in the module is configurable or state. I am not sure I understand the conclusion of these two points. Do you agree that both methods are acceptable?

<ALEX> There is no operational data defined.  The server-provided attributed guides whether network topology information is populated by the server or by the client application, but the information is the same - this not the config vs oper data (read: stats) separation.  To add stats, you should provide an additional subtree/branch as applicable when you are augmenting.
</ALEX>

<ALEX>
I’ll let Xufeng respond to that one.  We did not have a top-level container because it keeps paths shorter (one less level in the hierarchy) and because if you wanted to insert something at the “top level”, you could always do it in parallel.  I still think this is the design that’s preferable.
That said, having a top-level container may not really hurt.  There were concerns regarding the ramification to existing implementations of the model (notably, Open Daylight) if we were to add it but it seems the fallout would be manageable.

Thanks, Alex (end of my comments)
</ALEX>

Thank you,
Xian

From: i2rs [mailto:[email-masked]] On Behalf Of Xufeng Liu
Sent: 2015年10月9日 4:15
To: Susan Hares; [email-masked]<mailto:[email-masked]>
Cc: 'Jeffrey Haas'; 'Alia Atlas'
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Folks,

Regards,

To: [email-masked]<mailto:[email-masked]>

Implementation status:

Sue Hares

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Sat, 10 Oct 2015 01:44:15 +0000

Hi Juergen, Alex,

Currently in the L2 topology model there is one leaf "tp-state" which is state data. We can remove it first before we reach some agreement on how to introduce operational/state information to topology models.

Besides the approach Juergen suggested, do we also need to consider the approach proposed by openconfig-opstate? 

Best regards,
Jie

> From: i2rs [mailto:[email-masked]] On Behalf Of Alexander Clemm (alex)
> Sent: Saturday, October 10, 2015 5:55 AM
> To: Juergen Schoenwaelder; Susan Hares
> Cc: Andy Bierman; [email-masked]; Martin Bjorklund; Ladislav Lhotka; 'Alia Atlas';
> 'Jeffrey Haas'
> Essentially, in ietf-network and ietf-network-topology, we have all
> configuration information.  The same is true for the Layer 3 topology - all
> configuration information.  (I cannot comment on L2 topology as I am not
> involved there.)
> 
> Are you saying that we should add an additional branch as a placeholder (and
> perhaps an augmentation target) for operational data, even if we have not
> otherwise defined any operational data?
> 
> The only item in the topology that is read-only concerns the "server-provided"
> flag, but this concerns a separate issue that was also discussed (I am not sure if
> this is what you are referring to).  It is analogous to the other discussion
> concerning distinguishing configuration that has been intended, versus one that
> is in effect etc .  This concerns the issue that some topologies are populated by
> the server whereas some topologies can be populated by client applications.
> We have had discussions in the past whether to "split this up", having a (rw)
> branch to populate "intended" topologies and a (ro) branch for topologies "in
> effect".  We decided against it for various reasons - every piece of information
> would essentially be duplicated inside the model (this is not your normal config
> vs oper data distinction, but would essentially reflect a limitation of the
> framework), leading to unnecessary additional complexity in the model (every
> augmentation has to be conducted in two !
> places),
>  more complex validation rules, etc.
> 
> --- Alex
> 
> Cc: [email-masked]; Martin Bjorklund <[email-masked]>; Ladislav Lhotka
> <[email-masked]>; Andy Bierman <[email-masked]>; 'Jeffrey Haas'
> <[email-masked]>; 'Alia Atlas' <[email-masked]>
> there is a discussion on the yang doctors mailing list about the data model
> design that mixes state data and configuration data into one subtree and that
> uses a data model specific runtime object to identify what is config data and
> what is state data. One of the main outcomes of the IAB workshop back in 2002
> was the need to clearly separate configuration from operational state and this
> has been driven the design of NETCONF and YANG. YANG data model validation
> rules, for example, make a distinction between config true and config false
> data.
> The config true or false property impacts what is returned by NETCONF's
> get-config operation. Also note that config data is not allowed to refer to
> config false data in constraints.
> 
> Section 3.5 describes this approach in the 3rd paragraph and states "As most
> data is defined in those groupings, the amount of additional definitions required
> will be limited." and there are no strong reasons given why this approach has
> not been followed.
> 
> /js
>

---

From: Xufeng Liu <[email-masked]>
Date: Tue, 13 Oct 2015 03:15:24 +0000

Hi Xian,

Responding on item 6 inline below.
Thanks,

- Xufeng

From: Alexander Clemm (alex) [mailto:[email-masked]]
Sent: Friday, October 09, 2015 8:21 PM
To: Zhangxian (Xian); Xufeng Liu; Susan Hares; [email-masked]
Cc: 'Jeffrey Haas'; TEAS WG; 'Alia Atlas'
Subject: RE: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Xian,

responses inline, <ALEX>

Thanks
--- Alex

From: i2rs [mailto:[email-masked]] On Behalf Of Zhangxian (Xian)
Sent: Thursday, October 08, 2015 7:20 PM
To: Xufeng Liu <[email-masked]<mailto:[email-masked]>>; Susan Hares <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Cc: 'Jeffrey Haas' <[email-masked]<mailto:[email-masked]>>; TEAS WG <[email-masked]<mailto:[email-masked]>>; 'Alia Atlas' <[email-masked]<mailto:[email-masked]>>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi, Xufeng, Alex,

<ALEX> There is no operational data defined.  The server-provided attributed guides whether network topology information is populated by the server or by the client application, but the information is the same – this not the config vs oper data (read: stats) separation.  To add stats, you should provide an additional subtree/branch as applicable when you are augmenting.
</ALEX>

<ALEX>

Thanks, Alex (end of my comments)
</ALEX>
[Xufeng] This is for both future-proofness and current requirement. To allow operator easily configure TE topology attributes, we are defining templates that can be applied to all networks.

Thank you,
Xian

Hi Folks,

Regards,

Implementation status:

Sue Hares

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 15 Oct 2015 08:34:46 +0200

On Fri, Oct 09, 2015 at 09:55:19PM +0000, Alexander Clemm (alex) wrote:
> 
> The only item in the topology that is read-only concerns the
> "server-provided" flag, but this concerns a separate issue that was
> also discussed (I am not sure if this is what you are referring to).
> It is analogous to the other discussion concerning distinguishing
> configuration that has been intended, versus one that is in effect
> etc .  This concerns the issue that some topologies are populated by

Yes, this is what the concern is about.

> We have had discussions in the past whether to "split this up",
> having a (rw) branch to populate "intended" topologies and a (ro)
> branch for topologies "in effect".

This is the normal way to do this in YANG. And this goes back to what
was driving us for years, namely to clearly separate config from
state. This module makes this distinction a runtime property
controlled by a data model specific mechanism. None of the generic
tools out there will be able to understand this.

> We decided against it for various reasons - every piece of
> information would essentially be duplicated inside the model (this
> is not your normal config vs oper data distinction, but would
> essentially reflect a limitation of the framework), leading to
> unnecessary additional complexity in the model (every augmentation
> has to be conducted in two places), more complex validation rules,
> etc.

I do not understand why this is not a normal config vs oper data
distinction. Please explain.

I do not understand how this leads to more complex validation rules.
Please explain.

/js

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Thu, 15 Oct 2015 22:59:31 +0000

Hello Juergen, 

responses inline, delimited with <ALEX>

--- Alex

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Wednesday, October 14, 2015 11:35 PM
To: Alexander Clemm (alex) <[email-masked]>
Cc: Susan Hares <[email-masked]>; Andy Bierman <[email-masked]>; [email-masked]; Martin Bjorklund <[email-masked]>; Ladislav Lhotka <[email-masked]>; 'Alia Atlas' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

On Fri, Oct 09, 2015 at 09:55:19PM +0000, Alexander Clemm (alex) wrote:
> configuration that has been intended, versus one that is in effect etc 
> .  This concerns the issue that some topologies are populated by the 
> server whereas some topologies can be populated by client 
> applications.

> We have had discussions in the past whether to "split this up", having 
> a (rw) branch to populate "intended" topologies and a (ro) branch for 
> topologies "in effect".

This is the normal way to do this in YANG. And this goes back to what was driving us for years, namely to clearly separate config from state. This module makes this distinction a runtime property controlled by a data model specific mechanism. None of the generic tools out there will be able to understand this.

<ALEX>
I think the issue is more related to the current discussion with regards to openconfig and "intended configuration" and "applied configuration".  If YANG had an existing solution for this, we would not have this discussion.  The reason I believe this is similar is that you can view the "applied configuration" as the "server-provided configuration" (network topology, in our case), and the "intended configuration" as the, well, configured or intended network topology in our case.  That said, the issue is not identical - whereas in the openconfig case every "applied configuration" has an accompanying "intended configuration", in our case this is not necessarily the case - you can have "applied" [network topologies] that were provided by the server / the network itself, not configured by anybody.  
</ALEX>

> We decided against it for various reasons - every piece of information 
> would essentially be duplicated inside the model (this is not your 
> normal config vs oper data distinction, but would essentially reflect 
> a limitation of the framework), leading to unnecessary additional 
> complexity in the model (every augmentation has to be conducted in two 
> places), more complex validation rules, etc.

I do not understand why this is not a normal config vs oper data distinction. Please explain.

<ALEX>
A normal distinction would be e.g. the type of model we have in RFC 7233 - separate trees with distinct data, some clearly part of configuration, other clearly operational data.  
In this case, this is different.  You have the same data.  However, in some cases it is populated by a client, in other cases by the server.  YANG requires the categorization of data as config false or true.  In this case, this categorization does not always apply - or, the categorization depends on the particular instance.  
</ALEX> 

<ALEX>

One example concerns the supporting nodes/links/TPs.  

We want to be able to express that, for example, a node in one network is supported by a node in an underlay network.  For this purpose, we are referencing a node in another (underlay) network.  So that we cannot reference an arbitrary node in an arbitrary network, we want to make sure that the supporting node is part of a "supporting-network" of the same network. 

Currently, we have the following definition:

   list supporting-node {
        key "network-ref node-ref";
        description
          "Represents another node, in an underlay network, that 
           this node is supported by.  Used to represent layering 
           structure.";
        leaf network-ref {
          type leafref {
            path "../../../supporting-network/network-ref";
          }
          description
            "References the underlay network that the 
             underlay node is part of.";
        }
        leaf node-ref {
          type leafref {
            path "/network/node/node-id";
          }
          description
            "References the underlay node itself.";
        }
      }

If we were to split the model, when we configure a node, we will have to account for the fact that the supporting node could be either part of a "configured" network itself, or of a network that has been "server-provided".  That is, we need to be able to allow for both possibilities.  

To do this, we would no longer be able to have the network-ref to be part of the key for supporting-node - we would have to replace network-ref with a choice of two nodes that reference either a server-provided network ("branch 1"), or a configured network ("branch 2").  As a result, we will have to introduce a separate way to reference elements in list supporting-node.  All of this results in considerable additional complexity.  Or do you see an easier way?  

</ALEX>

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 18 Oct 2015 12:13:16 +0200

> I think the issue is more related to the current discussion with regards to openconfig and "intended configuration" and "applied configuration".  If YANG had an existing solution for this, we would…

I think this has nothing to do with intended or applied config. Your
'server supplied topology' appears to me to be operational state and
not configuration data.
 
> In this case, this is different.  You have the same data.  However, in some cases it is populated by a client, in other cases by the server.  YANG requires the categorization of data as config fals…

So you have operational state which is partially populated by the
server and partially populated from config. I fail to see how this is
any different from other cases, including network interfaces as
defined in RFC 7233. Recall also that YANG does not allow
configuration data to depend on state data.

>             "References the underlay node itself.";
/
> If we were to split the model, when we configure a node, we will have to account for the fact that the supporting node could be either part of a "configured" network itself, or of a network that ha…

Again note that YANG requires that configuration data does not depend
on state data. You seem to be breaking this rule, no?

> To do this, we would no longer be able to have the network-ref to be part of the key for supporting-node - we would have to replace network-ref with a choice of two nodes that reference either a se…

I do not think this is the solution. YANG requires that constraints on
config true nodes can only refer to other config true nodes in the
datastore where the node with the constraint exists. See section 7.5.3
and section 7.19.5. And concerning leafref, section 9.9 says that a
leafref may only point to configuration. I believe this I-D is
violating the distinction between configuration and state data.

/js

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Mon, 19 Oct 2015 18:24:10 +0000

Hi Juergen,

I think one of the key statements you make below is this:
" Recall also that YANG does not allow configuration data to depend on state data."

Note that this is not the case in the current model.  The current model is essentially all configuration data.  Of course, we have this flag to indicate who supplied that data (and is hence maintaining it).    

You argue that we should instead "split" the model and introduce operational data to reflect what is populated by the server.  However, doing that introduces precisely new issues - and you have just brought another argument why this may be a bad idea and may not even work.  Topologies _are_ layered, and we need to be able to express that in the model.  Now, if we have a topology that is server-provided, hence (per your statement) to be represented by operational data only, how do we build an overlay topology that is "configured" on top of it?  If the answer is "we can't, unless we replicate the server-provided topology into the network configuration (which makes no sense)", we are screwed.  Now, we might build it on top if we remove all references / dependencies on the underlay from the model and punt the problem to the user.  Basically, no longer have the model express vertical relationships.  Not a good solution, IMHO.  

How do you suggest we address this?  The ability to express layering relationships between topologies, including cases where topologies originate from different sources (discovered/server-provided vs configured), is a requirement.  It is not an artefact of our model, it is something that we need to capture as part of the model.  There may not be a "nice" way of doing this within the YANG framework, yet it is important that we find a way to do this.  The current solution to this - having the model as configuration data, and including a parameter to indicate who supplies the data and is maintaining it - appears to be cleanest and clearest solution (or perhaps the "least bad") that results in the model of least complexity.   

Perhaps there is something we can simply change about the "server-provided" object to address your concerns?  We can make it config (to address your issue that triggered this, the presence of a r/o object in a tree that is otherwise r/w).  

Thoughts?
--- Alex

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Sunday, October 18, 2015 3:13 AM
To: Alexander Clemm (alex) <[email-masked]>
Cc: Ladislav Lhotka <[email-masked]>; [email-masked]; Martin Bjorklund <[email-masked]>; Andy Bierman <[email-masked]>; 'Alia Atlas' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Susan Hares <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

On Thu, Oct 15, 2015 at 10:59:31PM +0000, Alexander Clemm (alex) wrote:
> From: Juergen Schoenwaelder 
> [mailto:[email-masked]]
> Cc: Susan Hares <[email-masked]>; Andy Bierman <[email-masked]>; 
> [email-masked]; Martin Bjorklund <[email-masked]>; Ladislav Lhotka 
> <[email-masked]>; 'Alia Atlas' <[email-masked]>; 'Jeffrey Haas' 
> <[email-masked]>

I think this has nothing to do with intended or applied config. Your 'server supplied topology' appears to me to be operational state and not configuration data.
 
> > has to be conducted in two places), more complex validation rules, etc.
> 

So you have operational state which is partially populated by the server and partially populated from config. I fail to see how this is any different from other cases, including network interfaces as defined in RFC 7233. Recall also that YANG does not allow configuration data to depend on state data.

/
> If we were to split the model, when we configure a node, we will have to account for the fact that the supporting node could be either part of a "configured" network itself, or of a network that ha…

Again note that YANG requires that configuration data does not depend on state data. You seem to be breaking this rule, no?

> To do this, we would no longer be able to have the network-ref to be part of the key for supporting-node - we would have to replace network-ref with a choice of two nodes that reference either a se…

I do not think this is the solution. YANG requires that constraints on config true nodes can only refer to other config true nodes in the datastore where the node with the constraint exists. See section 7.5.3 and section 7.19.5. And concerning leafref, section 9.9 says that a leafref may only point to configuration. I believe this I-D is violating the distinction between configuration and state data.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 20:32:16 +0200 (CEST)

Alex,

Is the idea that the server-provided data is normal config?  I.e., if
the server wants to modify this data it behaves like a normal client?
(conceptually...)  And the server-provided data can be modified by
anyone with proper access rights? 

/martin

"Alexander Clemm (alex)" <[email-masked]> wrote:
> Note that this is not the case in the current model.  The current
> model is essentially all configuration data.  Of course, we have this
> flag to indicate who supplied that data (and is hence maintaining it).
> 
> You argue that we should instead "split" the model and introduce
> operational data to reflect what is populated by the server.  However,
> doing that introduces precisely new issues - and you have just brought
> another argument why this may be a bad idea and may not even work.
> Topologies _are_ layered, and we need to be able to express that in
> the model.  Now, if we have a topology that is server-provided, hence
> (per your statement) to be represented by operational data only, how
> do we build an overlay topology that is "configured" on top of it?  If
> the answer is "we can't, unless we replicate the server-provided
> topology into the network configuration (which makes no sense)", we
> are screwed.  Now, we might build it on top if we remove all
> references / dependencies on the underlay from the model and punt the
> problem to the user.  Basically, no longer have the model express
> vertical relationships.  Not a good solution, IMHO.
> 
> How do you suggest we address this?  The ability to express layering
> relationships between topologies, including cases where topologies
> originate from different sources (discovered/server-provided vs
> configured), is a requirement.  It is not an artefact of our model, it
> is something that we need to capture as part of the model.  There may
> not be a "nice" way of doing this within the YANG framework, yet it is
> important that we find a way to do this.  The current solution to this
> - having the model as configuration data, and including a parameter to
> indicate who supplies the data and is maintaining it - appears to be
> cleanest and clearest solution (or perhaps the "least bad") that
> results in the model of least complexity.
> 
> Perhaps there is something we can simply change about the
> "server-provided" object to address your concerns?  We can make it
> config (to address your issue that triggered this, the presence of a
> r/o object in a tree that is otherwise r/w).
> 
> Thoughts?
> --- Alex
> 
> 
> 
> Cc: Ladislav Lhotka <[email-masked]>; [email-masked]; Martin Bjorklund
> <[email-masked]>; Andy Bierman <[email-masked]>; 'Alia Atlas'
> <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Susan Hares
> <[email-masked]>
> > I think the issue is more related to the current discussion with
> > regards to openconfig and "intended configuration" and "applied
> > configuration".  If YANG had an existing solution for this, we would
> > not have this discussion.  The reason I believe this is similar is
> > that you can view the "applied configuration" as the "server-provided
> > configuration" (network topology, in our case), and the "intended
> > configuration" as the, well, configured or intended network topology
> > in our case.  That said, the issue is not identical - whereas in the
> > openconfig case every "applied configuration" has an accompanying
> > "intended configuration", in our case this is not necessarily the case
> > - you can have "applied" [network topologies] that were provided by
> > the server / the network itself, not configured by anybody.
> > </ALEX>
> 
> > A normal distinction would be e.g. the type of model we have in RFC
> > 7233 - separate trees with distinct data, some clearly part of
> > configuration, other clearly operational data.
> > In this case, this is different.  You have the same data.  However, in
> > some cases it is populated by a client, in other cases by the server.
> > YANG requires the categorization of data as config false or true.  In
> > this case, this categorization does not always apply - or, the
> > categorization depends on the particular instance.
> > </ALEX>
> 
> > We want to be able to express that, for example, a node in one network
> > is supported by a node in an underlay network.  For this purpose, we
> > are referencing a node in another (underlay) network.  So that we
> > cannot reference an arbitrary node in an arbitrary network, we want to
> > make sure that the supporting node is part of a "supporting-network"
> > of the same network.
> > 
> > If we were to split the model, when we configure a node, we will have
> > to account for the fact that the supporting node could be either part
> > of a "configured" network itself, or of a network that has been
> > "server-provided".  That is, we need to be able to allow for both
> > possibilities.
> 
> > To do this, we would no longer be able to have the network-ref to be
> > part of the key for supporting-node - we would have to replace
> > network-ref with a choice of two nodes that reference either a
> > server-provided network ("branch 1"), or a configured network ("branch
> > 2").  As a result, we will have to introduce a separate way to
> > reference elements in list supporting-node.  All of this results in
> > considerable additional complexity.  Or do you see an easier way?
> > 
> > </ALEX>
>

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Mon, 19 Oct 2015 19:05:14 +0000

Hi Martin,

One model for the data that is server-provided is to assume an app (which could be embedded on the same server) that knows how to discover the network, then populates the data accordingly.  

[Of course, we would not want any random client app just being able to "mess" with that data.  The expectation is generally clearly access to this will be restricted / controlled.  The topology instances that were populated by the "server-provided app" should not be "touched" by other apps - it is the "server-provided" app that is responsible for maintaining them.]

So I assume the answer to your question is "yes", but with a bunch of caveats.  
--- Alex

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Martin Bjorklund
Sent: Monday, October 19, 2015 11:32 AM
To: Alexander Clemm (alex) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Alex,

Is the idea that the server-provided data is normal config?  I.e., if the server wants to modify this data it behaves like a normal client?
(conceptually...)  And the server-provided data can be modified by anyone with proper access rights? 

/martin

"Alexander Clemm (alex)" <[email-masked]> wrote:
> > Cc: Susan Hares <[email-masked]>; Andy Bierman 
> > <[email-masked]>; [email-masked]; Martin Bjorklund 
> > <[email-masked]>; Ladislav Lhotka <[email-masked]>; 'Alia Atlas' <[email-masked]>; 'Jeffrey Haas'
> > <[email-masked]>
> > > "server-provided" flag, but this concerns a separate issue that 
> > > was also discussed (I am not sure if this is what you are referring to).
> > > etc .  This concerns the issue that some topologies are populated 
> > > by the server whereas some topologies can be populated by client 
> > > applications.
> > 
> > This is the normal way to do this in YANG. And this goes back to 
> > what was driving us for years, namely to clearly separate config 
> > from state. This module makes this distinction a runtime property 
> > that you can view the "applied configuration" as the 
> > "server-provided configuration" (network topology, in our case), and 
> > the "intended configuration" as the, well, configured or intended 
> > network topology in our case.  That said, the issue is not identical 
> > - whereas in the openconfig case every "applied configuration" has 
> > an accompanying "intended configuration", in our case this is not 
> > necessarily the case
> > In this case, this is different.  You have the same data.  However, 
> > in some cases it is populated by a client, in other cases by the server.
> > YANG requires the categorization of data as config false or true.  
> > In this case, this categorization does not always apply - or, the 
> > We want to be able to express that, for example, a node in one 
> > network is supported by a node in an underlay network.  For this 
> > purpose, we are referencing a node in another (underlay) network.  
> > So that we cannot reference an arbitrary node in an arbitrary 
> > network, we want to make sure that the supporting node is part of a "supporting-network"
> > of the same network.
> > 
> > If we were to split the model, when we configure a node, we will 
> > have to account for the fact that the supporting node could be 
> > either part of a "configured" network itself, or of a network that 
> > has been "server-provided".  That is, we need to be able to allow 
> > for both possibilities.
> 
> > server-provided network ("branch 1"), or a configured network 
> > ("branch 2").  As a result, we will have to introduce a separate way 
> > to reference elements in list supporting-node.  All of this results 
> > in considerable additional complexity.  Or do you see an easier way?
> > 
> > </ALEX>
>

---

From: Xufeng Liu <[email-masked]>
Date: Mon, 19 Oct 2015 19:08:01 +0000

Hi Martin and Alex,

Some of such objects might not be changeable, no matter what access right that the client has. Such non-changeable objects are derived from other objects in the system.

Regards,

- Xufeng

> Sent: Monday, October 19, 2015 3:05 PM
> To: Martin Bjorklund
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> One model for the data that is server-provided is to assume an app (which could
> be embedded on the same server) that knows how to discover the network, then
> populates the data accordingly.
> 
> [Of course, we would not want any random client app just being able to "mess"
> with that data.  The expectation is generally clearly access to this will be
> restricted / controlled.  The topology instances that were populated by the
> "server-provided app" should not be "touched" by other apps - it is the "server-
> provided" app that is responsible for maintaining them.]
> 
> Cc: [email-masked]; [email-masked]; j.schoenwaelder@jacobs-
> university.de; [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> Is the idea that the server-provided data is normal config?  I.e., if the server
> wants to modify this data it behaves like a normal client?

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 19 Oct 2015 21:40:56 +0200

> Some of such objects might not be changeable, no matter what access right that the client has. Such non-changeable objects are derived from other objects in the system.

This sounds remarkably like operational state...

/js

---

From: Xufeng Liu <[email-masked]>
Date: Mon, 19 Oct 2015 19:55:14 +0000

> Sent: Monday, October 19, 2015 3:41 PM
> To: Xufeng Liu
> Cc: [email-masked]; [email-masked]; Martin Bjorklund; Alexander Clemm (alex);
> > Some of such objects might not be changeable, no matter what access right
> that the client has. Such non-changeable objects are derived from other objects
> in the system.
> >
> 

[Xufeng] But configuration objects can be built on top of these.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 19 Oct 2015 21:56:00 +0200

> Note that this is not the case in the current model.  The current model is essentially all configuration data.  Of course, we have this flag to indicate who supplied that data (and is hence maintai…

I expect that config true objects can be modified by clients. And I
expect that client maintained config true objects do not depend on
server provided operational state. The 'server-provided' leaf seems to
'overrule' both principles.

> How do you suggest we address this?  The ability to express layering relationships between topologies, including cases where topologies originate from different sources (discovered/server-provided…

In the interfaces model, this problem got solved via name bindings. I
can configure an interface but as long as the matching interface is
not present, the config won't be applied to the interface. Translated
to your domain, I can configure an overlay that refers by name to
other layers. If those layers do not operationally exist, the overlay
will exist in config but not in the operational state tree.

> Perhaps there is something we can simply change about the "server-provided" object to address your concerns?  We can make it config (to address your issue that triggered this, the presence of a r/o…

You can address my concern by doing away with 'server-provided' and by
properly separating config from operational state. ;-) Assuming proper
authorization, a NC/RC client should be able to change any object in a
configuration datastore. I am concerned about data model specific
'fences' that indicate that some portions of a configuration datastore
are behaving in special ways.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 22:26:45 +0200 (CEST)

"Alexander Clemm (alex)" <[email-masked]> wrote:
> Hi Martin,
> 
> One model for the data that is server-provided is to assume an app
> (which could be embedded on the same server) that knows how to
> discover the network, then populates the data accordingly.
> 
> [Of course, we would not want any random client app just being able to
> "mess" with that data.  The expectation is generally clearly access to
> this will be restricted / controlled.  The topology instances that
> were populated by the "server-provided app" should not be "touched" by
> other apps - it is the "server-provided" app that is responsible for
> maintaining them.]
> 

So how is the server-provided leaf supposed to be implemented, and how
is it supposed to be used?

/martin

> Cc: [email-masked]; [email-masked];
> [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 22:27:13 +0200 (CEST)

Xufeng Liu <[email-masked]> wrote:
> Hi Martin and Alex,
> 
> Some of such objects might not be changeable, no matter what access
> right that the client has. Such non-changeable objects are derived
> from other objects in the system.

That would be config false nodes, right?

/martin

> > (which could
> > "mess"
> > the server

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 22:28:13 +0200 (CEST)

> [Xufeng] But configuration objects can be built on top of these.

What does this mean?

/martin

---

From: Xufeng Liu <[email-masked]>
Date: Mon, 19 Oct 2015 20:39:52 +0000

> From: Martin Bjorklund [mailto:[email-masked]]
> Sent: Monday, October 19, 2015 4:27 PM
> To: Xufeng Liu
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked]
[Xufeng] Yes. Agree.
> > > From: i2rs [mailto:[email-masked]] On Behalf Of Alexander
> > > Clemm
> > > (alex)
> > > with that data.  The expectation is generally clearly access to this
> > > will be restricted / controlled.  The topology instances that were
> > > populated by the "server-provided app" should not be "touched" by
> > > other apps - it is the "server- provided" app that is responsible
> > > for maintaining them.]
> > >
> > > Is the idea that the server-provided data is normal config?  I.e.,
> > > if the server wants to modify this data it behaves like a normal
> > > client?
> > > > model is essentially all configuration data.  Of course, we have
> > > > this flag to indicate who supplied that data (and is hence maintaining it).
> > > >
> > > > operational data to reflect what is populated by the server.
> > > > However, doing that introduces precisely new issues - and you have
> > > > just brought another argument why this may be a bad idea and may not
> even work.
> > > > Topologies _are_ layered, and we need to be able to express that
> > > > in the model.  Now, if we have a topology that is server-provided,
> > > > hence (per your statement) to be represented by operational data
> > > > only, how do we build an overlay topology that is "configured" on
> > > > top of it?  If the answer is "we can't, unless we replicate the
> > > > server-provided topology into the network configuration (which
> > > > makes no sense)", we are screwed.  Now, we might build it on top
> > > > if we remove all references / dependencies on the underlay from
> > > > the model and punt the problem to the user.  Basically, no longer
> > > > have the model express vertical relationships.  Not a good solution, IMHO.
> > > >
> > > > How do you suggest we address this?  The ability to express
> > > > layering relationships between topologies, including cases where
> > > > topologies originate from different sources
> > > > (discovered/server-provided vs configured), is a requirement.  It
> > > > is not an artefact of our model, it is something that we need to
> > > > capture as part of the model.  There may not be a "nice" way of
> > > > doing this within the YANG framework, yet it is important that we
> > > > find a way to do this.  The current solution to this
> > > > - having the model as configuration data, and including a
> > > > parameter to indicate who supplies the data and is maintaining it
> > > > - appears to be cleanest and clearest solution (or perhaps the
> > > > "least bad") that results in the model of least complexity.
> > > >
> > > > config (to address your issue that triggered this, the presence of
> > > > a r/o object in a tree that is otherwise r/w).
> > > >
> > > > Thoughts?
> > > > --- Alex
> > > >
> > > >
> > > >
> > > > Cc: Ladislav Lhotka <[email-masked]>; [email-masked]; Martin
> > > > Bjorklund <[email-masked]>; Andy Bierman <[email-masked]>;
> 'Alia Atlas'
> > > > > > "server-provided" flag, but this concerns a separate issue
> > > > > > that was also discussed (I am not sure if this is what you are
> > > > > > referring to).
> > > > > > It is analogous to the other discussion concerning
> > > > > > distinguishing configuration that has been intended, versus
> > > > > > one that is in effect etc .  This concerns the issue that some
> > > > > > topologies are populated by the server whereas some topologies
> > > > > > can be populated by client applications.
> > > > >
> > > > > > We have had discussions in the past whether to "split this
> > > > > > up", having a (rw) branch to populate "intended" topologies
> > > > > > and a (ro) branch for topologies "in effect".
> > > > >
> > > > > from state. This module makes this distinction a runtime
> > > > > property controlled by a data model specific mechanism. None of
> > > > > the generic tools out there will be able to understand this.
> > > > >
> > > > > <ALEX>
> > > > > configuration".  If YANG had an existing solution for this, we
> > > > > would not have this discussion.  The reason I believe this is
> > > > > similar is that you can view the "applied configuration" as the
> > > > > "server-provided configuration" (network topology, in our case),
> > > > > and the "intended configuration" as the, well, configured or
> > > > > intended network topology in our case.  That said, the issue is
> > > > > not identical
> > > > > - whereas in the openconfig case every "applied configuration"
> > > > > has an accompanying "intended configuration", in our case this
> > > > > is not necessarily the case
> > > > > - you can have "applied" [network topologies] that were provided
> > > > > by the server / the network itself, not configured by anybody.
> > > > > </ALEX>
> > > >
> > > > I think this has nothing to do with intended or applied config.
> > > > Your 'server supplied topology' appears to me to be operational
> > > > state and not configuration data.
> > > >
> > > > > > information would essentially be duplicated inside the model
> > > > > > (this is not your normal config vs oper data distinction, but
> > > > > > would essentially reflect a limitation of the framework),
> > > > > > leading to unnecessary additional complexity in the model
> > > > > > (every augmentation has to be conducted in two places), more
> > > > > > complex validation rules, etc.
> > > > >
> > > > > In this case, this is different.  You have the same data.
> > > > > However, in some cases it is populated by a client, in other
> > > > > cases by the server.
> > > > > In this case, this categorization does not always apply - or,
> > > > > the categorization depends on the particular instance.
> > > > > </ALEX>
> > > >
> > > > server and partially populated from config. I fail to see how this
> > > > is any different from other cases, including network interfaces as
> > > > > network, we want to make sure that the supporting node is part
> > > > > of a
> > > "supporting-network"
> > > > > of the same network.
> > > > >
> > > > > either part of a "configured" network itself, or of a network
> > > > > that has been "server-provided".  That is, we need to be able to
> > > > > allow for both possibilities.
> > > >
> > > > Again note that YANG requires that configuration data does not
> > > > depend on state data. You seem to be breaking this rule, no?
> > > >
> > > > > To do this, we would no longer be able to have the network-ref
> > > > > to be part of the key for supporting-node - we would have to
> > > > > replace network-ref with a choice of two nodes that reference
> > > > > either a server-provided network ("branch 1"), or a configured
> > > > > network ("branch 2").  As a result, we will have to introduce a
> > > > > separate way to reference elements in list supporting-node.  All
> > > > > of this results in considerable additional complexity.  Or do you see an
> easier way?
> > > > >
> > > > > </ALEX>
> > > >
> > > > I do not think this is the solution. YANG requires that
> > > > constraints on config true nodes can only refer to other config
> > > > true nodes in the datastore where the node with the constraint
> > > > exists. See section 7.5.3 and section 7.19.5. And concerning
> > > > leafref, section 9.9 says that a leafref may only point to
> > > > configuration. I believe this I-D is violating the distinction between
> configuration and state data.
> > > >
> > > > /js
> > > >
> > > > --

---

From: Xufeng Liu <[email-masked]>
Date: Mon, 19 Oct 2015 20:41:39 +0000

> Sent: Monday, October 19, 2015 4:28 PM
> To: Xufeng Liu
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> > > Cc: [email-masked]; [email-masked]; Martin Bjorklund; Alexander Clemm
> > > (alex); [email-masked]; [email-masked]; [email-masked];
> > > [email-masked]
> > > that the client has. Such non-changeable objects are derived from
> > > other objects in the system.
> > > >
> > >
[Xufeng] The client may need to configure some other objects that have references to these derived objects.

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Mon, 19 Oct 2015 21:42:31 +0000

Hello Juergen,

can you please explain what you mean by name bindings?  YANG does not provide support for name bindings (at least not a la GDMO).  

Looking at RFC 7223, I can see the distinction between interface-ref and interface-state-ref.  There certainly is an analogy between interfaces and topology, and the discussion section 3.1 of RFC 7223 and the discussion here.  Still, the interfaces model is different and follows different requirements/patterns that don't make it directly applicable.  Layering etc are not included as part of the base model, but supported via a design pattern where it is added as part of augmentation specific to a given interface type, and being specific which type of interface to reference.  I don't see anything describing the analogy of having a (configured) set of data - e.g. a (configured) topology with a (configured) set of nodes, links, etc - and allowing the items in this to refer to e.g. a (server-provided) topology.  

In your email, you state "Translated to your domain, I can configure an overlay that refers by name to other layers. If those layers do not operationally exist, the overlay will exist in config but not in the operational state tree".  How would this work?  With "refer by name" you don't mean an actual reference, you mean use "refer by name" to refer to data that is operational state.  In other words, you are punting the problem and leave it to the user and implementation to deal with it.  Instead of using e.g. an actual reference, you are using simply a string.  The concern is that when using a reference, a framework might have problems ensuring referential integrity in case the referred object goes away etc - but this solution does not address the concern, it simply pushes it to somewhere else.  

At the same time, you do now have two parallel trees, providing additional complexity as outlined earlier, specifically if we want to be able to express some of the integrity constraints (supporting nodes of a node have to be part of a supporting topology, etc).  Well, perhaps we no longer need to be concerned about these, since we are pushing this problem to the user anyway.  However, I am not sure why we would trust users to deal with all those integrity problems, but not trust them e.g. with a "server-provided" attribute.  Additional complexity in the model would be fine if it resulted in a gain, but I don't think this would be the case - to the contrary.  Introducing what you call a "fence" instead seems to be the preferrable solution.  The ability to support this could be indicated by a feature, if that helps.    

--- Alex

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Monday, October 19, 2015 12:56 PM
To: Alexander Clemm (alex) <[email-masked]>
Cc: Andy Bierman <[email-masked]>; [email-masked]; Martin Bjorklund <[email-masked]>; Ladislav Lhotka <[email-masked]>; 'Alia Atlas' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Susan Hares <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

> Note that this is not the case in the current model.  The current model is essentially all configuration data.  Of course, we have this flag to indicate who supplied that data (and is hence maintai…

I expect that config true objects can be modified by clients. And I expect that client maintained config true objects do not depend on server provided operational state. The 'server-provided' leaf seems to 'overrule' both principles.

> How do you suggest we address this?  The ability to express layering relationships between topologies, including cases where topologies originate from different sources (discovered/server-provided…

In the interfaces model, this problem got solved via name bindings. I can configure an interface but as long as the matching interface is not present, the config won't be applied to the interface. Translated to your domain, I can configure an overlay that refers by name to other layers. If those layers do not operationally exist, the overlay will exist in config but not in the operational state tree.

> Perhaps there is something we can simply change about the "server-provided" object to address your concerns?  We can make it config (to address your issue that triggered this, the presence of a r/o…

You can address my concern by doing away with 'server-provided' and by properly separating config from operational state. ;-) Assuming proper authorization, a NC/RC client should be able to change any object in a configuration datastore. I am concerned about data model specific 'fences' that indicate that some portions of a configuration datastore are behaving in special ways.

/js

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Mon, 19 Oct 2015 21:57:05 +0000

Hi Martin,

"So how is the server-provided leaf supposed to be implemented, and how is it supposed to be used?"

When a network topology is populated by the server, the server-provided leaf is supposed to be set to true.
When a network topology is populated by a client app (through "regular" configuration), the server provided leaf is supposed to be set to false.  

For any given network topology, when the corresponding "server-provided" leaf is set to "true", attempts to edit the configuration of that topology are to be rejected.  

Alternatives to the current design include making the leaf "config true", or moving it outside (just this leaf) for a list that indicates for each topology whether it is server-provided or not (in a separate "state" branch).  
--- Alex

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Monday, October 19, 2015 1:27 PM
To: Alexander Clemm (alex) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

> So I assume the answer to your question is "yes", but with a bunch of 

So how is the server-provided leaf supposed to be implemented, and how is it supposed to be used?

/martin

> > just brought another argument why this may be a bad idea and may not even work.
> > the model.  Now, if we have a topology that is server-provided, 
> > server-provided topology into the network configuration (which makes 
> > no sense)", we are screwed.  Now, we might build it on top if we 
> > remove all references / dependencies on the underlay from the model 
> > and punt the problem to the user.  Basically, no longer have the 
> > model express vertical relationships.  Not a good solution, IMHO.
> > 
> > configured), is a requirement.  It is not an artefact of our model, 
> > it is something that we need to capture as part of the model.  There 
> > may not be a "nice" way of doing this within the YANG framework, yet 
> > it is important that we find a way to do this.  The current solution 
> > to this
> > - having the model as configuration data, and including a parameter 
> > to indicate who supplies the data and is maintaining it - appears to 
> > be cleanest and clearest solution (or perhaps the "least bad") that 
> > > > distinguishing configuration that has been intended, versus one 
> > > > that is in effect etc .  This concerns the issue that some 
> > > > having a (rw) branch to populate "intended" topologies and a 
> > > > (ro) branch for topologies "in effect".
> > > 
> > > > leading to unnecessary additional complexity in the model (every 
> > > > augmentation has to be conducted in two places), more complex 
> > > > validation rules, etc.
> > > 
> > > However, in some cases it is populated by a client, in other cases 
> > > by the server.
> > > To do this, we would no longer be able to have the network-ref to 
> > > be part of the key for supporting-node - we would have to replace 
> > > ("branch 2").  As a result, we will have to introduce a separate 
> > > way to reference elements in list supporting-node.  All of this 
> > > results in considerable additional complexity.  Or do you see an easier way?
> > > 
> > > </ALEX>
> > 
> > I do not think this is the solution. YANG requires that constraints 
> > on config true nodes can only refer to other config true nodes in 
> > the datastore where the node with the constraint exists. See section 
> > 7.5.3 and section 7.19.5. And concerning leafref, section 9.9 says 
> > that a leafref may only point to configuration. I believe this I-D 
> > is violating the distinction between configuration and state data.
> > 
> > /js
> > 
> > --

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 20 Oct 2015 08:04:26 +0200 (CEST)

"Alexander Clemm (alex)" <[email-masked]> wrote:
> Hello Juergen,
> 
> can you please explain what you mean by name bindings?  YANG does not
> provide support for name bindings (at least not a la GDMO).
> 
> Looking at RFC 7223, I can see the distinction between interface-ref
> and interface-state-ref.  There certainly is an analogy between
> interfaces and topology, and the discussion section 3.1 of RFC 7223
> and the discussion here.  Still, the interfaces model is different and
> follows different requirements/patterns that don't make it directly
> applicable.  Layering etc are not included as part of the base model,
> but supported via a design pattern where it is added as part of
> augmentation specific to a given interface type, and being specific
> which type of interface to reference.  I don't see anything describing
> the analogy of having a (configured) set of data - e.g. a (configured)
> topology with a (configured) set of nodes, links, etc - and allowing
> the items in this to refer to e.g. a (server-provided) topology.
> 
> In your email, you state "Translated to your domain, I can configure
> an overlay that refers by name to other layers. If those layers do not
> operationally exist, the overlay will exist in config but not in the
> operational state tree".  How would this work?  With "refer by name"
> you don't mean an actual reference, you mean use "refer by name" to
> refer to data that is operational state.  In other words, you are
> punting the problem and leave it to the user and implementation to
> deal with it.  Instead of using e.g. an actual reference, you are
> using simply a string.  The concern is that when using a reference, a
> framework might have problems ensuring referential integrity in case
> the referred object goes away etc - but this solution does not address
> the concern, it simply pushes it to somewhere else.

What happens in your model if a user-defined network has a reference
to a server-provided network, and the sever decides to remove its
network?  I see no special text in your document about this case.

/martin

> At the same time, you do now have two parallel trees, providing
> additional complexity as outlined earlier, specifically if we want to
> be able to express some of the integrity constraints (supporting nodes
> of a node have to be part of a supporting topology, etc).  Well,
> perhaps we no longer need to be concerned about these, since we are
> pushing this problem to the user anyway.  However, I am not sure why
> we would trust users to deal with all those integrity problems, but
> not trust them e.g. with a "server-provided" attribute.  Additional
> complexity in the model would be fine if it resulted in a gain, but I
> don't think this would be the case - to the contrary.  Introducing
> what you call a "fence" instead seems to be the preferrable solution.
> The ability to support this could be indicated by a feature, if that
> helps.
> 
> --- Alex
> 
> Cc: Andy Bierman <[email-masked]>; [email-masked]; Martin Bjorklund
> <[email-masked]>; Ladislav Lhotka <[email-masked]>; 'Alia Atlas'

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 20 Oct 2015 08:06:48 +0200 (CEST)

"Alexander Clemm (alex)" <[email-masked]> wrote:
> When a network topology is populated by the server, the
> server-provided leaf is supposed to be set to true.

But you earlier wrote that when the server wants to change something
it would behave as a normal client.

> When a network topology is populated by a client app (through
> "regular" configuration), the server provided leaf is supposed to be
> set to false.
> 
> For any given network topology, when the corresponding
> "server-provided" leaf is set to "true", attempts to edit the
> configuration of that topology are to be rejected.

This also goes against what you acknowledged previously - "the
server-provided data can be modified by anyone with proper access
rights"

/martin

> 
> Alternatives to the current design include making the leaf "config
> true", or moving it outside (just this leaf) for a list that indicates
> for each topology whether it is server-provided or not (in a separate
> "state" branch).
> --- Alex
> 
> Cc: [email-masked]; [email-masked];
> [email-masked]; [email-masked];
> > > > <[email-masked]>; 'Jeffrey Haas'
> > > > <[email-masked]>

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Fri, 23 Oct 2015 00:11:43 +0000

Hi Martin,

Agree, we need to add text for this.  Good catch.  

The reality is that the integrity of the links between overlay and underlay will be broken.  The fact that those links will be invalidated is something that ultimately needs to be indicated.  

Since the interface analogy was brought up, this problem is actually faced by layered interfaces that are user controlled as as well.  What is the solution that is applied there?  (I guess it is basically left up to the application, correct?)

--- Alex

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Monday, October 19, 2015 11:04 PM

.....

What happens in your model if a user-defined network has a reference to a server-provided network, and the sever decides to remove its network?  I see no special text in your document about this case.

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Fri, 23 Oct 2015 00:28:36 +0000

Hi Martin,

We can nitpick over how to best express this, but hopefully the general sense of what the requirement is and what I was trying to express is clear.  You can have an outside client application maintain some topologies / list elements, and have others maintained an populated by the server - or arguably an app embedded in the server.  The difference from the "normal" client is that really it is that we want the server / embedded app that is the authoritative owner of the information.  

Cheers
--- Alex

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Monday, October 19, 2015 11:07 PM
To: Alexander Clemm (alex) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

> server-provided leaf is supposed to be set to true.

But you earlier wrote that when the server wants to change something it would behave as a normal client.

> configuration of that topology are to be rejected.

This also goes against what you acknowledged previously - "the server-provided data can be modified by anyone with proper access rights"

/martin

> > [Of course, we would not want any random client app just being able 
> > to "mess" with that data.  The expectation is generally clearly 
> > access to this will be restricted / controlled.  The topology 
> > instances that were populated by the "server-provided app" should 
> > not be "touched" by other apps - it is the "server-provided" app 
> > that is responsible for maintaining them.]
> > 
> > if the server wants to modify this data it behaves like a normal client?
> > > Bjorklund <[email-masked]>; Andy Bierman <[email-masked]>; 'Alia Atlas'
> > > > of a "supporting-network"
> > > > of the same network.
> > > > 
> > > > of this results in considerable additional complexity.  Or do 
> > > > you see an easier way?
> > > > 
> > > > </ALEX>
> > > 
> > > exists. See section

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 23 Oct 2015 09:17:16 +0200 (CEST)

"Alexander Clemm (alex)" <[email-masked]> wrote:
> Hi Martin,
> 
> We can nitpick over how to best express this

This isn't nitpicking; I am trying to understand what you had in mind.

And it seems Juergen is correct.  You essentially want to mix config
and operational state in a single list.  This is not possible to
express in YANG, so you change the semantics of the formal statements
with descriptions.  I don't think this a good idea.

I can think of two alternatives to the current design that don't
violate YANG:

1)  Keep the single config true list, w/o the server-provided leaf,
    and explain in text that there might be a client
    "internal-to-the-server" that behaves just like any other client
    (specifically respects locks, makes sure the end result is
    valid, and any other client can modify what this client wrote
    (module access rights)).

2)  Split the model into two, one config list and one oper list.
    References in the config list can either be by name (implicit) or
    use the new YANG 1.1 syntax "require-instance false";

/martin

> , but hopefully the
> general sense of what the requirement is and what I was trying to
> express is clear.  You can have an outside client application maintain
> some topologies / list elements, and have others maintained an
> populated by the server - or arguably an app embedded in the server.
> The difference from the "normal" client is that really it is that we
> want the server / embedded app that is the authoritative owner of the
> information.
> 
> Cheers
> --- Alex
>

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 23 Oct 2015 09:21:59 +0200 (CEST)

"Alexander Clemm (alex)" <[email-masked]> wrote:
> The reality is that the integrity of the links between overlay and
> underlay will be broken.  The fact that those links will be
> invalidated is something that ultimately needs to be indicated.

But then they cannot be normal leafrefs.  You can use the new YANG 1.1
statement "require-instance false" to handle this.

> Since the interface analogy was brought up, this problem is actually
> faced by layered interfaces that are user controlled as as well.  What
> is the solution that is applied there?  (I guess it is basically left
> up to the application, correct?)

There is no generic model for this in RFC 7223, but there are three
options:  (i) use YANG 1.1 require-instance false (ii) require the
underlying interface to be configured (even if it is
system-controlled) or (iii) use an implicit reference.

/martin

---

From: Nikolay Milovanov <[email-masked]>
Date: Fri, 23 Oct 2015 11:07:41 +0300

In relationship to that the discussion about can or can't the model mix
operational state there are a couple of sentences in
"draft-ietf-i2rs-yang-l3-topology-00.txt" thats seems crucial to this and I
guess should be reworked.

"
 There are several reasons to choose YANG to define the data model. Data
defined using YANG can be exposed by a server to client applications and
controllers via Netconf [RFC6241] or via a ReST Interface
[I-D.draft-ietf-netconf-restconf] [I-D.draft-ietf-netmod-yang-json]. The
fact that it can be used with different protocols and interfaces provides
for a degree of "future- proofing" of model implementations. Also, YANG can
serve as the basis for model-driven toolchains, such as used in the Open
Daylight project.

"

I would suggest to explain a bit what is a model-driven tool chain instead
of citing a tool that is based on a such.
Or just put a reference to draft-ietf-i2rs-yang-network-topo where the
topic is explained much better.
In that one could also be added the suggestions of Martin on different
approaches on handling dynamic operational state with static config state.

This in particular was always an issue for me as a network service
fulfillment guy. It is a big debate in network services provisioning
should I consider the configuration as a guideline or the current state and
how to match those well especially in heterogeneous networks.

I always boils down to matching config with a bunch of state data coming
from SNMP which is typically a horrible stuff. Hope i2rs to solve me that
kind of an issues :)
Also in that direction in OSS people typically use SID for modeling
networks, resources (logical or physical) and the services provided by
them.
Did anybody considered any kind of a mapping between the yang models and
the SID kind of models?

Nikolay

-- 
BR,

Nikolay Milovanov
Network Engineer
Email: [email-masked]

---

From: Igor Bryskin <[email-masked]>
Date: Fri, 23 Oct 2015 13:00:06 +0000

Hi Martin,

It is possible that a user configures a link of a customized (user defined) topology providing also the path defined in underlay server defined topology. Said underlay path needs to be considered as intended configuration. Actual underlay path will be state data in this case ( personally I also do not see a difference between state and server provided data), which may or may not match the intended underlay path. The actual underlay path may also change over time due, for example, network failure restoration procedures affecting the underlay topology.
The link of the user defined topology is not invalidated in this case. User just has to deal with the fact that the intended configuration may not necessarily match the actual configuration and state data.

Cheers,
Igor

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Alexander Clemm (alex)
Sent: Thursday, October 22, 2015 8:12 PM
To: Martin Bjorklund <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Martin,

--- Alex

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Monday, October 19, 2015 11:04 PM

.....

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Sat, 24 Oct 2015 01:15:55 +0000

Hi Martin,

maybe option 1 provides a way out here.  This also has the least impact on the existing design and implementation, hence preferable.  

However, I still feel that we are dealing with a limitation of the YANG framework here.  I think we are dealing with a use case that was not really foreseen in the YANG design, i.e. that we might run into data that has instances that can indeed be authoritatively owned by a server versus others representing more traditional configuration.  The one aspect that not really address by making it regular config concerns your assertion that "any other client can modify what this client wrote".  Access rights could provide a solution, but not in the way as currently defined via NACM, since we would need to differentiate access rights at the instance level, not at the module definition level.  Rather than seeing how we can make our requirements somehow fit a rigid interpretation of the YANG framework (that does not allow for special treatment / behavior of special cases), I would like to see whether the YANG framework can be flexible enough to still support what we are trying to do.  

Kent made what I thought was a very interesting suggestion at the interim, asking whether this would be an application for metadata.  I think this is something that we might leverage here.  We could introduce a metadata item that indicates for configuration information whether that was populated by a server, so really should not be modified by other clients (or, when modified, will presumably be "changed back" by the server).  Or, that might simply be locked by the server.  

Thoughts regarding that suggestion?

--- Alex

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Friday, October 23, 2015 12:17 AM
To: Alexander Clemm (alex) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

And it seems Juergen is correct.  You essentially want to mix config and operational state in a single list.  This is not possible to express in YANG, so you change the semantics of the formal statements with descriptions.  I don't think this a good idea.

I can think of two alternatives to the current design that don't violate YANG:

/martin

> > true", or moving it outside (just this leaf) for a list that 
> > indicates for each topology whether it is server-provided or not (in 
> > a separate "state" branch).
> > --- Alex
> > 
> > > [Of course, we would not want any random client app just being 
> > > able to "mess" with that data.  The expectation is generally 
> > > clearly access to this will be restricted / controlled.  The 
> > > topology instances that were populated by the "server-provided 
> > > app" should not be "touched" by other apps - it is the 
> > > "server-provided" app that is responsible for maintaining them.]
> > > 
> > > > Note that this is not the case in the current model.  The 
> > > > current model is essentially all configuration data.  Of course, 
> > > > we have this flag to indicate who supplied that data (and is 
> > > > hence maintaining it).
> > > > 
> > > > However, doing that introduces precisely new issues - and you 
> > > > have just brought another argument why this may be a bad idea 
> > > > and may not even work.
> > > > in the model.  Now, if we have a topology that is 
> > > > server-provided, hence (per your statement) to be represented by 
> > > > operational data only, how do we build an overlay topology that 
> > > > is "configured" on top of it?  If the answer is "we can't, 
> > > > unless we replicate the server-provided topology into the 
> > > > network configuration (which makes no sense)", we are screwed.  
> > > > Now, we might build it on top if we remove all references / 
> > > > dependencies on the underlay from the model and punt the problem 
> > > > to the user.  Basically, no longer have the model express 
> > > > (discovered/server-provided vs configured), is a requirement.  
> > > > It is not an artefact of our model, it is something that we need 
> > > > to capture as part of the model.  There may not be a "nice" way 
> > > > of doing this within the YANG framework, yet it is important 
> > > > that we find a way to do this.  The current solution to this
> > > > "server-provided" object to address your concerns?  We can make 
> > > > it config (to address your issue that triggered this, the 
> > > > presence of a r/o object in a tree that is otherwise r/w).
> > > > 
> > > > Thoughts?
> > > > --- Alex
> > > > 
> > > > 
> > > > 
> > > > <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Susan 
> > > > Hares <[email-masked]>
> > > > > On Fri, Oct 09, 2015 at 09:55:19PM +0000, Alexander Clemm 
> > > > > (alex)
> > > > > wrote:
> > > > > > 
> > > > > > that was also discussed (I am not sure if this is what you 
> > > > > > are referring to).
> > > > > > one that is in effect etc .  This concerns the issue that 
> > > > > > some topologies are populated by the server whereas some 
> > > > > > topologies can be populated by client applications.
> > > > > 
> > > > > This is the normal way to do this in YANG. And this goes back 
> > > > > to what was driving us for years, namely to clearly separate 
> > > > > config from state. This module makes this distinction a 
> > > > > runtime property controlled by a data model specific 
> > > > > mechanism. None of the generic tools out there will be able to understand this.
> > > > > 
> > > > > <ALEX>
> > > > > I think the issue is more related to the current discussion 
> > > > > with regards to openconfig and "intended configuration" and 
> > > > > "applied configuration".  If YANG had an existing solution for 
> > > > > this, we would not have this discussion.  The reason I believe 
> > > > > this is similar is that you can view the "applied 
> > > > > configuration" as the "server-provided configuration" (network 
> > > > > topology, in our case), and the "intended configuration" as 
> > > > > the, well, configured or intended network topology in our 
> > > > > case.  That said, the issue is not identical
> > > > > - you can have "applied" [network topologies] that were 
> > > > > provided by the server / the network itself, not configured by anybody.
> > > > > </ALEX>
> > > > 
> > > > > > (this is not your normal config vs oper data distinction, 
> > > > > > but would essentially reflect a limitation of the 
> > > > > > framework), leading to unnecessary additional complexity in 
> > > > > > the model (every augmentation has to be conducted in two 
> > > > So you have operational state which is partially populated by 
> > > > the server and partially populated from config. I fail to see 
> > > > how this is any different from other cases, including network 
> > > > interfaces as defined in RFC 7233. Recall also that YANG does 
> > > > not allow configuration data to depend on state data.
> > > > 
> > > > > network is supported by a node in an underlay network.  For 
> > > > > this purpose, we are referencing a node in another (underlay) network.
> > > > > If we were to split the model, when we configure a node, we 
> > > > > will have to account for the fact that the supporting node 
> > > > > could be either part of a "configured" network itself, or of a 
> > > > > network that has been "server-provided".  That is, we need to 
> > > > > be able to allow for both possibilities.
> > > > 
> > > > > network ("branch 2").  As a result, we will have to introduce 
> > > > > a separate way to reference elements in list supporting-node.  
> > > > > All of this results in considerable additional complexity.  Or 
> > > > > do you see an easier way?
> > > > > 
> > > > > </ALEX>
> > > > 
> > > > 7.5.3 and section 7.19.5. And concerning leafref, section 9.9 
> > > > says that a leafref may only point to configuration. I believe 
> > > > this I-D is violating the distinction between configuration and state data.
> > > > 
> > > > /js
> > > > 
> > > > --

---

From: "Russ White" <[email-masked]>
Date: Sun, 25 Oct 2015 18:34:24 -0400

> However, I still feel that we are dealing with a limitation of the YANG
> framework here.  I think we are dealing with a use case that was not
really
> foreseen in the YANG design, i.e. that we might run into data that has
> instances that can indeed be authoritatively owned by a server versus
others
> representing more traditional configuration.  

I've not followed this entire thread, but I'm a bit baffled by the
discussion at this point. I thought I2RS was based on an atomic/RESTful
model, rather than being transactional. What's being discussed here, though,
smells a lot like a transaction based model -- one device sets some part of
the "configuration" (BTW, I object to the term "configuration" in the first
place, as I2RS is not dealing with configuration, but rather ephemeral state
-- completely different things) in such a way that no other controller can
change it.

This all seems contrary to the spirit of what we're working on here... There
should be priorities to understand which controller/agent has priority over
the others, and there should be callbacks when your piece of state is
overwritten by some other device (whether local or another controller), but
there shouldn't be this concept of a "completed transaction" that we seem to
be talking about here.

I get the distinct feeling we're trying to boil the ocean once again -- I2RS
is not going to configure the entire device (minus some trivial stuff no-one
cares about). I2RS should not be trying to build a fully transactional
database on each device, with locks and such. This should be fast and light,
operating at the speed of any other routing protocol (on the order of
milliseconds), not on the order of minutes/days/weeks/months. 

Can someone please explain why we're talking about this sort of
"configuration?"

Russ

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Oct 2015 19:18:34 +0100

In YANG, it is assumed that config true nodes exist in a datastore
where all config true nodes are treated the same. A datastore is the
unit of validation and we don't want to have a valid configuration
datatore become invalid arbitrarily. This is why config true nodes are
not allowed to depend on config false nodes (state data that can
change arbitrarily).

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Oct 2015 19:24:59 +0100

Alex,

I can have config that applies to a node named 'foo'. If the node
named 'foo' does not exist, the config will not be applied but it
remains valid config. This is how we deal with interfaces. You write
config for an interface with a certain name. As long as a
corresponding interface does not exist, this config will be ignored.
The point of this exercise is that it decouples the lifetime and
validity of the config from the lifetime of something that may change
arbitrarily.

/js

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Mon, 26 Oct 2015 19:10:56 +0000

Hi Juergen,
Understood, but this doesn't really address our problem here.  We want to see how we can apply YANG to meet our needs, not the other way round.  Hence the suggestion to explore the use of metadata here.
--- Alex

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Monday, October 26, 2015 11:19 AM
To: Alexander Clemm (alex) <[email-masked]>
Cc: Martin Bjorklund <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; Kent Watsen <[email-masked]>
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

In YANG, it is assumed that config true nodes exist in a datastore where all config true nodes are treated the same. A datastore is the unit of validation and we don't want to have a valid configuration datatore become invalid arbitrarily. This is why config true nodes are not allowed to depend on config false nodes (state data that can change arbitrarily).

/js

On Sat, Oct 24, 2015 at 01:15:55AM +0000, Alexander Clemm (alex) wrote:
> > express is clear.  You can have an outside client application 
> > maintain some topologies / list elements, and have others maintained 
> > an populated by the server - or arguably an app embedded in the server.
> > want the server / embedded app that is the authoritative owner of 
> > the information.
> > 
> > Cheers
> > --- Alex
> > 
> > > "regular" configuration), the server provided leaf is supposed to 
> > > be set to false.
> > > 
> > > indicates for each topology whether it is server-provided or not 
> > > (in a separate "state" branch).
> > > --- Alex
> > > 
> > > > One model for the data that is server-provided is to assume an 
> > > > app (which could be embedded on the same server) that knows how 
> > > > to discover the network, then populates the data accordingly.
> > > > 
> > > > Is the idea that the server-provided data is normal config?  
> > > > I.e., if the server wants to modify this data it behaves like a 
> > > > normal client?
> > > > > current model is essentially all configuration data.  Of 
> > > > > course, we have this flag to indicate who supplied that data 
> > > > > (and is hence maintaining it).
> > > > > 
> > > > > You argue that we should instead "split" the model and 
> > > > > introduce operational data to reflect what is populated by the server.
> > > > > Topologies _are_ layered, and we need to be able to express 
> > > > > that in the model.  Now, if we have a topology that is 
> > > > > server-provided, hence (per your statement) to be represented 
> > > > > by operational data only, how do we build an overlay topology 
> > > > > that is "configured" on top of it?  If the answer is "we 
> > > > > can't, unless we replicate the server-provided topology into 
> > > > > the network configuration (which makes no sense)", we are screwed.
> > > > > dependencies on the underlay from the model and punt the 
> > > > > problem to the user.  Basically, no longer have the model 
> > > > > express vertical relationships.  Not a good solution, IMHO.
> > > > > 
> > > > > layering relationships between topologies, including cases 
> > > > > where topologies originate from different sources 
> > > > > It is not an artefact of our model, it is something that we 
> > > > > need to capture as part of the model.  There may not be a 
> > > > > "nice" way of doing this within the YANG framework, yet it is 
> > > > > "server-provided" object to address your concerns?  We can 
> > > > > make it config (to address your issue that triggered this, the 
> > > > > On Thu, Oct 15, 2015 at 10:59:31PM +0000, Alexander Clemm 
> > > > > (alex)
> > > > > wrote:
> > > > > > Hello Juergen,
> > > > > > 
> > > > > > > The only item in the topology that is read-only concerns 
> > > > > > > the "server-provided" flag, but this concerns a separate 
> > > > > > > issue that was also discussed (I am not sure if this is 
> > > > > > > what you are referring to).
> > > > > > > distinguishing configuration that has been intended, 
> > > > > > > versus one that is in effect etc .  This concerns the 
> > > > > > > issue that some topologies are populated by the server 
> > > > > > > whereas some topologies can be populated by client applications.
> > > > > > 
> > > > > > > up", having a (rw) branch to populate "intended" 
> > > > > > > topologies and a
> > > > > > This is the normal way to do this in YANG. And this goes 
> > > > > > back to what was driving us for years, namely to clearly 
> > > > > > separate config from state. This module makes this 
> > > > > > distinction a runtime property controlled by a data model 
> > > > > > specific mechanism. None of the generic tools out there will be able to understand this.
> > > > > > 
> > > > > > <ALEX>
> > > > > > "applied configuration".  If YANG had an existing solution 
> > > > > > for this, we would not have this discussion.  The reason I 
> > > > > > believe this is similar is that you can view the "applied 
> > > > > > configuration" as the "server-provided configuration" 
> > > > > > (network topology, in our case), and the "intended 
> > > > > > configuration" as the, well, configured or intended network 
> > > > > > topology in our case.  That said, the issue is not identical
> > > > > > has an accompanying "intended configuration", in our case 
> > > > > > this is not necessarily the case
> > > > > Your 'server supplied topology' appears to me to be 
> > > > > operational state and not configuration data.
> > > > >  
> > > > > > > information would essentially be duplicated inside the 
> > > > > > > model (this is not your normal config vs oper data 
> > > > > > > distinction, but would essentially reflect a limitation of 
> > > > > > > the framework), leading to unnecessary additional 
> > > > > > > complexity in the model (every augmentation has to be 
> > > > > > > conducted in two places), more complex validation rules, etc.
> > > > > > 
> > > > > > 7233 - separate trees with distinct data, some clearly part 
> > > > > > of configuration, other clearly operational data.
> > > > > > In this case, this categorization does not always apply - 
> > > > > > or, the categorization depends on the particular instance.
> > > > > > </ALEX>
> > > > > 
> > > > > > We want to be able to express that, for example, a node in 
> > > > > > one network is supported by a node in an underlay network.  
> > > > > > For this purpose, we are referencing a node in another (underlay) network.
> > > > > > So that we cannot reference an arbitrary node in an 
> > > > > > arbitrary network, we want to make sure that the supporting 
> > > > > > node is part of a "supporting-network"
> > > > > > of the same network.
> > > > > > 
> > > > > > could be either part of a "configured" network itself, or of 
> > > > > > a network that has been "server-provided".  That is, we need 
> > > > > > to be able to allow for both possibilities.
> > > > > 
> > > > > > To do this, we would no longer be able to have the 
> > > > > > network-ref to be part of the key for supporting-node - we 
> > > > > > would have to replace network-ref with a choice of two nodes 
> > > > > > that reference either a server-provided network ("branch 
> > > > > > 1"), or a configured network ("branch 2").  As a result, we 
> > > > > > will have to introduce a separate way to reference elements in list supporting-node.
> > > > > > All of this results in considerable additional complexity.  
> > > > > > Or do you see an easier way?
> > > > > > 
> > > > > > </ALEX>
> > > > > 
> > > > > constraints on config true nodes can only refer to other 
> > > > > config true nodes in the datastore where the node with the 
> > > > > constraint exists. See section

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Oct 2015 20:13:07 +0100

I do not think metadata changes the way configuration datastore
validation works.

/js

---

From: Igor Bryskin <[email-masked]>
Date: Fri, 23 Oct 2015 20:51:26 +0000

Gert,

You are right. When the user configures an abstract link specifying the underlay topology path, the user essentially provides a set of inclusion constraints for the trail supporting the link. Although it is not reflected yet in the TE-TOPO model, it should be possible to specify whether it is acceptable or not to deviate from said constraints on the trail setup and/or during its lifetime (e.g. in case of restoration). One server side implementation you and I are aware of has this capability already.

Kind regards,
Igor

-----Original Message-----
From: Gert Grammel [mailto:[email-masked]] 
Sent: Friday, October 23, 2015 4:16 PM
To: Igor Bryskin <[email-masked]>
Cc: Alexander Clemm (alex) <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Igor,

It would be advisable to let the client decide whether a different underlay shall invalidate the intended config. 

Best

Gert

Sent from my Apple ][

> On 23 Oct 2015, at 15:00, Igor Bryskin <[email-masked]> wrote:
> 
> Hi Martin,
>

---

From: "Alexander Clemm (alex)" <[email-masked]>
Date: Sat, 24 Oct 2015 00:57:57 +0000

This is an interesting suggestion.  Taking this a little further rises the question how this would be represented. 
One option would be the introduction of another flag indicating the "validation policy" that should be applied.  Or, perhaps preferrably, introducing a separate state (can be in a separate subtree) to indicate the current integrity state with regards to layering relationships, where any changes in the underlay that have not been properly tracked in configured layering relationships would be tracked/flagged.  
--- Alex

-----Original Message-----
From: Igor Bryskin [mailto:[email-masked]] 
Sent: Friday, October 23, 2015 1:51 PM
To: Gert Grammel <[email-masked]>
Cc: Alexander Clemm (alex) <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [i2rs] WG LC for Topology (10/1 to 10/14)

Gert,

Kind regards,
Igor

-----Original Message-----
From: Gert Grammel [mailto:[email-masked]] 
Sent: Friday, October 23, 2015 4:16 PM
To: Igor Bryskin <[email-masked]>
Cc: Alexander Clemm (alex) <[email-masked]>; Martin Bjorklund <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] WG LC for Topology (10/1 to 10/14)

Hi Igor,

Best

Gert

Sent from my Apple ][

---

From: "Russ White" <[email-masked]>
Date: Sun, 25 Oct 2015 18:16:09 -0400

really
others

Russ

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Oct 2015 12:29:51 -0700

Hi,

I agree that adding meta-data will have no affect on
a client that thinks it can edit config=true.  Especially
when the knob that overrides the semantics of the config-stmt
is in the configuration itself.

So why can't the interfaces + interfaces-state design be applied here?
Given that charter explicitly states read-only topology, why is this
even in there?

You can define objects in a grouping later.
It is OK to reorg objects into/out of groupings if the
same "final" syntax and semantics is maintained.

Andy

On Mon, Oct 26, 2015 at 12:13 PM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > Understood, but this doesn't really address our problem here.  We want
> to see how we can apply YANG to meet our needs, not the other way round.
> Hence the suggestion to explore the use of metadata here.
> > --- Alex
> >
> > Cc: Martin Bjorklund <[email-masked]>; [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]; [email-masked];
> Kent Watsen <[email-masked]>
> > In YANG, it is assumed that config true nodes exist in a datastore where
> all config true nodes are treated the same. A datastore is the unit of
> validation and we don't want to have a valid configuration datatore become
> invalid arbitrarily. This is why config true nodes are not allowed to
> depend on config false nodes (state data that can change arbitrarily).
> >
> > /js
> >
> > > maybe option 1 provides a way out here.  This also has the least
> impact on the existing design and implementation, hence preferable.
> > >
> > > However, I still feel that we are dealing with a limitation of the
> YANG framework here.  I think we are dealing with a use case that was not
> really foreseen in the YANG design, i.e. that we might run into data that
> has instances that can indeed be authoritatively owned by a server versus
> others representing more traditional configuration.  The one aspect that
> not really address by making it regular config concerns your assertion that
> "any other client can modify what this client wrote".  Access rights could
> provide a solution, but not in the way as currently defined via NACM, since
> we would need to differentiate access rights at the instance level, not at
> the module definition level.  Rather than seeing how we can make our
> requirements somehow fit a rigid interpretation of the YANG framework (that
> does not allow for special treatment / behavior of special cases), I would
> like to see whether the YANG framework can be flexible enough to still
> support what we are trying to do.
> > >
> > > Kent made what I thought was a very interesting suggestion at the
> interim, asking whether this would be an application for metadata.  I think
> this is something that we might leverage here.  We could introduce a
> metadata item that indicates for configuration information whether that was
> populated by a server, so really should not be modified by other clients
> (or, when modified, will presumably be "changed back" by the server).  Or,
> that might simply be locked by the server.
> > >
> and operational state in a single list.  This is not possible to express in
> YANG, so you change the semantics of the formal statements with
> descriptions.  I don't think this a good idea.
> > >

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 28 Oct 2015 03:22:17 +0000

Hi Alex, 

> Sent: Saturday, October 24, 2015 9:16 AM
> To: Martin Bjorklund
> Cc: [email-masked]; [email-masked]; Kent Watsen;
> maybe option 1 provides a way out here.  This also has the least impact on the
> existing design and implementation, hence preferable.
> 
> framework here.  I think we are dealing with a use case that was not really
> foreseen in the YANG design, i.e. that we might run into data that has instances
> that can indeed be authoritatively owned by a server versus others representing
> more traditional configuration.  The one aspect that not really address by
> making it regular config concerns your assertion that "any other client can
> modify what this client wrote".  Access rights could provide a solution, but not
> in the way as currently defined via NACM, since we would need to differentiate
> access rights at the instance level, not at the module definition level.  Rather
> than seeing how we can make our requirements somehow fit a rigid
> interpretation of the YANG framework (that does not allow for special
> treatment / behavior of special cases), I would like to see whether the YANG
> framework can be flexible enough to still support what we are trying to do.

I'd agree that representation of the layered architecture of network topology models/instances is different from traditional Yang models, thus we need to explore the appropriate solution under the Yang framework.

> Kent made what I thought was a very interesting suggestion at the interim,
> asking whether this would be an application for metadata.  I think this is
> something that we might leverage here.  We could introduce a metadata item
> that indicates for configuration information whether that was populated by a
> server, so really should not be modified by other clients (or, when modified, will
> presumably be "changed back" by the server).  Or, that might simply be locked
> by the server.

Metadata seems like a possible mechanism as it can specify instance-specific data. If metadata is used to indicate whether the instance is server-provided or client-provided, one question is does an instance with annotation "server-provided" still belong to configuration datastore?

Best regards,
Jie

> And it seems Juergen is correct.  You essentially want to mix config and
> operational state in a single list.  This is not possible to express in YANG, so you
> change the semantics of the formal statements with descriptions.  I don't think
> this a good idea.
>

---

From: Gert Grammel <[email-masked]>
Date: Fri, 23 Oct 2015 20:16:18 +0000

Hi Igor,

Best

Gert

Sent from my Apple ][