YANG Data Models in the Industry: Current State of Affairs (November 2017)

With a pragmatic SDN definition in mind, such that “SDN functionally enables the network to be accessed by operators programmatically, allowing for automated management and orchestration techniques; application of configuration policy across multiple routers, switches, and servers; and the decoupling of the application that performs these operations from the network device’s operating system.”, we’ve been executing on all of these requirements. Our IETF deliverables are: the YANG modeling language, protocols such as NETCONF or RESTCONF, encodings such as XML and JSON, and some YANG modules.

Just after IETF 100 in Singapore in November, let’s analyze the current state of affairs in the YANG Data Models world. Note also the previous “YANG Data Models in the Industry: Current State of Affairs” from a year and half ago.

Let me start with a reflection. How do we know we’ve been fighting this uphill battle long enough and that it’s all downhill from here? Is it when we have published a core set of YANG models? When the technology is implemented by vendors? When the technology is deployed by operators? When other Standard Development Organizations/consortia/open-source projects embrace this technology? Probably most (or even all) of the above.

Now, an anecdote from this last IETF meeting. I was in a bar, connecting with IETF friends when, at some point in time, the discussion centered around YANG. In the past I would have lead the discussion, trying to convince and influence the crowd, but this time it was not necessary. I was quietly and happily sipping my beer while, Alia (an IETF routing Area Director), debated the importance of YANG. I enjoyed that moment so much and I remember observing that it is a success when someone else does your job. When someone else makes your speech. Then you know you can safely pass the baton. Now, downhill does not mean that there are no more issues to resolve, so let’s review the YANG models state of affairs.

1. The Network Management Datastore Architecture (NMDA) Impact

We keep specifying YANG modules at the IETF. See the graphical evolution here and all the published YANG data modules here. Why does it take so long, you may ask? Well, the world of standardization is never fast enough as quality and consensus come at a price. However, in this particular case, the main reason we aren’t fast enough is that we’re busy finalizing the “Network Management Datastore Architecture“, a new way to design YANG modules. To help grasp the concepts of this architecture we can look at pieces of the draft:

Network management data objects can often take two different values, the value configured by the user or an application (configuration) and the value that the device is actually using (operational state). 

The original model of datastores required these data objects to be modeled twice in the YANG schema, as "config true" objects and as"config false" objects. The convention adopted by the interfaces data model ([RFC7223]) and the IP data model ([RFC7277]) was using two separate branches rooted at the root of the data tree, one branch for configuration data objects and one branch for operational state data objects. 

 The duplication of definitions and the ad-hoc separation of operational state data from configuration data leads to a number of problems. Having configuration and operational state data in separate branches in the data model is operationally complicated and impacts the readability of module definitions. Furthermore, the relationship between the branches is not machine readable and filter expressions operating on configuration and on related operational state are different.

With the revised architectural model of datastores defined in this document, the data objects are defined only once in the YANG schema but independent instantiations can appear in two different datastores, one for configured values and one for operational state values. This provides a more elegant and simpler solution to the problem.

To illustrate the NMDA principles, below is a comparison of the pyang tree for the RFC 7223 “Original Datastores Model” ietf-interfaces on the left and the new NMDA-compliant RFC7223bis draft ietf-interfaces on the right. With NMDA, the “intended” and “applied” values would be reported in different datastores and the link between those “intended” and “applied” values is now machine readable. This will lead to a “cleaner” tree definition. Indeed, the interfaces-state sub-tree disappears in the new YANG module.

Part of the delay induced by NMDA stems from the fact that some of the key YANG modules already produced need to be updated to be NMDA-compliant: ietf-interfaces RFC 7223, ietf-ip RFC 7277, ietf-routing RFC 8022, ietf-yang-library RFC 7895. The good news is this NMDA transition is almost over. Those RFC bis YANG modules, along with the Network Management Datastore Architecture document are now in last call, and most IETF YANG modules in development today are already NMDA-compliant.

An easy way to check if your YANG module is NMDA-compliant is to look it up in the YANG catalog metadata tool. For example, the report for the new ietf-interfaces YANG module is here: the following entry shows the NMDA compliance.

All possible values are described in this “YANG module for the YANG catalog” IETF draft: nmda-compatible, transitional-extra, openconfig, and unclassified.

YANG Modules Publication Soon

I like this definition of “Soon”, just for the fun of it! To be more serious, with Routing Area Directors, we evaluated the situation of most IETF YANG modules, as one of the wrap up meetings at IETF 100. Many are currently in last call and/or under YANG doctors review and will end up on the IESG plate soon. I’ll do my best to progress those before I step down as Area Director next March.

The Tooling and YANG Module Metadata Become More and More Important

Producing YANG modules is one step in the right direction, but having the right tools and the right YANG module metadata (like health metrics) becomes equally important. Instead of repeating myself, let me point you to the progress on that front, in this YANG Catalog Latest Developments (IETF 100 Hackathon) recent blog.

Collaboration Across Standard Development Organizations

During the IETF 100, we had an IEEE/IESG breakfast meeting with a primary topic of discussion: YANG. We obviously discussed the NMDA compliance for the IEEE YANG modules. A single query to the yangcatalog  produced the right output for our discussion: “return the latest version of all YANG modules where the organization is IEEE”.

From there, it’s easy to check whether each YANG module reports the tree-type metadata as “nmda-compatible”.

Next to IEEE, the Broadband Forum also inquired about the NMDA status, timeline, and possible transition in this liaison statement. With the NMDA architecture and the couple of key NMDA-complaint YANG modules being close to completion, it’s time to reply to those liaisons and also proactively warns the other SDOs, consortia, and open-source projects. Helping with the NMDA transition, we have to understand which specific YANG modules those SDOs care about (basically, which YANG modules they will augment) and work on a transition together; directly moving to the NMDA complaint version or still relying on the previous non-NMDA version?

YANG Module Update Procedure

YANG specifies strict rules [RFC7950] for updating YANG modules (when keeping the same YANG module name), imposing backward compatible changes. However, this causes some issues in the world of automation! For example, the YANG paths must be changed in controller/orchestration when an YANG module with a new YANG name is introduced. It is not possible to know that one YANG module obsolete or update another YANG module without going through a level of indirection that is the RFC document obsolete or update tag. The IETF, as openconfig did some time ago with the semver concept, faced the first occurrences of this issues. Some more background information in this IETF draft. which was discussed in the NETMOD Working Group.

A Lot of Telemetry Documents

Building on YANG, there was much work on YANG based Telemetry at IETF 100.  These successes include an award at the IETF Hackathon, the progression of six adopted working group drafts in NETCONF, the addition of four new proposals from new authors, and NETCONF closing in on Working Group Last Call on three of these drafts.

Based on the traction of the technology in the industry, it is also becoming relevant to working groups beyond NETCONF. Some progress includes:

• An addition to the charter of the Security and Continuous Monitoring (SACM) Working Group for discovering attacks in progress on Network Elements.
• The socialization of a new draft on how to push info from IoT devices within the Constrained RESTful Environments (CoRE) Working Group.
• Joint work with the Interface to Network Security Functions (I2NSF) Working Group to frame their drafts so they can use YANG-dbase Telemetry.