YANG-Push Operational Data Observability Enhancements
Internet-Draft draft-wilton-netconf-yp-observability-00
Robert Wilton
Cisco Systems
rwilton@cisco.com
21 October 2024
Operations and Management
Network Configuration
YANG Push
Observability
Network Telemetry
Operational Data
This early draft proposes some enhancements to YANG-Push to optimize its behavior for operational data telemetry. It also lists some additional issues that could potentially be discussed if there is further interest in this work, in particular whether we should be attempting extensions to YANG-Push (as this document is currently framed) or instead should attempt to define a new 'YANG-Push lite'.
About This Document
The latest revision of this draft can be found at https://rgwilton.github.io/draft-yp-observability/draft-wilton-netconf-yp-observability.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-wilton-netconf-yp-observability/.
Discussion of this document takes place on the Network Configuration Working Group mailing list (mailto:netconf@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/netconf/. Subscribe at https://www.ietf.org/mailman/listinfo/netconf/.
Source for this draft and an issue tracker can be found at https://github.com/rgwilton/draft-yp-observability.
Conventions and Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
Introduction
[I-D.ietf-nmop-yang-message-broker-integration] describes an architecture for how YANG-Push [RFC8641] can be integrated effectively with message brokers, e.g., [Kafka], that is part of a wider architecture for a Network Anomaly Detection Framework, specified in [I-D.ietf-nmop-network-anomaly-architecture].
YANG-Push is a key part of these architectures, but through experience of implementing YANG-Push specifically for the use cases described in the above architecture documents, it became clear that there are aspects of YANG-Push that are not optimal for these use cases for neither producer or consumer, particular as they relate to operational data.
For the consumer of the telemetry data, there is a requirement to associate a schema with the provided data. It is much more helpful for the schema to be associated with the individual messages rather than at the root of the operational datastore. As such, it is helpful for the encoded instance data to be rooted at the subscription path rather than at the schema root of the operational datastore.
Complexities in Modelling the Operational State Datastore
The YANG abstraction of a single datastore of related consistent data works very well for configuration that has a strong requirement to be self consistent, and that is always updated, and validated, in a transactional way. But for producers of telemetry data, the YANG abstraction of a single operational datastore is not really possible for devices managing a non-trivial quantity of operational data.
Some systems may store their operational data in a single logical database, yet it is less likely that the operational data can always be updated in a transactional way, and often for memory efficiency reasons such a database does not store individual leaves, but instead semi-consistent records of data at a container or list entry level.
For other systems, the operational information may be distributed across multiple internal nodes (e.g., linecards), and potentially many different process daemons within those distributed nodes. Such systems generally cannot exhibit full consistency [Consistency] of the operational data (which would require transactional semantics across all daemons and internal nodes), only offering an eventually consistent [Eventual_Consistency] view of the data instead.
In practice, many network devices will manage their operational data as a combination of some data being stored in a central operational datastore, and other, higher scale, and potentially more frequently changing data (e.g., statistics or FIB information) being stored elsewhere in a more memory efficient and performant way.
YANG-Push enhancements
To address the needs described in the introduction and architecture documents, this document defines some minor extensions to YANG-Push that are designed to make YANG-Push work better both for producers and consumers of YANG telemetry data.
Currently, it:
- defines a new YANG-Push encoding format that can be used for both on-change and periodic subscriptions that reports the data from the subscription filter point.
- defines a combined periodic and on-change subscription that reports events both on a periodical cadence and also if changes to the data have occurred.
These are detailed in the following sections:
New encoding format
This document proposes a new opt-in YANG-Push encoding format to use instead of the "push-update" and "push-change-update" notifications defined in [RFC8641].
There are a few reasons for specifying a new encoding format:
1. To use the same encoding format for both periodic and on-change messages, allowing the same messages to be easily received and stored in a time-series database, making use of the same message schema when traversing message buses, such as Apache Kafka.
2. To allow the schema of the notifications to be rooted to the subscription point rather than always being to the root of the operational datastore schema. This allows messages to be slightly less indented, and makes it easier to convert from a YANG schema to an equivalent message bus schema, where each message is defined with its own schema, rather than a single datastore schema.
3. To move away from the somewhat verbose YANG Patch format [RFC8072], that is not really a great fit for encoding changes of operational data. Many systems cannot necessarily distinguish between create versus update events (particularly for new subscriptions or after recovering from internal failures within the system), and hence cannot faithfully implement the full YANG Patch semantics defined in [RFC8641].
4. To allow the device to split a subscription into smaller child subscriptions for more efficient independent and concurrent processing. I.e., reusing the ideas from [I-D.ietf-netconf-distributed-notif]. However, all child subscriptions are still encoded from the same subscription point.
The practical differences in the encodings may be better illustrated via the examples in Appendix B.
Combined periodic and on-change subscription
Sometimes it is helpful to have a single subscription that covers both periodic and on-change notifications (perhaps with dampening).
There are two ways in which this may be useful:
1. For generally slow changing data (e.g., a device's physical inventory), then on-change notifications may be most appropriate. However, in case there is any lost notification that isn't always detected, for any reason, then it may also be helpful to have a slow cadence periodic backup notification of the data (e.g., once every 24 hours), to ensure that the management systems would should always eventually converge on the current state in the network.
2. For data that is generally polled on a periodic basis (e.g., once every 10 minutes) and put into a time series database, then it may be helpful for some data trees to also get more immediate notifications that the data has changed. Hence, a combined periodic and on-change subscription, potentially with some dampening, would facilitate more frequent notifications of changes of the state, to reduce the need of having to always wait for the next periodic event.
Hence, this document introduces the fairly intuitive "periodic-and-on-change" update trigger that creates a combined periodic and on-change subscription, and allows the same parameters to be configured. For some use cases, e.g., where a time-series database is being updated, the new encoding format proposed previously may be most useful.
Open Issues & Other Potential Enhancements/Changes
This section lists some other potential issues and enhancements that should be considered as part of this work. If there is working group interest in progressing this work, then the issues in this section could potentially be better managed as github issues.
1. Should we consider a version of the JSON encoding that excludes module prefixes (either everywhere, or perhaps only include the top module prefix). The reasoning for considering this is to potentially better align the JSON data with how the schema data may be modeled in other data systems, e.g., Kafka. Obviously, this requires that there be no duplicate data node names in different module namespaces, but most sane device schemas would avoid this anyway.
2. Do we make use of the new notification-envelope format [I-D.netana-netconf-notif-envelope] as the mandatory and only required notification format for these new forms of subscriptions. I.e., reducing complexity by removing unnecessary flexibility and options?
3. Document how sub-subscriptions can be used to split a higher level subscription into multiple smaller more efficient subscriptions for the device (that can be handled concurrently).
4. The document's current focus is on configured subscriptions, aligned to the proposed initial deployment requirements, but the solution should probably be extended to support dynamic subscriptions, presuming that it is not hard to do so.
5. Some of the YANG-Push behavior is more complex and expensive to implement (e.g., the SHOULD requirement to suggest suitable alternative subscription parameters if a subscription is rejected, subscription dependencies). Should this document update RFC 8639 or RFC 8641 to indicate that those requirements do not apply to these new extended subscriptions? The goal of this work should be to specify the minimal required functionality to meet the requirements.
6. What document format should this work take? The currently proposed approach is to add extra extensions to YANG-Push to cover the required functionality. An alternative approach could be to write a RFC 8641-bis, or a 'YANG-Push lite'.
7. Currently the encoding and transport parameters are per subscription, but it may make more sense for these to be per receiver definition. I.e., if you want to use different transports and encodings to the same receiver this should still be possible, but would require a second receiver to be defined with the same destination IP address, but a different name. Currently, the newly proposed encoding format is configured per subscription (mirroring equivalent transport and encoding configuration), but alternatively it could be configured per receiver.
8. We should consider how a subscription could support multiple subscription paths. The potential tricky aspect of this to consider the subscription bind point. Related to this is whether XPath 1.0 is the best way of specifying these bind points, or whether it should something closer to the NACM node-instance-identifier [RFC6536], but perhaps using something closer to the JSON style encoding of instance identifier [RFC7951], section 6.11; or JSON PATH [RFC9535].
9. What level of subscription filtering do we need and want to support? For example, I doubt that anyone allows for full XPath filtering of operational data subscriptions because they are likely to be very computationally expensive to implement. Is there an easier way of expressing the filter requirements rather than using subtree filtering. Note, this could be added in a future release.
YANG Extensions Data Model
This sections shows the raw YANG tree output just for the ietf-yp-ext.yang module, and the proposed YANG module.
YANG Tree
YANG tree for ietf-yp-ext.yang

module: ietf-yp-ext

  augment /sn:subscription-started/yp:update-trigger:
    +--:(periodic-and-on-change) {yp:on-change}?
       +-- periodic-and-on-change!
          +-- period              yp:centiseconds
          +-- anchor-time?        yang:date-and-time
          +-- dampening-period?   yp:centiseconds
          +-- sync-on-start?      boolean
          +-- excluded-change*    yp:change-type
  augment /sn:subscription-started:
    +--ro common-notification-format?   boolean
  augment /sn:subscription-modified/yp:update-trigger:
    +--:(periodic-and-on-change) {yp:on-change}?
       +-- periodic-and-on-change!
          +-- period              yp:centiseconds
          +-- anchor-time?        yang:date-and-time
          +-- dampening-period?   yp:centiseconds
          +-- sync-on-start?      boolean
          +-- excluded-change*    yp:change-type
  augment /sn:subscription-modified:
    +--ro common-notification-format?   boolean
  augment /sn:subscriptions/sn:subscription/yp:update-trigger:
    +--:(periodic-and-on-change) {yp:on-change}?
       +--rw periodic-and-on-change!
          +--rw period              yp:centiseconds
          +--rw anchor-time?        yang:date-and-time
          +--rw dampening-period?   yp:centiseconds
          +--rw sync-on-start?      boolean
          +--rw excluded-change*    yp:change-type
  augment /sn:subscriptions/sn:subscription:
    +--rw common-notification-format?   boolean

  notifications:
    +---n update
       +--ro id?                   sn:subscription-id
       +--ro subscription-path?    yang:xpath1.0
       +--ro target-path?          string
       +--ro snapshot-type?        enumeration
       +--ro observation-time?     yang:date-and-time
       +--ro datastore-snapshot?   <anydata>
       +--ro incomplete?           empty

YANG file
YANG module ietf-yp-ext

module ietf-yp-ext {
  yang-version 1.1;
  namespace "urn:ietf:params:xml:ns:yang:ietf-yp-ext";
  prefix yp-ext;

  import ietf-yang-types {
    prefix yang;
    reference
      "RFC 6991: Common YANG Data Types";
  }
  import ietf-subscribed-notifications {
    prefix sn;
    reference
      "RFC 8639: Subscription to YANG Notifications";
  }
  import ietf-yang-push {
    prefix yp;
    reference
      "RFC 8641: Subscription to YANG Datastores";
  }

  organization
    "IETF NETCONF (Network Configuration) Working Group";
  contact
    "WG Web:  <https:/datatracker.ietf.org/wg/netconf/>
     WG List: <mailto:netconf@ietf.org>

     Author:  Robert Wilton
              <mailto:rwilton@cisco.com>";

  description
    "This module contains YANG specifications for YANG-Push extensions.

     The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL
     NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'NOT RECOMMENDED',
     'MAY', and 'OPTIONAL' in this document are to be interpreted as
     described in BCP 14 (RFC 2119) (RFC 8174) when, and only when,
     they appear in all capitals, as shown here.

     Copyright (c) 2024 IETF Trust and the persons identified as
     authors of the code.  All rights reserved.

     Redistribution and use in source and binary forms, with or
     without modification, is permitted pursuant to, and subject to
     the license terms contained in, the Simplified BSD License set
     forth in Section 4.c of the IETF Trust's Legal Provisions
     Relating to IETF Documents
     (https://trustee.ietf.org/license-info).

     This version of this YANG module is part of RFC 8641; see the
     RFC itself for full legal notices.";

  revision 2024-10-18 {
    description
      "Initial revision.";
    reference
      "XXX: YANG Push Observability Extensions ";
  }

  /*
   * FEATURES
   */

  /*
   * IDENTITIES
   */

  /*
   * TYPE DEFINITIONS
   */

  /*
   * GROUP DEFINITIONS
   */

  grouping periodic-and-on-change {
    description
      "This grouping describes the datastore-specific subscription
       that represents a combined periodic and on-change subscription.";
    container periodic-and-on-change {
      presence
        "A combined periodic and on-change subscription";
      description
        "The publish notifies the receiver on the specific
          periodic, and also whenever any of the data changes.";
      leaf period {
        type yp:centiseconds;
        mandatory true;
        description
          "Duration of time that should occur between periodic
          push updates, in units of 0.01 seconds.";
      }
      leaf anchor-time {
        type yang:date-and-time;
        description
          "Designates a timestamp before or after which a series
          of periodic push updates are determined.  The next
          update will take place at a point in time that is a
          multiple of a period from the 'anchor-time'.
          For example, for an 'anchor-time' that is set for the
          top of a particular minute and a period interval of a
          minute, updates will be sent at the top of every
          minute that this subscription is active.";
      }
      leaf dampening-period {
        type yp:centiseconds;
        default "0";
        description
          "Specifies the minimum interval between the assembly of
            successive update records for a single receiver of a
            subscription.  Whenever subscribed objects change and
            a dampening-period interval (which may be zero) has
            elapsed since the previous update record creation for
            a receiver, any subscribed objects and properties
            that have changed since the previous update record
            will have their current values marshalled and placed
            in a new update record.";
      }
      leaf sync-on-start {
        type boolean;
        default "true";
        description
          "When this object is set to 'false', (1) it restricts an
          on-change subscription from sending 'push-update'
          notifications and (2) pushing a full selection per the
          terms of the selection filter MUST NOT be done for
          this subscription.  Only updates about changes
          (i.e., only 'push-change-update' notifications)
          are sent.  When set to 'true' (the default behavior),
          in order to facilitate a receiver's synchronization,
          a full update is sent, via a 'push-update' notification,
          when the subscription starts.  After that,
          'push-change-update' notifications are exclusively sent,
          unless the publisher chooses to resync the subscription
          via a new 'push-update' notification.";
      }
      leaf-list excluded-change {
        type yp:change-type;
        description
          "Used to restrict which changes trigger an update.  For
           example, if a 'replace' operation is excluded, only the
           creation and deletion of objects are reported.

           TODO - Do we want this leaf?";
      }
    }
  }

  grouping common-notification-format {
    description
      "This grouping describes the common-notif-format leaf that
       controls whether the new common message format is used for periodic
       and on-change notifications.";
    leaf common-notification-format {
      type boolean;
      default "false";
      description
        "This leaf controls whether the new common message format
         is used for periodic and on-change notifications.  If set
         to 'true', the new common message format is used.  If set
         to 'false', the old message format is used.

         TODO - Should we support hierarchical configuration here
         (e.g., also add the leaf to the subscription container,
         or it could be added to the receiver instead)?";
    }
  }

  /*
   * RPCs
   */

  /*
   * NOTIFICATIONS
   */

  notification update {
    description
      "This notification contains a push update that in turn contains
       data subscribed to via a subscription.  In the case of a
       periodic subscription, this notification is sent for periodic
       updates.  It can also be used for synchronization updates of
       an on-change subscription.  This notification shall only be
       sent to receivers of a subscription.  It does not constitute
       a general-purpose notification that would be subscribable as
       part of the NETCONF event stream by any receiver.";
    leaf id {
      type sn:subscription-id;
      description
        "This references the subscription that drove the
         notification to be sent.";
    }

    leaf subscription-path {
      type yang:xpath1.0;
      description
        "The subscription path indicates the schema path from the root of the
         schema tree to the node, for which the updated state is being
         notified, or a notification that is has been deleted.

         TODO - Make this an instance data ref";
    }

    leaf target-path {
      type string;
    }

    leaf snapshot-type {
      type enumeration {
        enum "periodic" {
          description
            "The update message is due to a periodic update.";
        }
        enum "on-change-update" {
          description
            "The update message is due to an on-change update.  This
             means that one or more fields have changed under the
             snapshot path.

             TODO - Split this into a on-change-delete msg?";
        }
        enum "on-change-delete" {
          description
            "The update message is due to an on-change event where
             the data node at the target path has been delete.";
        }
        enum "resync" {
          description
            "This indicates that the update is to resynchronize the
             state, e.g., after a subscription started notification.

             Ideally, the resync message SHOULD be the first
             notification sent when a subscription has started, but
             it is not gauranteed or required to be the first
             (e.g., if an on-change event occurs).

             These messages can be used to ensure that all state
             has been sent to the client, and can be used to purge
             stale data.

             TODO - In the distributed notification case, need a
             notification to indicate that all child subscriptions
             have been sent.";
        }
      }
      description
        "This indicates the type of notification message that is being sent.";
    }

    // Could add observation time here.
    leaf observation-time {
      type yang:date-and-time;
      description
        "The time that the update was observed by the publisher.";
    }

    anydata datastore-snapshot {
      description
        "This contains the updated data.  It constitutes a snapshot
        at the time of update of the set of data that has been
        subscribed to.  The snapshot corresponds to the same
        snapshot that would be returned in a corresponding 'get'
        operation with the same selection filter parameters
        applied.

        Used for snapshot types except for 'on-change-delete'.";
    }

    leaf incomplete {
      type empty;
      description
        "This is a flag that indicates that not all datastore
        nodes subscribed to are included with this update.  Receivers
        of this data SHOULD NOT assume that any missing data has been
        implicitly deleted.

        TODO - Do we still want this flag, or would it be
        better to notify this event out of band?";
    }
  }

  augment "/sn:subscription-started/yp:update-trigger" {
    if-feature "yp:on-change";
    description
      "Allow a subscription started notification to signal a
       combined periodic and on-change subscription.";
    case periodic-and-on-change {
      uses periodic-and-on-change;
    }
  }

  augment "/sn:subscription-started" {
    description
      "Allow a subscription started notification to signal that it
       uses the common notification format.";
    uses common-notification-format;
  }

  augment "/sn:subscription-modified/yp:update-trigger" {
    if-feature "yp:on-change";
    description
      "Allow a subscription started notification to signal a
       combined periodic and on-change subscription.";
    case periodic-and-on-change {
      uses periodic-and-on-change;
    }
  }

  augment "/sn:subscription-modified" {
    description
      "Allow a subscription modified notification to signal that it
       uses the common notification format.";
    uses common-notification-format;
  }

  /*
   * DATA NODES
   */

  augment "/sn:subscriptions/sn:subscription/yp:update-trigger" {
    when 'yp:datastore';
    if-feature "yp:on-change";
    description
      "This augmentation adds objects to a subscription that are
       specific to a datastore subscription, i.e., a subscription to
       a stream of datastore node updates.";
    case periodic-and-on-change {
      uses periodic-and-on-change;
    }
  }

  augment "/sn:subscriptions/sn:subscription" {
    uses common-notification-format;
  }
}

Security Considerations
TODO. New YANG models will be defined that need to document their security considerations, but otherwise the security considerations in YANG-Push should be sufficient.
IANA Considerations
TODO - This document will need to register new YANG models with IANA.
Acknowledgments
This inital draft is early work is based on discussions with various folk, particularly Thomas Graf, Holger Keller, Dan Voyer, Nils Warnke, and Alex Huang Feng; but also wider conversations that include: Benoit Claise, Pierre Francois, Paolo Lucente, Jean Quilbeuf, among others.
References
Normative References
[RFC8641]

Subscription to YANG Notifications for Datastore Updates
A. Clemm
E. Voit
September 2019
RFC 8641
DOI 10.17487/RFC8641
[RFC2119]

Key words for use in RFCs to Indicate Requirement Levels
S. Bradner
March 1997
BCP 14
RFC 2119
DOI 10.17487/RFC2119
[RFC8174]

Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words
B. Leiba
May 2017
BCP 14
RFC 8174
DOI 10.17487/RFC8174
[RFC5277]

NETCONF Event Notifications
S. Chisholm
H. Trevino
July 2008
RFC 5277
DOI 10.17487/RFC5277
Informative References
[I-D.ietf-nmop-network-anomaly-architecture]

An Architecture for a Network Anomaly Detection Framework
Thomas Graf
Swisscom
Wanting Du
Swisscom
Pierre Francois
INSA-Lyon
20 October 2024
Internet-Draft draft-ietf-nmop-network-anomaly-architecture-01
[I-D.ietf-nmop-yang-message-broker-integration]

An Architecture for YANG-Push to Message Broker Integration
Thomas Graf
Swisscom
Ahmed Elhassany
Swisscom
19 October 2024
Internet-Draft draft-ietf-nmop-yang-message-broker-integration-05
[Kafka]

Apache Kafka
Apache.org
[Consistency]

Consistency (database systems)
Wikipedia
[Eventual_Consistency]

Eventual Consistency
Margaret Rouse
[I-D.ietf-netconf-distributed-notif]

Subscription to Distributed Notifications
Tianran Zhou
Huawei
Guangying Zheng
Huawei
Eric Voit
Cisco Systems
Thomas Graf
Swisscom
Pierre Francois
INSA-Lyon
18 October 2024
Internet-Draft draft-ietf-netconf-distributed-notif-10
[I-D.netana-netconf-notif-envelope]

Extensible YANG model for YANG-Push Notifications
Alex Huang Feng
INSA-Lyon
Pierre Francois
INSA-Lyon
Thomas Graf
Swisscom
Benoît Claise
Huawei
18 October 2024
Internet-Draft draft-netana-netconf-notif-envelope-00
[RFC6536]

Network Configuration Protocol (NETCONF) Access Control Model
A. Bierman
M. Bjorklund
March 2012
RFC 6536
DOI 10.17487/RFC6536
[RFC7951]

JSON Encoding of Data Modeled with YANG
L. Lhotka
August 2016
RFC 7951
DOI 10.17487/RFC7951
[RFC9535]

JSONPath: Query Expressions for JSON
S. Gössner, Ed.
G. Normington, Ed.
C. Bormann, Ed.
February 2024
RFC 9535
DOI 10.17487/RFC9535
[I-D.tgraf-netconf-notif-sequencing]

Support of Hostname and Sequencing in YANG Notifications
Thomas Graf
Swisscom
Jean Quilbeuf
Huawei
Alex Huang Feng
INSA-Lyon
29 June 2024
Internet-Draft draft-tgraf-netconf-notif-sequencing-06
[I-D.tgraf-netconf-yang-push-observation-time]

Support of Observation Timestamp in YANG-Push Notifications
Thomas Graf
Swisscom
Benoît Claise
Huawei
Alex Huang Feng
INSA-Lyon
3 July 2024
Internet-Draft draft-tgraf-netconf-yang-push-observation-time-02
[RFC8072]

YANG Patch Media Type
A. Bierman
M. Bjorklund
K. Watsen
February 2017
RFC 8072
DOI 10.17487/RFC8072
Combined YANG tree
This section shows the relevant subsets of the combined Subscribed Notification YANG trees that are augmented by the ietf-yp-ext.yang additions.
YANG tree for ietf-subscribed-notifications.yang with ietf-yang-push.yang and ietf-yp-ext.yang (and a couple of others).

module: ietf-subscribed-notifications
  +--ro streams
  |  +--ro stream* [name]
  |     +--ro name                        string
  |     +--ro description?                string
  |     +--ro replay-support?             empty {replay}?
  |     +--ro replay-log-creation-time    yang:date-and-time
  |     |       {replay}?
  |     +--ro replay-log-aged-time?       yang:date-and-time
  |             {replay}?
  +--rw filters
  |  +--rw stream-filter* [name]
  |  |  +--rw name                           string
  |  |  +--rw (filter-spec)?
  |  |     +--:(stream-subtree-filter)
  |  |     |  +--rw stream-subtree-filter?   <anydata> {subtree}?
  |  |     +--:(stream-xpath-filter)
  |  |        +--rw stream-xpath-filter?     yang:xpath1.0 {xpath}?
  |  +--rw yp:selection-filter* [filter-id]
  |     +--rw yp:filter-id                         string
  |     +--rw (yp:filter-spec)?
  |        +--:(yp:datastore-subtree-filter)
  |        |  +--rw yp:datastore-subtree-filter?   <anydata>
  |        |          {sn:subtree}?
  |        +--:(yp:datastore-xpath-filter)
  |           +--rw yp:datastore-xpath-filter?     yang:xpath1.0
  |                   {sn:xpath}?
  +--rw subscriptions
     +--rw subscription* [id]
     |  +--rw id
     |  |       subscription-id
     |  +--rw (target)
     |  |  +--:(stream)
     |  |  |  +--rw (stream-filter)?
     |  |  |  |  +--:(by-reference)
     |  |  |  |  |  +--rw stream-filter-name
     |  |  |  |  |          stream-filter-ref
     |  |  |  |  +--:(within-subscription)
     |  |  |  |     +--rw (filter-spec)?
     |  |  |  |        +--:(stream-subtree-filter)
     |  |  |  |        |  +--rw stream-subtree-filter?
     |  |  |  |        |          <anydata> {subtree}?
     |  |  |  |        +--:(stream-xpath-filter)
     |  |  |  |           +--rw stream-xpath-filter?
     |  |  |  |                   yang:xpath1.0 {xpath}?
     |  |  |  +--rw stream
     |  |  |  |       stream-ref
     |  |  |  +--ro replay-start-time?
     |  |  |  |       yang:date-and-time {replay}?
     |  |  |  +--rw configured-replay?                         empty
     |  |  |          {configured,replay}?
     |  |  +--:(yp:datastore)
     |  |     +--rw yp:datastore
     |  |     |       identityref
     |  |     +--rw (yp:selection-filter)?
     |  |        +--:(yp:by-reference)
     |  |        |  +--rw yp:selection-filter-ref
     |  |        |          selection-filter-ref
     |  |        +--:(yp:within-subscription)
     |  |           +--rw (yp:filter-spec)?
     |  |              +--:(yp:datastore-subtree-filter)
     |  |              |  +--rw yp:datastore-subtree-filter?
     |  |              |          <anydata> {sn:subtree}?
     |  |              +--:(yp:datastore-xpath-filter)
     |  |                 +--rw yp:datastore-xpath-filter?
     |  |                         yang:xpath1.0 {sn:xpath}?
     |  +--rw stop-time?
     |  |       yang:date-and-time
     |  +--rw dscp?
     |  |       inet:dscp {dscp}?
     |  +--rw weighting?                                       uint8
     |  |       {qos}?
     |  +--rw dependency?
     |  |       subscription-id {qos}?
     |  +--rw transport?
     |  |       transport {configured}?
     |  +--rw encoding?
     |  |       encoding
     |  +--rw purpose?                                         string
     |  |       {configured}?
     |  +--rw (notification-message-origin)? {configured}?
     |  |  +--:(interface-originated)
     |  |  |  +--rw source-interface?
     |  |  |          if:interface-ref {interface-designation}?
     |  |  +--:(address-originated)
     |  |     +--rw source-vrf?
     |  |     |       -> /ni:network-instances/network-instance/name
     |  |     |       {supports-vrf}?
     |  |     +--rw source-address?
     |  |             inet:ip-address-no-zone
     |  +--ro configured-subscription-state?
     |  |       enumeration {configured}?
     |  +--rw receivers
     |  |  +--rw receiver* [name]
     |  |     +--rw name                         string
     |  |     +--ro sent-event-records?
     |  |     |       yang:zero-based-counter64
     |  |     +--ro excluded-event-records?
     |  |     |       yang:zero-based-counter64
     |  |     +--ro state                        enumeration
     |  |     +---x reset {configured}?
     |  |     |  +--ro output
     |  |     |     +--ro time    yang:date-and-time
     |  |     +--rw snr:receiver-instance-ref?   leafref
     |  +--rw (yp:update-trigger)?
     |  |  +--:(yp:periodic)
     |  |  |  +--rw yp:periodic!
     |  |  |     +--rw yp:period         centiseconds
     |  |  |     +--rw yp:anchor-time?   yang:date-and-time
     |  |  +--:(yp:on-change) {on-change}?
     |  |  |  +--rw yp:on-change!
     |  |  |     +--rw yp:dampening-period?   centiseconds
     |  |  |     +--rw yp:sync-on-start?      boolean
     |  |  |     +--rw yp:excluded-change*    change-type
     |  |  +--:(yp-ext:periodic-and-on-change) {yp:on-change}?
     |  |     +--rw yp-ext:periodic-and-on-change!
     |  |        +--rw yp-ext:period              yp:centiseconds
     |  |        +--rw yp-ext:anchor-time?        yang:date-and-time
     |  |        +--rw yp-ext:dampening-period?   yp:centiseconds
     |  |        +--rw yp-ext:sync-on-start?      boolean
     |  |        +--rw yp-ext:excluded-change*    yp:change-type
     |  +--ro dn:message-publisher-ids*                        uint32
     |  +--rw yp-ext:common-notification-format?
     |          boolean
     +--rw snr:receiver-instances
        +--rw snr:receiver-instance* [name]
           +--rw snr:name                        string
           +--rw (snr:transport-type)
              +--:(hnt:https)
              |  +--rw hnt:https-receiver
              |     +--rw hnt:receiver-identity {receiver-identity}?
              |        +--rw hnt:cert-maps
              |           +--rw hnt:cert-to-name* [id]
              |              +--rw hnt:id             uint32
              |              +--rw hnt:fingerprint
              |              |       x509c2n:tls-fingerprint
              |              +--rw hnt:map-type       identityref
              |              +--rw hnt:name           string
              +--:(unt:udp-notif)
                 +--rw unt:udp-notif-receiver
                    +--rw unt:dtls! {dtls13}?
                    +--rw unt:enable-segmentation?   boolean
                    |       {segmentation}?
                    +--rw unt:max-segment-size?      uint32
                            {segmentation}?

  rpcs:
    +---x establish-subscription
    |  +---w input
    ...

  notifications:
    +---n replay-completed {replay}?
    |  ...
    +---n subscription-completed {configured}?
    |  ...
    +---n subscription-modified
    |  +--ro id
    |  |       subscription-id
    |  +--ro (target)
    |  |  +--:(stream)
    |  |  |  +--ro (stream-filter)?
    |  |  |  |  +--:(by-reference)
    |  |  |  |  |  +--ro stream-filter-name
    |  |  |  |  |          stream-filter-ref
    |  |  |  |  +--:(within-subscription)
    |  |  |  |     +--ro (filter-spec)?
    |  |  |  |        +--:(stream-subtree-filter)
    |  |  |  |        |  +--ro stream-subtree-filter?
    |  |  |  |        |          <anydata> {subtree}?
    |  |  |  |        +--:(stream-xpath-filter)
    |  |  |  |           +--ro stream-xpath-filter?
    |  |  |  |                   yang:xpath1.0 {xpath}?
    |  |  |  +--ro stream
    |  |  |  |       stream-ref
    |  |  |  +--ro replay-start-time?
    |  |  |          yang:date-and-time {replay}?
    |  |  +--:(yp:datastore)
    |  |     +--ro yp:datastore
    |  |     |       identityref
    |  |     +--ro (yp:selection-filter)?
    |  |        +--:(yp:by-reference)
    |  |        |  +--ro yp:selection-filter-ref
    |  |        |          selection-filter-ref
    |  |        +--:(yp:within-subscription)
    |  |           +--ro (yp:filter-spec)?
    |  |              +--:(yp:datastore-subtree-filter)
    |  |              |  +--ro yp:datastore-subtree-filter?
    |  |              |          <anydata> {sn:subtree}?
    |  |              +--:(yp:datastore-xpath-filter)
    |  |                 +--ro yp:datastore-xpath-filter?
    |  |                         yang:xpath1.0 {sn:xpath}?
    |  +--ro stop-time?
    |  |       yang:date-and-time
    |  +--ro dscp?
    |  |       inet:dscp {dscp}?
    |  +--ro weighting?                                       uint8
    |  |       {qos}?
    |  +--ro dependency?
    |  |       subscription-id {qos}?
    |  +--ro transport?
    |  |       transport {configured}?
    |  +--ro encoding?
    |  |       encoding
    |  +--ro purpose?                                         string
    |  |       {configured}?
    |  +--ro (yp:update-trigger)?
    |  |  +--:(yp:periodic)
    |  |  |  +--ro yp:periodic!
    |  |  |     +--ro yp:period         centiseconds
    |  |  |     +--ro yp:anchor-time?   yang:date-and-time
    |  |  +--:(yp:on-change) {on-change}?
    |  |  |  +--ro yp:on-change!
    |  |  |     +--ro yp:dampening-period?   centiseconds
    |  |  |     +--ro yp:sync-on-start?      boolean
    |  |  |     +--ro yp:excluded-change*    change-type
    |  |  +--:(yp-ext:periodic-and-on-change) {yp:on-change}?
    |  |     +--ro yp-ext:periodic-and-on-change!
    |  |        +--ro yp-ext:period              yp:centiseconds
    |  |        +--ro yp-ext:anchor-time?        yang:date-and-time
    |  |        +--ro yp-ext:dampening-period?   yp:centiseconds
    |  |        +--ro yp-ext:sync-on-start?      boolean
    |  |        +--ro yp-ext:excluded-change*    yp:change-type
    |  +--ro dn:message-publisher-ids*                        uint32
    |  +--ro yp-ext:common-notification-format?               boolean
    +---n subscription-resumed
    |  ...
    +---n subscription-started {configured}?
    |  +--ro id
    |  |       subscription-id
    |  +--ro (target)
    |  |  +--:(stream)
    |  |  |  +--ro (stream-filter)?
    |  |  |  |  +--:(by-reference)
    |  |  |  |  |  +--ro stream-filter-name
    |  |  |  |  |          stream-filter-ref
    |  |  |  |  +--:(within-subscription)
    |  |  |  |     +--ro (filter-spec)?
    |  |  |  |        +--:(stream-subtree-filter)
    |  |  |  |        |  +--ro stream-subtree-filter?
    |  |  |  |        |          <anydata> {subtree}?
    |  |  |  |        +--:(stream-xpath-filter)
    |  |  |  |           +--ro stream-xpath-filter?
    |  |  |  |                   yang:xpath1.0 {xpath}?
    |  |  |  +--ro stream
    |  |  |  |       stream-ref
    |  |  |  +--ro replay-start-time?
    |  |  |  |       yang:date-and-time {replay}?
    |  |  |  +--ro replay-previous-event-time?
    |  |  |          yang:date-and-time {replay}?
    |  |  +--:(yp:datastore)
    |  |     +--ro yp:datastore
    |  |     |       identityref
    |  |     +--ro (yp:selection-filter)?
    |  |        +--:(yp:by-reference)
    |  |        |  +--ro yp:selection-filter-ref
    |  |        |          selection-filter-ref
    |  |        +--:(yp:within-subscription)
    |  |           +--ro (yp:filter-spec)?
    |  |              +--:(yp:datastore-subtree-filter)
    |  |              |  +--ro yp:datastore-subtree-filter?
    |  |              |          <anydata> {sn:subtree}?
    |  |              +--:(yp:datastore-xpath-filter)
    |  |                 +--ro yp:datastore-xpath-filter?
    |  |                         yang:xpath1.0 {sn:xpath}?
    |  +--ro stop-time?
    |  |       yang:date-and-time
    |  +--ro dscp?
    |  |       inet:dscp {dscp}?
    |  +--ro weighting?                                       uint8
    |  |       {qos}?
    |  +--ro dependency?
    |  |       subscription-id {qos}?
    |  +--ro transport?
    |  |       transport {configured}?
    |  +--ro encoding?
    |  |       encoding
    |  +--ro purpose?                                         string
    |  |       {configured}?
    |  +--ro (yp:update-trigger)?
    |  |  +--:(yp:periodic)
    |  |  |  +--ro yp:periodic!
    |  |  |     +--ro yp:period         centiseconds
    |  |  |     +--ro yp:anchor-time?   yang:date-and-time
    |  |  +--:(yp:on-change) {on-change}?
    |  |  |  +--ro yp:on-change!
    |  |  |     +--ro yp:dampening-period?   centiseconds
    |  |  |     +--ro yp:sync-on-start?      boolean
    |  |  |     +--ro yp:excluded-change*    change-type
    |  |  +--:(yp-ext:periodic-and-on-change) {yp:on-change}?
    |  |     +--ro yp-ext:periodic-and-on-change!
    |  |        +--ro yp-ext:period              yp:centiseconds
    |  |        +--ro yp-ext:anchor-time?        yang:date-and-time
    |  |        +--ro yp-ext:dampening-period?   yp:centiseconds
    |  |        +--ro yp-ext:sync-on-start?      boolean
    |  |        +--ro yp-ext:excluded-change*    yp:change-type
    |  +--ro dn:message-publisher-ids*                        uint32
    |  +--ro yp-ext:common-notification-format?               boolean
    +---n subscription-suspended
    |  +--ro id        subscription-id
    |  +--ro reason    identityref
    +---n subscription-terminated
       +--ro id        subscription-id
       +--ro reason    identityref

module: ietf-yp-ext

  notifications:
    +---n update
       +--ro id?                   sn:subscription-id
       +--ro subscription-path?    yang:xpath1.0
       +--ro target-path?          string
       +--ro snapshot-type?        enumeration
       +--ro observation-time?     yang:date-and-time
       +--ro datastore-snapshot?   <anydata>
       +--ro incomplete?           empty

Examples
Notes on examples:
- These examples have been given using a JSON encoding of the regular YANG-Push notification format, i.e., encoded using [RFC5277], but it is anticipated that these notifications could be defined to exclusive use the new format proposed by [I-D.netana-netconf-notif-envelope].
- Some additional meta data fields, e.g., like those defined in [I-D.tgraf-netconf-notif-sequencing] would also likely be included, but have also been excluded to allow for slightly more concise examples.
- The examples include the [I-D.tgraf-netconf-yang-push-observation-time] field for the existing YANG-Push Notification format, and the proposed equivalent "observation-time" leaf for the new update notification format.
- All these examples are created by hand, may contain errors, and may not parse correctly.
Example of an [RFC8641] style push-update notification
This example illustrates a periodic update message for a subscription at "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces" that is returning data from two internal data providers, one of which is returning data for "interface-summary" and the other that is returning data for "interfaces". The device resident management agents stitches the reply together from the two data providers into a single data tree before returning it in a single message.
For the periodic message to be correct, and to allow it to replace any previous periodic message published on that subscription, it must include all data below the subscription path. This may increase the total amount of internal IPC within the device and make the timestamps less accurate, since the observation timestamp only reports when the device starts polling the data providers. If those providers are distributed across multiple processes and linecards then it may take a bit of time to complete the periodic on-change.

{
  "ietf-notification:notification": {
    "eventTime": "2024-09-27T14:16:27.773Z",
    "ietf-yang-push:push-update": {
      "id": 1,
      "ietf-yp-observation:timestamp": "2024-09-27T14:16:27.773Z",
      "ietf-yp-observation:point-in-time": "current-accounting",
      "datastore-contents": {
        "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces": {
          "interface-summary": {
            "interface-type": [
              {
                "interface-type-name": "IFT_GETHERNET",
                "interface-type-description": "GigabitEthernet",
                "interface-counts": {
                  "interface-count": 5,
                  "up-interface-count": 2,
                  "down-interface-count": 0,
                  "admin-down-interface-count": 3
                }
              }
            ],
            "interface-counts": {
              "interface-count": 8,
              "up-interface-count": 5,
              "down-interface-count": 0,
              "admin-down-interface-count": 3
            }
          },
          "interfaces": {
            "interface": [
              {
                "interface-name": "GigabitEthernet0/0/0/0",
                "interface": "GigabitEthernet0/0/0/0",
                "state": "im-state-admin-down",
                "line-state": "im-state-admin-down"
              },
              {
                "interface-name": "GigabitEthernet0/0/0/4",
                "interface": "GigabitEthernet0/0/0/4",
                "state": "im-state-admin-down",
                "line-state": "im-state-admin-down"
              },
            ]
          }
        }
      }
    }
  }
}

Example of periodic updates using the new style update message.
The subscription was made on "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces", but for efficiency reasons, the device has split the subscription into separate child subscriptions for the different data providers, and makes use of the new message format.
Hence, this first periodic message is being published for the "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-summary" container, but it is encoded rooted relative to the schema for "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces".

{
  "ietf-notification:notification": {
    "eventTime": "2024-09-27T14:16:27.773Z",
    "ietf-yp-ext:update": {
      "id": 1,
      "subscription-path": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces",
      "target-path":
        "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interface-summary",
      "snapshot-type": "periodic"
      "observation-time": "2024-09-27T14:16:27.773Z",
      "datastore-snapshot": {
        "interface-summary" : {
          "interface-type": [
            {
              "interface-type-name": "IFT_GETHERNET",
              "interface-type-description": "GigabitEthernet",
              "interface-counts": {
                "interface-count": 5,
                "up-interface-count": 2,
                "down-interface-count": 0,
                "admin-down-interface-count": 3
              }
            }
          ],
          "interface-counts": {
            "interface-count": 8,
            "up-interface-count": 5,
            "down-interface-count": 0,
            "admin-down-interface-count": 3
          }
        }
      }
    }
  }
}

The second periodic message is being published for the "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interfaces/interface" list, but again, it is encoded rooted relative to the schema for "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces". This message has a separate observation time that represents the more accurate time that this periodic date was read.

{
  "ietf-notification:notification": {
    "eventTime": "2024-09-27T14:16:27.973Z",
    "ietf-yp-ext:update": {
      "id": 1,
      "subscription-path": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces",
      "target-path":
        "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interfaces/interface[]",
      "snapshot-type": "periodic"
      "observation-time": "2024-09-27T14:16:27.973Z",
      "datastore-snapshot": {
        "interfaces": {
          "interface": [
            {
              "interface-name": "GigabitEthernet0/0/0/0",
              "interface": "GigabitEthernet0/0/0/0",
              "state": "im-state-admin-down",
              "line-state": "im-state-admin-down"
            },
            {
              "interface-name": "GigabitEthernet0/0/0/4",
              "interface": "GigabitEthernet0/0/0/4",
              "state": "im-state-admin-down",
              "line-state": "im-state-admin-down"
            },
          ]
        }
      }
    }
  }
}

Each child subscription would use the same period and anchor time as the configured subscription, possibly with a little bit of initial jitter to avoid all daemons attempting to publish the data at exactly the same time.
Example of an on-change-update notification using the new style update message.
If the subscription is for on-change notifications, or periodic-and-on-change-notifications, then, if the interface state changed (specifically if the 'state' leaf of the interface changed state), and if the device was capable of generating on-change notifications, then you may see the following message. A few points of notes here:
- The on-change notification contains all of the state at the "target-path"
  - Not present in the below example, but if the notification excluded some state under an interfaces list entry (e.g., the line-state leaf) then this would logically represent the implicit deletion of that field under the given list entry.
  - In this example it is restricted to a single interface. It could also publish an on-change notification for all interfaces, by indicating a target-path without any keys specified. TODO - Can it represent notifications for a subset of interfaces?
- The schema of the change message is exactly the same as for the equivalent periodic message. It doesn't use the YANG Patch format [RFC8072] for on-change messages.
- The "observation time" leaf represents when the system first observed the on-change event occurring.
- The on-change event doesn't differentiate the type of change to operational state. The on-change-update snapshot type is used to indicate the creation of a new entry or some update to some existing state. Basically, the message can be thought of as the state existing with some current value.

{
  "ietf-notification:notification": {
    "eventTime": "2024-09-27T14:16:30.973Z",
    "ietf-yp-ext:update": {
      "id": 1,
      "subscription-path": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces",
      "target-path":
        "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interfaces/
         interface[interface=GigabitEthernet0/0/0/0]",
      "snapshot-type": "on-change-update"
      "observation-time": "2024-09-27T14:16:30.973Z",
      "datastore-snapshot": {
        "interfaces": {
          "interface": [
            {
              "interface-name": "GigabitEthernet0/0/0/0",
              "interface": "GigabitEthernet0/0/0/0",
              "state": "im-state-up",
              "line-state": "im-state-up"
            }
          ]
        }
      }
    }
  }
}

Example of an on-change-delete notification using the new style update message.
If the interface was deleted, and if the system was capable of reporting on-change events for the delete event, then an on-change delete message would be encoded as per the following message. Of note:
- The on-change-delete snapshot type doesn't include a "datastore-snapshot", instead it represents a delete of the list entry at the path identified by the target-path, which is similar to a YANG Patch delete notification.

{
  "ietf-notification:notification": {
    "eventTime": "2024-09-27T14:16:40.973Z",
    "ietf-yp-ext:update": {
      "id": 1,
      "subscription-path": "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces",
      "target-path":
        "Cisco-IOS-XR-pfi-im-cmd-oper:interfaces/interfaces/
         interface[interface=GigabitEthernet0/0/0/0]",
      "snapshot-type": "on-change-delete"
      "observation-time": "2024-09-27T14:16:40.973Z",
    }
  }
}