Experiment: Network Anomaly Lifecycle
Vincenzo Riccobene
Huawei
Dublin
Ireland
vincenzo.riccobene@huawei-partners.com
Antonio Roberto
Huawei
Dublin
Ireland
antonio.roberto@huawei.com
Thomas Graf
Swisscom
Binzring 17
Zurich
8045
Switzerland
thomas.graf@swisscom.com
Wanting Du
Swisscom
Binzring 17
Zurich
8045
Switzerland
wanting.du@swisscom.com
Alex Huang Feng
INSA-Lyon
Lyon
France
alex.huang-feng@insa-lyon.fr
08 July 2024
Operations and Management
NMOP
Network Anomaly Detection is the act of detecting problems in the network. Accurately detect problems is very challenging for network operators in production networks. Good results require a lot of expertise and knowledge around both the implied network technologies and the specific service provided to consumers, apart from a proper monitoring infrastructure. In order to facilitate network anomaly detection, novel techniques are being introduced, including programmatical, rule-based and AI-based, with the promise of improving scalability and the hope to keep a high detection accuracy. To guarantee acceptable results, the process needs to be properly designed, adopting well-defined stages to accurately collect evidence of anomalies, validate their relevancy and improve the detection systems over time, iteratively.
This document describes the lifecycle process to iteratively improve network anomaly detection accurately. Three key stages are proposed, along with a YANG model specifying the required metadata for the network anomaly detection covering the exchange of information between different stages of the lifecycle.

Discussion Venues
This note is to be removed before publishing as an RFC.
Discussion of this document takes place on the Operations and Management Area Working Group Working Group mailing list (nmop@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/nmop/.
Source for this draft and an issue tracker can be found at https://github.com/network-analytics/draft-netana-nmop-network- anomaly-lifecycle.

Status of this document
This document is experimental. The main goal of this document is to propose an iterative lifecycle process to network anomaly detection by proposing a data model for metadata to be addressed at different lifecycle stages.
The experiment consists of verifying whether the approach is usable in real use case scenarios to support proper refinement and adjustments of network anomaly detection algorithms. The experiment can be deemed successful if validated at least with an open-source implementation sucessfully applied in real production networks.

Introduction
In [I-D.netana-nmop-network-anomaly-architecture] network anomalies are defined as "Whatever would let an operator frown and investigate when looking at the collected forwarding plane, control plane and management plane network data relative to a customer" .
In [I-D.netana-nmop-network-anomaly-semantics] a semantic for the annotation of network anomalies has been defined in order to support the exchange of related metadata between different actors, formalizing a semantically consistent representation of the behaviors worth investigating. In the same document, symptoms are defined as the essential piece of information to analyze network anomalies and problems.
The intention is to enable operators detecting problems in the network timely. A network problem is defined as "A state regarded as undesirable and may require remedial action" (see [I-D.ietf-nmop-terminology]).
With all this in mind, this document starts from the assumption that it is still remarkably difficult to gain a full understanding and a complete perspective of "if" and "how" the network is deviating from the desired state: on the one side, symptoms are not necessarily a guarantee of a problem happening (e.g. there might be false positives), on the other side, the lack of symptom is not a guarantee of the absence of an problem (e.g. there might be false negatives). The concept of network anomaly in this document plays the role of a bridge between symptoms and problem: a network anomaly is defined as a collection of symptoms, but without the guarantee that the observed symptoms are impacting existing services. This opens up to the necessity of further validating the network anomalies to understand if the detected symptoms are actually impacting services and it requires different actors (both human and algorithmic) to jump in during the process and refine their understanding across the network anomaly lifecycle.
Performing network anomaly detection is a process that requires a continuous learning and continuous improvement. Network anomalies are detected by collecting and understanding symptoms, then validated by confirming that there actually were service impacting and eventually need to be further analyzed by performing postmortem analysis to identify any potential adjustment to improve the detection capability. Each of these stages is an opportunity to learn and refine the process, and since implementations of these stages might also be provided by different parties and/or products, this document also contributes a formal structure to capture and exchange symptom information across the lifecycle.

Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
This document makes use of the terms defined in [I-D.ietf-nmop-terminology].
- State
- Problem
- Event
- Alarm
- Symptom
The following terms are used as defined in [RFC9417].
- Metric
- Intent
The following terms are defined in this document.
- Annotator: Is a human or an algorithm which produces metadata by describing anomalies with symptoms.
- False Positive: Is a detected anomaly which has been identified during the postmortem to be not anomalous.
- False Negative: Is anomalous but has been not been identified by the anomaly detection system.

Defining Desired States
The above definitions of network problem provide the scope for what to be looking for when detecting network anomalies. Concepts like "desirable state" and "required state" are introduced. This poses the attention on a significant problem that network operators have to face: the definition of what is to be considered "desirable" or "undesirable". It is not always easy to detect if a network is operating in an undesired state at a given point in time. To approach this, network operators can rely on different methodologies, more or less deterministic and more or less sensitive: on the one side, the definition of intents (including Service Level Objectives and Service Level Agreements) which approaches the problem top-down; on the other side, the definition of symptoms, by mean of solutions like SAIN, [RFC9418] and [I-D.netana-nmop-network-anomaly-architecture], which approaches the problem bottom-up. At the center of these approaches, there are the so-called symptoms, defined as reasons explaining what is not working as expected in the network, sometimes also providing hints towards issues and their causes.
One of the more deterministic approaches is to rely on symptoms based on measurable service-based KPIs, for example, by using Service Level Indicators, Objectives and Agreements:
Service Level Agreement (SLA)
An SLA is an agreement between parties that a service provider makes to its customers on the behavior of the provided service. SLAs are a tool to define exactly what customers can expect out of the service provided to them. In many cases, SLA breaches also come with contractual penalties.
Service Level Objectives (SLOs)
An SLO is a threshold above which the service provider acts to prevent a breach of an SLA. SLOs are a tool for service providers to know when they should start becoming concerned about a service not behaving as expected. SLOs are rarely connected to penalties as they usually are internal metrics for the service providers.
Service Level Indicators (SLIs)
An SLI is an observable metric that describes the state of a monitored subsystem. SLIs are a tool to gain measurable visibility about the behavior of a subsystem in the network. SLIs usually differ from SLOs as SLOs are usually expressed as thresholds, while SLIs would often be expressed e.g. as percentages.
However, the definition of these KPIs turns out to be very challenging in some cases, as accurate KPIs could require computationally expensive techniques to be collected or substantial modifications to existing network protocols.
Alternative methodologies rely on symptoms as the way to generate analytical data out of operational data. For instance:
SAIN
introduces the definition and exposure of symptoms as a mechanism for detecting those concerning behaviors in more deterministic ways. Moreover, the concept of "impact score" has been introduced by SAIN, to indicate what is the expected degree of impact that a given symptom will have on the services relying on the related subservice to which the symptom is attached.
Daisy
introduces the concept of concern score to indicate what is the degree of concern that a given symptom could cause a degradation for a service.
In general, defining boundaries between desirable vs. undesirable in an accurate fashion requires continuous iterations and improvements coming from all the stages of the network anomaly detection lifecycle, by which network engineers can transfer what they learn through the process into new symptom definitions and, ultimately, into refinements of the detection algorithms.

Lifecycle of a Network Anomaly
The lifecycle of a network anomaly can be articulated in three phases, structured as a loop: Detection, Validation, Refinement.

Anomaly Detection Refinement Lifecycle

                       +-------------+
            +--------> |  Detection  | ---------+
Adjustments |          +-------------+          | Symptoms
            |                                   |
            |                                   v
    +------------+                       +------------+
    | Refinement |<--------------------- | Validation |
    +------------+        Problem        +------------+
                        Confirmation

Each of these phases can either be performed by a network expert or an algorithm or complementing each other.
The network anomaly metadata is generated by an annotator, which can be either a human expert or an algorithm. The annotator can produce the metadata for a network anomaly, for each stage of the cycle and even multiple versions for the same stage. In each version of the network anomaly metadata, the annotator indicates the list of symptoms that are part of the network anomaly taken into account. The iterative process is about the identification of the right set of symptoms.

Network Anomaly Detection
The Network Anomaly Detection stage is about the continuous monitoring of the network through Network Telemetry [RFC9232] and the identification of symptoms. One of the main requirements that operator have on network anomaly detection systems is the high accuracy. This means having a small number of false negatives, symptoms causing service impact are not missed, and false positives, symptoms that are actually innocuous are not picked up.
As the detection stage is becoming more and more automated for production networks, the identified symptoms might point towards three potential kinds of behaviors:
i. those that are surely corresponding to an impact on services, (e.g. the breach of an SLO),
ii. those that will cause problems in the future (e.g. rising trends on a timeseries metric hitting towards saturation),
iii. those or which the impact to services cannot be confirmed (e.g. sudden increase/decrease of timeseries metrics, anomalous amounts of log entries, etc.).
The first category requires immediate intervention (a.k.a. the problem is "confirmed"), the second one provides pointers towards early signs of an problem potentially happening in the near future (a.k.a. the problem is "forecasted"), and the third one requires some analysis to confirm if the detected symptom requires any attention or immediate intervention (a.k.a. the problem is "potential"). As part of the iterative improvement required in this stage, one that is very relevant is the gradual conversion of the third category into one of the first two, which would make the network anomaly detection system more deterministic. The main objective is to reduce uncertainty around the raised alarms by refining the detection algorithms. This can be achieved by either generating new symptom definitions, adjusting the weights of automated algorithms or other similar approaches.

Network Anomaly Validation
The key objective for the validation stage is clearly to decide if the detected symptoms are signaling a real problem (a.k.a. requires action) or if they are to be treated as false positives (a.k.a. suppressing the alarm). For those symptoms surely having impact on services, 100% confidence on the fact that a network problem is happening can be assumed. For the other two categories, "forecasted" and "potential", further analysis and validation is required.

Network Anomaly Refinement
After validation of a problem, the service provider performs troubleshooting and resolution of the problem. Although the network might be back in a desired state at this point, network operators can perform detailed postmortem analysis of network problems with the objective to identify useful adjustments to the prevention and detection mechanisms (for instance improving or extending the definition of SLIs and SLOs, refining concern/impact scores, etc.), and improving the accuracy of the validation stage (e.g. automating parts of the validation, implementing automated root cause analysis and automation for remediation actions). In this stage of the lifecycle it is assumed that the problem is under analysis.
After the adjustments are performed to the network anomaly detection methods, the cycle starts again, by "replaying" the network anomaly and checking if there is any measurable improvement in the ability to detect problems by using the updated method.

Network Anomaly State Machine
In the context of this document, from a network anomaly detection point of view a network problem is defined as a collection of interrelated symptoms, as specified in [I-D.netana-nmop-network-anomaly-semantics].
The understanding of a network problem can change over time. Moreover, multiple actors are involved in the process of refining this understanding in the different phases.
From this perspective, a problem can be refined according to the following states (Figure 2).

Network Anomaly State Machine

                                             +---------+
                                             | Initial |-----------------+
                                             +---------+                 |
                                                  |                      |
                                            +-----+---------+            |
                                   +--------|---------------|------+     |
                                   | +------v-----+  +------v----+ |     |
                                   | |  Problem   |  |  Problem  | |     |
                             +---->| | Forecasted |  | Potential | |     |
                             |     | +------------+  +-----------+ |     |
                             |     +--------|--Detection---|-------+     |
                             |              |              |             |
        +-------+            |              +------- ----- +             |
        | Final |            |                      |                    |
        +---^---+            |                      |                    |
            |                |                      |                    |
            |                |                      v                    |
            |                |     +-----------Validation------------+   |
+-----------------------+    |     |  +-----------+                  |   |
|           |           |    |     |  |  Problem  |   |  Problem  |  |   |
|  +-----------------+  |    |     |  | Discarded |   | Confirmed |<-|---+
|  |    Detection    |  |    |     |  +-----|-----+   +-----------+  |
|  |     Adjusted    |-------+     +---------------------------------+
|  +--------^--------+  |                   |               |
|           |           |                   |               |
|           |           |               +---v---+           |
|           |           |               | Final |           |
|           |           |               +-------+           |
| +---------|--------+  |                                   |
| |     Problem      |  |                                   |
| |     Analyzed     |<-|-----------------------------------+
| +------------------+  |
+-------Refinement------+

Overview of the Model for the Network Anomaly Metadata

YANG tree diagram for ietf-network-anomaly-metadata

module: ietf-network-anomaly-metadata
  +--rw network-anomalies
     +--rw network-anomaly* [id version]
        +--rw id             yang:uuid
        +--rw version        uint32
        +--rw description?   string
        +--rw annotator
        |  +--rw (annotator-type)
        |  |  +--:(human)
        |  |  |  +--rw human        empty
        |  |  +--:(algorithm)
        |  |     +--rw algorithm    empty
        |  +--rw name?              empty
        +--rw state          identityref
        +--rw symptoms* [symptom_id]
           +--rw symptom_id    yang:uuid

YANG module for ietf-network-anomaly-metadata

<CODE BEGINS> file "ietf-network-anomaly-metadata@2024-07-01.yang"
module ietf-network-anomaly-metadata {
  yang-version 1.1;
  namespace "urn:ietf:params:xml:ns:yang:ietf-network-anomaly-metadata";
  prefix network_anomaly_metadata;

  import ietf-yang-types {
    prefix yang;
    reference "RFC 6991: Common YANG Data Types";
  }

  organization
    "IETF NMOP Working Group";
  contact
    "WG Web:   <https://datatracker.ietf.org/wg/nmop/>
     WG List:  <mailto:nmop@ietf.org>

     Authors:  Vincenzo Riccobene
               <mailto:vincenzo.riccobene@huawei-partners.com>
               Antonio Roberto
               <mailto:antonio.roberto@huawei.com>
               Thomas Graf
               <mailto:thomas.graf@swisscom.com>
               Wanting Du
               <mailto:wanting.du@swisscom.com>
               Alex Huang Feng
               <mailto:alex.huang-feng@insa-lyon.fr>";
  description
    "This module defines objects for the description of network anomalies.
      Network anomalies are a collection of symptoms observed on
      the network nodes.

      Copyright (c) 2024 IETF Trust and the persons identified as
      authors of the code.  All rights reserved.

      Redistribution and use in source and binary forms, with or
      without modification, is permitted pursuant to, and subject
      to the license terms contained in, the Revised BSD License
      set forth in Section 4.c of the IETF Trust's Legal Provisions
      Relating to IETF Documents
      (https://trustee.ietf.org/license-info).

      This version of this YANG module is part of RFC XXXX; see the RFC
      itself for full legal notices.";

  revision 2024-07-01 {
    description
      "Initial version";
    reference
      "RFCXXXX: Experiment: Network Anomaly Postmortem Lifecycle";
  }

  identity network-anomaly-state {
    description
      "Base identity for representing the state of the network anomaly";
  }
  identity problem-forecasted {
    base network-anomaly-state;
    description
      "A problem has been forecasted, as it is expected that
      the indicated list of symptoms will impact a service
      in the near future";
  }
  identity problem-potential {
    base network-anomaly-state;
    description
      "A problem has been detected with a confidence
      lower than 100%. In order to confirm that this set of
      symptoms are generating service impact, it requires further
      validation";
  }
  identity problem-confirmed {
    base network-anomaly-state;
    description
      "After validation, the problem has been confirmed";
  }
  identity discarded {
    base network-anomaly-state;
    description
      "After validation, the network anomaly has been
      discarded, as there is no evindence that it is causing an
      problem";
  }
  identity analysed {
    base network-anomaly-state;
    description
      "The anomaly detection went through analysis to identify
      potential ways to further improve the detection process in
      for future anomalies";
  }
  identity adjusted {
    base network-anomaly-state;
    description
      "The network anomaly has been solved and analysed.
      No further action is required.";
  }

  container network-anomalies {
    description "Container having the network anomalies";
    list network-anomaly {
      key "id version";
      description "A network anomaly identified by an id, version
        and state.";
      leaf id {
        type yang:uuid;
        description
            "Unique ID of the network network anomaly";
      }
      leaf version {
        type uint8;
        description
          "Version of the problem metadata object.
          It allows multiple versions of the metadata to be
          generated in order to support the definition of
          multiple problem objects from the same source to
          facilitate improvements overtime";
      }
      leaf description {
        type string;
        mandatory "false";
        description
          "Textual description of the network anomaly";
      }
      container annotator {
        description "Container defining the type of the annotator and the
          version of the algorithm if it is an algorithm who reported the anomaly.";
        choice annotator-type {
          description "The type of annotator who reported the anomaly.";
          mandatory "true";
          case human {
            leaf human {
              mandatory "true";
              type empty
            }
          }
          case algorithm {
            leaf algorithm {
              mandatory "true";
              type empty
            }
          }
        }
        leaf name {
          description "Name of the user annotator or the algorithm";
          mandatory "false";
          type empty;
        }
      }
      leaf state {
        type identityref {
          base network-anomaly-state;
        }
        mandatory true;
        description "State of the anomaly.";
      }
      list symptoms {
        key "symptom_id";
        description "List of symptoms identified by the symptom_id.";
        leaf symptom_id {
          type yang:uuid;
          description "UUID of the symptom that is part of this problem";
        }
      }
    }
  }
}
<CODE ENDS>

Implementation status
This section provides pointers to existing open source implementations of this draft. Note to the RFC-editor: Please remove this before publishing.

Antagonist
An open source implementation for this draft is called AnTagOnIst (Anomaly Tagging On hIstorical data), and it has been implemented in order to validate the application of the YANG model defined in this draft. Antagonist provides visual support for two important use cases in the scope of this document:
- the generation of a ground truth in relation to symptoms and problems in timeseries data
- the visual validation of results produced by automated network anomaly detection tools.
The open source code can be found here: [Antagonist]

Security Considerations
The security considerations will have to be updated according to "https://wiki.ietf.org/group/ops/yang-security-guidelines".

Acknowledgements
The authors would like to thank xxx for their review and valuable comments.

Normative References
[Antagonist]

Antagonist: Anomaly tagging on historical data
Vincenzo Riccobene
Antonio Roberto
Wanting Du
Thomas Graf
Alex Huang Feng
[I-D.netana-nmop-network-anomaly-architecture]
[I-D.netana-nmop-network-anomaly-semantics]
[I-D.ietf-nmop-terminology]
[RFC2119]
[RFC8174]
[RFC9417]
[RFC9418]
[RFC9232]