Knowledge Graphs for Enhanced Cross-Operator Incident Management and Network Design

Experiments

Experimental Plan In terms of experimentation, we consider the YANG-KG-SEMANTIC-GENERALIZATION case defined in as the reference approach and recommend implementing a data processing pipeline that performs the following use cases:

Y-MODEL-FROM-DATA:: Based on a dataset of configuration data expressed in YANG data models, the goal is to enable extracting the list of models involved for their conversion to their RDFS/OWL equivalent.
Y-MODEL-DEPENDENCIES:: Based on a given YANG data model, the goal is to enable identifying and retrieving all the YANG data models that the model refers to, in order to build a complete corpus of models for their conversion to their RDFS/OWL equivalent as a coherent set.
Y-MODEL-TO-RDFS-OWL:: Based on a YANG data model and the associated model corpus (i.e., Y-MODEL-DEPENDENCIES), the goal is to enable producing a semantically equivalent RDFS/OWL representation (i.e., ONTO-YANG-MODEL).
: Ideally, a YANG to RDFS/OWL/YANG projection algebra would be used to provide a formal proof of semantic equivalence; testing mechanisms should be implemented as a fallback to provide a proof of equivalence.
Y-INSTANCE-TO-KG:: Based on a dataset of configuration data expressed in YANG data models and the related (set of) ONTO-YANG-MODEL, the goal is to enable constructing a knowledge graph from the configuration data, with the knowledge graph structured by the (set of) ONTO-YANG-MODEL.
Y-MODEL-META-KG-ALIGNMENT:: Based on a corpus of YANG data models transformed into RDFS/OWL (i.e., Y-MODEL-TO-RDFS-OWL) and a reference ontology structuring the ITSM-KG, the goal is to enable querying of the configuration entities present in the graph (i.e., data derived from the Y-INSTANCE-TO-KG case) through the concepts of the reference ontology.
: In addition to identifying the class and property correspondences between the resulting Y-MODEL-TO-RDFS-OWL models and the reference ontology, this capability requires implementing a necessary and sufficient number of class equivalence relations and property equivalence relations.
META-KG-BEHAVIORAL-MODEL:: Based on the ITSM-KG, which results from the composition of the Y-INSTANCE-TO-KG case with Y-MODEL-META-KG-ALIGNMENT and additional operational data structured by ONTO-META, the goal is to learn behavioral models (e.g., incident signatures) in a formalism that can be interpreted through the lenses of ONTO-ITSM and shared with other stakeholders with minimal discrepancies in the underlying configuration data.

Implementation Status This section provides pointers to existing open source implementations of this document or in close relation to it.

NORIA The NORIA project aims at enabling advanced network anomaly detection using knowledge graphs. Among the components resulting from this project, the following ones serve the use case described in this document:

NORIA-O , is a data model for IT networks, events and operations information. The ontology is developed using web technologies (e.g., RDF, OWL, SKOS) and is intended as a structure for realizing an ITSM knowledge graph for Anomaly Detection (AD) and Risk Management applications. The NORIA-O implementation is available as open source at https://w3id.org/noria/. Its use for anomaly detection is discussed in:
- with a model-based design approach (i.e., query the graph to retrieve anomalies and their context) and a statistical learning approach (i.e., relate entities based on context similarities, then use this relatedness to alert and guide the repair).
- with a process mining approach to align a sequence of entities to activity models, then use this relatedness to guide the repair actions.
- a Web-based knowledge graph exploration design for incident management that combines the above and techniques for broader coverage of anomaly cases and knowledge capitalization.
A knowledge graph-based platform design using Semantic Web technologies and open source data integration tools to build an ITSM knowledge graph:
- SMASSIF-RML, a Semantic Web stream processing solution with declarative data mapping capability. Available as open source at https://github.com/Orange-OpenSource/smassif-rml.
- ssb-consum-up, a Kafka to SPARQL gateway enabling end-to-end Semantic Web data flow architecture with a Semantic Service Bus (SSB) approach. Available as open source at https://github.com/Orange-OpenSource/ssb-consum-up.
- grlc, a fork of CLARIAH/grlc with SPARQL UPDATE and GitLab interface features to facilitate the call and versioning of stored user queries in SPARQL syntax (e.g., for anomaly detection following the model-based design approach). Available as open source at https://github.com/Orange-OpenSource/grlc.
SemNIDS , a test bench involving network trafic generation, open source Network Intrusion Detection Systems (NIDS), knowledge graphs, process mining and conformance checking components.

Note that the NORIA project does not currently address the Y-MODEL-FROM-DATA, Y-MODEL-DEPENDENCIES, and Y-MODEL-TO-RDFS-OWL use cases.

YANG2OWL The YANG2OWL framework aims at facilitating the implementation of a Network Digital Twin (NDT) that would leverage the representation and reasoning capabilities typically associated with knowledge graphs for anomaly detection needs, as well as for network management purposes by enabling network configuration based on modifications at the level of the ITSM-KG itself. Basically, the approach consists of reusing YANG data models used in network operations in a nearly equivalent form within Semantic Web technologies (i.e., producing ONTO-YANG-MODEL instances) to create a bijection between network configuration data and the NDT. The YANG2OWL framework addresses the use cases Y-MODEL-TO-RDFS-OWL and Y-INSTANCE-TO-KG (as defined in ). illustrates the top-level tasks of the semantization process at play. Subsequent sections detail how the framework builds ontologies that captures the specificities of the telco domain and models any telco network instance as an ITSM-KG. Please note that the publication of the related tools and algorithms is in progress.

The YANG2OWL framework. Labels within boxes represent automated or human actions, while labels between top/bottom lines represent datasets

Motivations and Principles The document (Knowledge Graph Framework for Network Operations) emphasizes the importance of ontologies alongside knowledge graphs for network management automation. However, it lacks guidance on creating these ontologies and provides limited details on generating knowledge graphs or their relationship with the ontologies. To address these topics, the following principles have been considered to underpin the development of the YANG2OWL approach:

The ontologies should intimately reflect YANG data models,
The generation of ontologies should be mostly automatized,
The knowledge graphs should intimately reflect the payload of messages that YANG compliant network equipments and controlers publish or emit in response to a Remote Procedure Call (RPC) request,
The generation of knowledge graphs should be automated,
The nodes and predicates of the knowledge graphs should be defined as instances of classes and properties of the ontologies.

Point 1 of the proposed principles is essential for ensuring the engagement of network administrators and experts in semantic technology. Aligning the ontology's vocabulary (class and relationship naming) and semantics (relationship constraints) with that of network managers is crucial. The YANG language is currently the reference in this area and will continue to be so, given its specification by the IETF and support from major telco industry players. This necessity has driven the development of the YANG2OWL framework for converting YANG data models into OWL models, which corresponds to point 2 of the proposal. Points 3, 4, and 5 are direct outcomes of the commitment to points 1 and 2.

The Y-MODEL-TO-RDFS-OWL step YANG and OWL are both data modeling languages. They define a vocabulary and a grammar. The vocabulary defines the concept of the domain. YANG domain is the telco domain. In a natural language, the vocabulary defines nouns, verbs, adjectives, and adverbs that are useful for discussing the world. The grammar specifies how these elements should be assembled into sentences that describe a state of the world. In a YANG data model, the vocabulary is defined in terms of containers, lists, leaves, leaf-lists, and other categories, while the grammar is defined in terms of statements that relate these elements to one another. In an OWL ontology, the vocabulary is defined in terms of classes, subclasses, object properties, and data properties, which is somewhat similar to YANG but does not directly map. As ontologies have been introduced as a modeling language meant to share a common view (or knowledge) of a domain among different stakeholders , the terms defined by the ontologies should reflect those used by equipment manufacturers, telecom solutions developers, systems integrators, network operators, and ultimately end users. A YANG data model is a document containing declarations. The document has a tree-like structure: declarations can contain other declarations. There are about half hundred types of declarations. The main ones are container, list, leaf and leaf-list:

CONTAINER:: It is a concept, something we can talk about ; it is the the basic type of elements of the domain, such as a network, a node, a link. A container declaration can contain another container declaration that can be called a sub-container. This sub-container allows to define a concept that will characterize the container that contains it (e.g., link, source, and destination).
LIST:: It is a concept that can have multiple instances, such as nodes of a network.
LEAF:: It is a property of this concept, such as an identifier or a geographical location.
LEAF-LIST:: It is a multivalued property, such as hours of the day the device is in sleep mode.

By applying the above principles, and in line with the reasons sketched in , we have developed the YANG2OWL that automatically generates OWL ontologies from YANG modules (i.e., computes ONTO-YANG-MODELs). sketches the use of the YANG2OWL tool to compute the org.opendaylight.yangtools ONTO-YANG-MODEL.

Computing the org.opendaylight.yangtools ONTO-YANG-MODEL with YANG2OWL. In more detail, we have defined mapping rules between YANG constructs and OWL concepts and implemented these in YANG2OWL. The main YANG constructs (container, list, leaf, and leaf-list) are transformed as follows:

The container and list declarations are converted into OWL classes. The name of the OWL class correponds to the name of the container or list in the YANG data model.
The leaf and leaf-list declarations are converted into OWL data properties. The name of the OWL data property corresponds the name of the leaf or leaf-list in the YANG data model.

An example of this conversion is presented in the following section.

The Y-INSTANCE-TO-KG step As introduced above, YANG data models define the vocabulary and grammar to describe factual knowledge about the state of the network. For example if a YANG module defines the container node, and this container has a leaf identifier which has the type string, then a valid JSON document with configuration data describing a node should be a JSON object containing a key named identifier which value should be a string such as router_253. So, in line with the mapping rules of YANG statement into OWL concepts defined in , when parsing a JSON tree that comply to a given YANG data model we can assume that if we get a key which value is a JSON object then the key should be the name of a container or a list and its value should be a description to be further analyzed. Thus, in terms of knowledge graph modeling, this JSON object should be interpreted as an instance of a class which name is the name of the container or of the list. Conversely, if the value is a litteral, the key should be the name of a leaf or a leaf-list. Thus, in terms of knowledge graph modeling, the litteral should be interpreted as the object of a DataProperty which name is the name of the leaf. The JSON2RDF tool (which is part of the YANG2OWL framework) implements these principles, realizing the Y-INSTANCE-TO-KG use case. shows the algorithm implemented by JSON2RDF as pseudo code.

Pseudo code of the algorithm implemented by JSON2RDF. of this annotation search the key in the jsonObject append the key to the namespace to create the URI } else { generate a unique URI } return the URI created } function createObject(URI, class) { return an instance of the class with the given URI } function parse(object, parentURI, class, namespace, ontology) { objectURI = createURI(object,class, namespace, ontology) createObject(objectURI, class) for each key of object { if the value of object[key] is a list { for each elt of the list { if elt is an object { parse(elt, objectURI, key, namespace, ontology) create the triple } else if elt is a literal create the triple } else if the value of object[key] is an object { eltURI = createURI(elt,key, namespace, ontology) create the triple parse(elt, objectURI, key, namespace, ontology) } else if the value of object[key] is literal { create the triple } } } ]]> The algorithm is initiated by calling the parse function as follows, where top is the root of the JSON object (i.e., configuration data as a JSON tree that complies to a given YANG data model), and ontology is the output of the Y-MODEL-TO-RDFS-OWL step:

Example of Implementation To illustrate the YANG2OWL approach, this section briefly reports on an experiment conducted in an industrial setting with data from a virtualized 5G infrastructure. In the context of the Network Change Management process, impact analysis prior to conducting a scheduled operation can be run on an ITSM-KG. It aims to determine all the components of the 5G core network that are dependent of a given (set of) network infrastructure element. For example, for a scheduled operation on a leaf node (i.e., a network element in a 2-tier spine-leaf architecture), the impact calculus will return all the servers connected to the leaf, all the Virtual Machines (VMs) hosted on these servers, all the Network Functions (NFs) deployed on these VMs, and ideally all the telecom services using these NFs. provides an overview of the data processing workflow used for the experiment. The tasks of the diagram are described below.

Flowchart for the YANG2OWL experiment. A left vertical bar on a step indicates that it is scripted; otherwise, steps require user or operator action.

Model Gathering:: This task corresponds to the realization of the Y-MODEL-FROM-DATA use case with the manual selection of YANG modules in relation to the 3GPP application domain. The YANG modules from have been selected for this experiment.
Model Translation:: For a given YANG module, this task implements the Y-MODEL-DEPENDENCIES use case by fetching sub-YANG modules from well-known GitHub repositories used for storing YANG modules (e.g., IETF, IEEE, IANA, ETSI, broadband forum, OpenROADM, OpenConfig, Cisco, Huawei, to name a few). This is achieved by scrutinizing import clauses (including imports of imports) and examining module locations and relationships from the . Additionally, it addresses the Y-MODEL-TO-RDFS-OWL use case using the YANG2OWL solution defined in . For this experiment, the resulting ontology is referred to as MOBILE-O.
Model Curation:: This task involves providing a streamlined ontology by manually filtering (selection of classes and relationships based on the data available) and grouping (compression of the model hierarchy, i.e., class of classes) the model resulting from the Model Translation task. This simplification aims to enhance the readability of the model for an operator and facilitate the implementation of potentially more concise queries in the downstream Use Cases-Related Querying task.
Model-Related Knowledge Graph Construction:: It realizes the Y-INSTANCE-TO-KG use case using the JSON2RDF solution described in .
NetOps-Related Knowledge Graph Construction:: It corresponds to the execution of RML transformation rules with definitions from the NORIA-O ontology for the integration of complementary data to that of the 5G network derived from YANG configurations (i.e., the Model-Related Knowledge Graph Construction task), such as the topology of connected networks, scheduled operations, incident tickets, and organization-related data.
Global Knowledge Graph Construction:: It is achieved through parallel insertions into a graph database of the results from the Model-Related and NetOps-Related tasks, after ensuring that: 1) the URI patterns implemented in the RML rules of the NetOps-Related step are consistent with the URIs produced by the Model-Related step to benefit from automatic linking of triples within the graph database through the uniqueness of the URIs; 2) the definition of mappings between MOBILE-O and NORIA-O has been implemented and inserted into the graph database (i.e., realization of the Y-MODEL-META-KG-ALIGNMENT use case through the implementation of the ONTO-LINKER concept as illustrated in ). For this experiment, the graph database is a Neo4j database instance, and the loading is performed using the Neo4j Neosemantics toolkit.
Use Cases-Related Pre-Processing:: Dependency relationships are, in general, knowledge elements that cannot be directly derived from field data; they are part of the business knowledge regarding the operation of the network systems. It may therefore be beneficial to support the downstream Use Cases-Related Querying task by performing pre-processing, particularly by calculating these dependency relationships retrospectively from business rules and the data loaded into the database. For example, one can create a (Server)-[DEPENDS_ON]->(Leaf) relationship by searching instances of the (Server)-(Server Interface)-(Network Link)-(Leaf Interface)-(Leaf) graph pattern. The same principle can apply to different network configurations to create other kinds of dependency relationships.
: For this experiment, the dependency relationships are calculated directly in the graph database using Neo4j Cypher language queries, or externally to the graph database using SHACL shapes according to the principles described in . As another example, more specific to the 3GPP models included in MOBILE-O and the Neo4j setup, one could calculate a dependency relationship between a 5G NF and the Kubernetes cluster that hosts it, as shown in . It is important to note that subclass inference with Neo4j is not automatic and must be performed through dedicated queries, as illustrated in .

Dependency calculation query, in Cypher syntax, for relating a 5G NF and the Kubernetes cluster that hosts it. (k) ]]>

Subclass inference query, in Cypher syntax, to tag 5G NF entities as `ManagedFunction` based on prior annotation of the entities at creation time with a specific class described in the YANG data model, which is also a subclass of `ManagedFunction` as per MOBILE-O.

Use Cases-Related Querying:: The exploitation of dependency relationships is carried out through queries on the graph, e.g., during the insertion of an entity of type noria:ChangeRequest or by following an exploratory approach by coupling a query such as that in with a visualization tool like Neo4j NeoDash.

User query, in Cypher syntax using a quantified path pattern, for rendering dependency relationships in a Neo4j NeoDash display. The query seeks paths starting from the node `e1` and propagates up to 8 times using the `DEPENDS_ON` relationships. The depth of 8 has been defined in relation to the characteristics of the networks addressed in the experimentation.

Situation Analysis:: Decision-making based on the results of the upstream task is the responsibility of the network administrator, potentially supported by a complementary exploration of the ITSM-KG performed algorithmically or interactively to analyze a broader technical and operational context.

Discussion While the YANG2OWL approach has proven its validity as a proof of concept, several R&D questions remain for exploration with the NMOP community, including:

Are the conversion principles based on statement types (class vs. data property) in the Y-MODEL-TO-RDFS-OWL use case universally applicable?
How to ensure that an ITSM-KG can still be generically constructed from JSON/YANG data and queried when a Model Curation task is applied on an ONTO-YANG-MODEL?
What techniques can automate the Y-MODEL-META-KG-ALIGNMENT use case?
What principles should guide the implementation of the Y-MODEL-META-KG-ALIGNMENT use case to extract an aggregated view from ONTO-META of infrastructures/configurations represented by an ONTO-YANG-MODEL (e.g., distinguishing devices from sub-devices)?
As evoked in (NEW-OPS-REQ-QUICK-BUT-WELL), how can we ensure reliable retrieval of dependencies between YANG modules for the Y-MODEL-DEPENDENCIES use case? Indeed, while browsing the GitHub projects of module developers, we observe a lack of uniformity in the way modules are presented and managed (e.g., differences in project structure, replication and local modifications of reference modules), which hinders dependency calculation and the sound inclusion of sub-modules in the YANG2OWL translation process.

Furthermore, it is noteworthy that the YANG2OWL approach is complementary to the YANG2RDF approach , which consists in translating YANG data models into RDF. More specifically, YANG2RDF defines an ontology of the YANG language, where RDF graph instances model a YANG module. This approach is useful for querying YANG data models. In contrast, the YANG2OWL approach defines an ontology of a YANG data model, where RDF graph instances model an operational network. Future work may aim to combine the YANG2RDF and YANG2OWL approaches. Finally, it is noteworthy that the YANG2OWL framework automates the Ontology Implementation and Ontology Update activities of the LOT4KG methodology (a methodology that extends the well-known LOT ontology engineering methodology to include knowledge graph lifecycle management) by linking YANG modules with ITSM-KG fragment construction. This streamlines the development of NDT architectures based on knowledge graphs and simplifies ITSM-KG updates when YANG modules change.