For small, deployed teams to gain insight into sensor data, they require the ability to fuse and integrate multi-domain sensing data from a variety of UxS (Unmanned Systems) platforms, Tactical Intelligence & electronic warfare (EW), publicly available information (PAI), as well as other necessary data sources for specific outcomes and effects. Certus Core asserts that to fuse that data and make it valuable it needs context which must be generated by, and from the mission those users are supporting. Certus Core provides a method for non-technical users to generate that context and integrate it as guiding elements for data integration enabling better, more impactful data products that can be queried in natural language, connected to analytic tools, or have models deployed to. This further enables non-technical users to be more connected to the output of the data work, for data professionals to be more impactful in their organizations, and analysts to be generating value based on the context of the mission. Certus Core is currently working on a project with a defense customer during which we are integrating UxS-based sensor data with PAI to provide that valuable context and rapid insight for a non- technical user by integrating the users into the data engineering process.

Certus Core is integrating data from sensor missions using man portable UxSs collaboratively over a large inaccessible area. The returned data from the mission is Light Detection and Ranging (LiDAR) data which needs to be integrated with telemetry data from the UxS, as well as PAI from the local area such as Open Street Maps. Complementing this data, Certus is integrating vessel specific data from Automatic Identification System sources (AIS) in order to aid Strategic Reconnaissance in contested environments.

In order to incorporate and integrate these domains of data for missions of strategic reconnaissance and autonomous ISR related collection, processing, exploitation, and dissemination, Certus Core is using their Semantic Knowledge Graph (sKG) software as the foundational solution. sKG is an ontology-based graph approach to structuring data that integrates mission concepts into the development of knowledge graphs, it then makes those Knowledge Graphs accessible to non-technical users through a Natural Language (English Chat) interface that can be accessed via any application (such as ATAK) via API. In this project we are using sKG to support unmanned systems architecture and infrastructure to apply and leverage artificial intelligence / machine learning (AI/ML) algorithms and models to those systems. sKG includes a Data Operations framework enabling Data Management, Data Governance, and Data Life Cycle Management; a semantic Extract Transform and Load (ETL) process that enables output to the ontology structure, if the data source is updated or changes, this structure allows for ease in scalability and extensibility; and a semantic Knowledge Graph datastore that enables the storage of data and applicable query indices into a graph database enabling multiple front-end user query mechanisms (natural language query via LLM, API-based through third party analytic tools, keyword or graph traversal) thereby supporting users dynamic missions sets and data visualization preferences.

Enabling the creation of a consistent data product, data management solutions, and data operations methodology enables our customer to perform novel analytics such as Graph Neural Networks (GNN) based classification of sub-graphs as well as other novel ML methodologies that can be maintained independent of the data product and if the data sources change. Graph based query methodologies (such as shortest path queries) and GNN based classification could lead to rapid deconfliction between UxS and further scalable systems. sKG users could apply a variety of graph-based querying and machine learning techniques to gain insight from the knowledge graph once created based on priorities and requirements. Due to the geographic elements of the structure of the knowledge graph it could also be used to support strategic operations enabling immediate answers to geographic based queries.

In the specific use case referenced, Certus Core is deploying sKG as a Proof-of-Concept (PoC) for transition to a production deployment. Due to the nature of the UxS data being generated (underwater LiDAR) and the fact the first effort was a PoC we generated synthetic data to model the output for the customer. The produced integrated data set incorporates telemetry data from the UxS, PAI that includes geographic and vessel information, and detection events with the resulting model classification. In the figure below you can see the ontology (which is now the graph schema) and an example natural language query prompt “Show me vehicles that were detected in China” The resulting gremlin query it generates is also depicted as well as the summary response being in the screenshot below it.

The unique Retrieval Augmented Generation (RAG) architecture for sKG is such that we are generating the property graph query in gremlin using the prompt supplied by the user. That query produces the database query result which is then summarized by the Large Language Model (LLM).

Certus Core believes that a knowledge graph system is the best data product to support integrating data sources to support decision-making. This method enables users to interact with relevant data sources that have been transformed into concepts from their mission goals so they can ask the most relevant questions of the data to them. sKG supports both out-of-the-box machine learning models and customized (trained on the data sources) models to provide force-multiplying automation that enables users to answer questions more rapidly.

‍