-
A Remedy for Over-Squashing in Graph Learning via Forman-Ricci Curvature based Graph-to-Hypergraph Structural Lifting
Authors:
Michael Banf,
Dominik Filipiak,
Max Schattauer,
Liliya Imasheva
Abstract:
Graph Neural Networks are highly effective at learning from relational data, leveraging node and edge features while maintaining the symmetries inherent to graph structures. However, many real-world systems, such as social or biological networks, exhibit complex interactions that are more naturally represented by higher-order topological domains. The emerging field of Geometric and Topological Dee…
▽ More
Graph Neural Networks are highly effective at learning from relational data, leveraging node and edge features while maintaining the symmetries inherent to graph structures. However, many real-world systems, such as social or biological networks, exhibit complex interactions that are more naturally represented by higher-order topological domains. The emerging field of Geometric and Topological Deep Learning addresses this challenge by introducing methods that utilize and benefit from higher-order structures. Central to TDL is the concept of lifting, which transforms data representations from basic graph forms to more expressive topologies before the application of GNN models for learning. In this work, we propose a structural lifting strategy using Forman-Ricci curvature, which defines an edge-based network characteristic based on Riemannian geometry. Curvature reveals local and global properties of a graph, such as a network's backbones, i.e. coarse, structure-preserving graph geometries that form connections between major communities - most suitably represented as hyperedges to model information flows between clusters across large distances in the network. To this end, our approach provides a remedy to the problem of information distortion in message passing across long distances and graph bottlenecks - a phenomenon known in graph learning as over-squashing.
△ Less
Submitted 15 August, 2025;
originally announced August 2025.
-
Tripartite-GraphRAG via Plugin Ontologies
Authors:
Michael Banf,
Johannes Kuhn
Abstract:
Large Language Models (LLMs) have shown remarkable capabilities across various domains, yet they struggle with knowledge-intensive tasks in areas that demand factual accuracy, e.g. industrial automation and healthcare. Key limitations include their tendency to hallucinate, lack of source traceability (provenance), and challenges in timely knowledge updates. Combining language models with knowledge…
▽ More
Large Language Models (LLMs) have shown remarkable capabilities across various domains, yet they struggle with knowledge-intensive tasks in areas that demand factual accuracy, e.g. industrial automation and healthcare. Key limitations include their tendency to hallucinate, lack of source traceability (provenance), and challenges in timely knowledge updates. Combining language models with knowledge graphs (GraphRAG) offers promising avenues for overcoming these deficits. However, a major challenge lies in creating such a knowledge graph in the first place. Here, we propose a novel approach that combines LLMs with a tripartite knowledge graph representation, which is constructed by connecting complex, domain-specific objects via a curated ontology of corresponding, domain-specific concepts to relevant sections within chunks of text through a concept-anchored pre-analysis of source documents starting from an initial lexical graph. Subsequently, we formulate LLM prompt creation as an unsupervised node classification problem allowing for the optimization of information density, coverage, and arrangement of LLM prompts at significantly reduced lengths. An initial experimental evaluation of our approach on a healthcare use case, involving multi-faceted analyses of patient anamneses given a set of medical concepts as well as a series of clinical guideline literature, indicates its potential to optimize information density, coverage, and arrangement of LLM prompts while significantly reducing their lengths, which, in turn, may lead to reduced costs as well as more consistent and reliable LLM outputs.
△ Less
Submitted 9 September, 2025; v1 submitted 28 April, 2025;
originally announced April 2025.
-
ICML Topological Deep Learning Challenge 2024: Beyond the Graph Domain
Authors:
Guillermo Bernárdez,
Lev Telyatnikov,
Marco Montagna,
Federica Baccini,
Mathilde Papillon,
Miquel Ferriol-Galmés,
Mustafa Hajij,
Theodore Papamarkou,
Maria Sofia Bucarelli,
Olga Zaghen,
Johan Mathe,
Audun Myers,
Scott Mahan,
Hansen Lillemark,
Sharvaree Vadgama,
Erik Bekkers,
Tim Doster,
Tegan Emerson,
Henry Kvinge,
Katrina Agate,
Nesreen K Ahmed,
Pengfei Bai,
Michael Banf,
Claudio Battiloro,
Maxim Beketov
, et al. (48 additional authors not shown)
Abstract:
This paper describes the 2nd edition of the ICML Topological Deep Learning Challenge that was hosted within the ICML 2024 ELLIS Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM). The challenge focused on the problem of representing data in different discrete topological domains in order to bridge the gap between Topological Deep Learning (TDL) and other types of…
▽ More
This paper describes the 2nd edition of the ICML Topological Deep Learning Challenge that was hosted within the ICML 2024 ELLIS Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM). The challenge focused on the problem of representing data in different discrete topological domains in order to bridge the gap between Topological Deep Learning (TDL) and other types of structured datasets (e.g. point clouds, graphs). Specifically, participants were asked to design and implement topological liftings, i.e. mappings between different data structures and topological domains --like hypergraphs, or simplicial/cell/combinatorial complexes. The challenge received 52 submissions satisfying all the requirements. This paper introduces the main scope of the challenge, and summarizes the main results and findings.
△ Less
Submitted 8 September, 2024;
originally announced September 2024.
-
TopoBench: A Framework for Benchmarking Topological Deep Learning
Authors:
Lev Telyatnikov,
Guillermo Bernardez,
Marco Montagna,
Mustafa Hajij,
Martin Carrasco,
Pavlo Vasylenko,
Mathilde Papillon,
Ghada Zamzmi,
Michael T. Schaub,
Jonas Verhellen,
Pavel Snopov,
Bertran Miquel-Oliver,
Manel Gil-Sorribes,
Alexis Molina,
Victor Guallar,
Theodore Long,
Julian Suk,
Patryk Rygiel,
Alexander Nikitin,
Giordan Escalona,
Michael Banf,
Dominik Filipiak,
Max Schattauer,
Liliya Imasheva,
Alvaro Martinez
, et al. (12 additional authors not shown)
Abstract:
This work introduces TopoBench, an open-source library designed to standardize benchmarking and accelerate research in topological deep learning (TDL). TopoBench decomposes TDL into a sequence of independent modules for data generation, loading, transforming and processing, as well as model training, optimization and evaluation. This modular organization provides flexibility for modifications and…
▽ More
This work introduces TopoBench, an open-source library designed to standardize benchmarking and accelerate research in topological deep learning (TDL). TopoBench decomposes TDL into a sequence of independent modules for data generation, loading, transforming and processing, as well as model training, optimization and evaluation. This modular organization provides flexibility for modifications and facilitates the adaptation and optimization of various TDL pipelines. A key feature of TopoBench is its support for transformations and lifting across topological domains. Mapping the topology and features of a graph to higher-order topological domains, such as simplicial and cell complexes, enables richer data representations and more fine-grained analyses. The applicability of TopoBench is demonstrated by benchmarking several TDL architectures across diverse tasks and datasets.
△ Less
Submitted 25 August, 2025; v1 submitted 9 June, 2024;
originally announced June 2024.
-
Artificial Intelligence based tool wear and defect prediction for special purpose milling machinery using low-cost acceleration sensor retrofits
Authors:
Mahmoud Kheir-Eddine,
Michael Banf,
Gregor Steinhagen
Abstract:
Milling machines form an integral part of many industrial processing chains. As a consequence, several machine learning based approaches for tool wear detection have been proposed in recent years, yet these methods mostly deal with standard milling machines, while machinery designed for more specialized tasks has gained only limited attention so far. This paper demonstrates the application of an a…
▽ More
Milling machines form an integral part of many industrial processing chains. As a consequence, several machine learning based approaches for tool wear detection have been proposed in recent years, yet these methods mostly deal with standard milling machines, while machinery designed for more specialized tasks has gained only limited attention so far. This paper demonstrates the application of an acceleration sensor to allow for convenient condition monitoring of such a special purpose machine, i.e. round seam milling machine. We examine a variety of conditions including blade wear and blade breakage as well as improper machine mounting or insufficient transmission belt tension. In addition, we presents different approaches to supervised failure recognition with limited amounts of training data. Hence, aside theoretical insights, our analysis is of high, practical importance, since retrofitting older machines with acceleration sensors and an on-edge classification setup comes at low cost and effort, yet provides valuable insights into the state of the machine and tools in particular and the production process in general.
△ Less
Submitted 7 February, 2022;
originally announced February 2022.
-
Who supervises the supervisor? Model monitoring in production using deep feature embeddings with applications to workpiece inspection
Authors:
Michael Banf,
Gregor Steinhagen
Abstract:
The automation of condition monitoring and workpiece inspection plays an essential role in maintaining high quality as well as high throughput of the manufacturing process. To this end, the recent rise of developments in machine learning has lead to vast improvements in the area of autonomous process supervision. However, the more complex and powerful these models become, the less transparent and…
▽ More
The automation of condition monitoring and workpiece inspection plays an essential role in maintaining high quality as well as high throughput of the manufacturing process. To this end, the recent rise of developments in machine learning has lead to vast improvements in the area of autonomous process supervision. However, the more complex and powerful these models become, the less transparent and explainable they generally are as well. One of the main challenges is the monitoring of live deployments of these machine learning systems and raising alerts when encountering events that might impact model performance. In particular, supervised classifiers are typically build under the assumption of stationarity in the underlying data distribution. For example, a visual inspection system trained on a set of material surface defects generally does not adapt or even recognize gradual changes in the data distribution - an issue known as "data drift" - such as the emergence of new types of surface defects. This, in turn, may lead to detrimental mispredictions, e.g. samples from new defect classes being classified as non-defective. To this end, it is desirable to provide real-time tracking of a classifier's performance to inform about the putative onset of additional error classes and the necessity for manual intervention with respect to classifier re-training. Here, we propose an unsupervised framework that acts on top of a supervised classification system, thereby harnessing its internal deep feature representations as a proxy to track changes in the data distribution during deployment and, hence, to anticipate classifier performance degradation.
△ Less
Submitted 17 January, 2022;
originally announced January 2022.
-
Learning Theory and Support Vector Machines - a primer
Authors:
Michael Banf
Abstract:
The main goal of statistical learning theory is to provide a fundamental framework for the problem of decision making and model construction based on sets of data. Here, we present a brief introduction to the fundamentals of statistical learning theory, in particular the difference between empirical and structural risk minimization, including one of its most prominent implementations, i.e. the Sup…
▽ More
The main goal of statistical learning theory is to provide a fundamental framework for the problem of decision making and model construction based on sets of data. Here, we present a brief introduction to the fundamentals of statistical learning theory, in particular the difference between empirical and structural risk minimization, including one of its most prominent implementations, i.e. the Support Vector Machine.
△ Less
Submitted 12 February, 2019;
originally announced February 2019.
