-
Predicting New Research Directions in Materials Science using Large Language Models and Concept Graphs
Authors:
Thomas Marwitz,
Alexander Colsmann,
Ben Breitung,
Christoph Brabec,
Christoph Kirchlechner,
Eva Blasco,
Gabriel Cadilha Marques,
Horst Hahn,
Michael Hirtz,
Pavel A. Levkin,
Yolita M. Eggeler,
Tobias Schlöder,
Pascal Friederich
Abstract:
Due to an exponential increase in published research articles, it is impossible for individual scientists to read all publications, even within their own research field. In this work, we investigate the use of large language models (LLMs) for the purpose of extracting the main concepts and semantic information from scientific abstracts in the domain of materials science to find links that were not…
▽ More
Due to an exponential increase in published research articles, it is impossible for individual scientists to read all publications, even within their own research field. In this work, we investigate the use of large language models (LLMs) for the purpose of extracting the main concepts and semantic information from scientific abstracts in the domain of materials science to find links that were not noticed by humans and thus to suggest inspiring near/mid-term future research directions. We show that LLMs can extract concepts more efficiently than automated keyword extraction methods to build a concept graph as an abstraction of the scientific literature. A machine learning model is trained to predict emerging combinations of concepts, i.e. new research ideas, based on historical data. We demonstrate that integrating semantic concept information leads to an increased prediction performance. The applicability of our model is demonstrated in qualitative interviews with domain experts based on individualized model suggestions. We show that the model can inspire materials scientists in their creative thinking process by predicting innovative combinations of topics that have not yet been investigated.
△ Less
Submitted 20 June, 2025;
originally announced June 2025.
-
On the Encapsulation of Medical Imaging AI Algorithms
Authors:
Hans Meine,
Yongli Mou,
Guido Prause,
Horst Hahn
Abstract:
In the context of collaborative AI research and development projects, it would be ideal to have self-contained encapsulated algorithms that can be easily shared between different parties, executed and validated on data at different sites, or trained in a federated manner. In practice, all of this is possible but greatly complicated, because human supervision and expert knowledge is needed to set u…
▽ More
In the context of collaborative AI research and development projects, it would be ideal to have self-contained encapsulated algorithms that can be easily shared between different parties, executed and validated on data at different sites, or trained in a federated manner. In practice, all of this is possible but greatly complicated, because human supervision and expert knowledge is needed to set up the execution of algorithms based on their documentation, possibly implicit assumptions, and knowledge about the execution environment and data involved.
We derive and formulate a range of detailed requirements from the above goal and from specific use cases, focusing on medical imaging AI algorithms. Furthermore, we refer to a number of existing APIs and implementations and review which aspects each of them addresses, which problems are still open, and which public standards and ontologies may be relevant. Our contribution is a comprehensive collection of aspects that have not yet been addressed in their entirety by any single solution.
Working towards the formulated goals should lead to more sustainable algorithm ecosystems and relates to the FAIR principles for research data, where this paper focuses on interoperability and (re)usability of medical imaging AI algorithms.
△ Less
Submitted 30 April, 2025;
originally announced April 2025.
-
Requirements for Quality Assurance of AI Models for Early Detection of Lung Cancer
Authors:
Horst K. Hahn,
Matthias S. May,
Volker Dicken,
Michael Walz,
Rainer Eßeling,
Bianca Lassen-Schmidt,
Robert Rischen,
Jens Vogel-Claussen,
Konstantin Nikolaou,
Jörg Barkhausen
Abstract:
Lung cancer is the second most common cancer and the leading cause of cancer-related deaths worldwide. Survival largely depends on tumor stage at diagnosis, and early detection with low-dose CT can significantly reduce mortality in high-risk patients. AI can improve the detection, measurement, and characterization of pulmonary nodules while reducing assessment time. However, the training data, fun…
▽ More
Lung cancer is the second most common cancer and the leading cause of cancer-related deaths worldwide. Survival largely depends on tumor stage at diagnosis, and early detection with low-dose CT can significantly reduce mortality in high-risk patients. AI can improve the detection, measurement, and characterization of pulmonary nodules while reducing assessment time. However, the training data, functionality, and performance of available AI systems vary considerably, complicating software selection and regulatory evaluation. Manufacturers must specify intended use and provide test statistics, but they can choose their training and test data, limiting standardization and comparability. Under the EU AI Act, consistent quality assurance is required for AI-based nodule detection, measurement, and characterization.
This position paper proposes systematic quality assurance grounded in a validated reference dataset, including real screening cases plus phantom data to verify volume and growth rate measurements. Regular updates shall reflect demographic shifts and technological advances, ensuring ongoing relevance. Consequently, ongoing AI quality assurance is vital. Regulatory challenges are also adressed. While the MDR and the EU AI Act set baseline requirements, they do not adequately address self-learning algorithms or their updates. A standardized, transparent quality assessment - based on sensitivity, specificity, and volumetric accuracy - enables an objective evaluation of each AI solution's strengths and weaknesses. Establishing clear testing criteria and systematically using updated reference data lay the groundwork for comparable performance metrics, informing tenders, guidelines, and recommendations.
△ Less
Submitted 24 February, 2025;
originally announced February 2025.
-
Comparison of different automatic solutions for resection cavity segmentation in postoperative MRI volumes including longitudinal acquisitions
Authors:
Luca Canalini,
Jan Klein,
Nuno Pedrosa de Barros,
Diana Maria Sima,
Dorothea Miller,
Horst Hahn
Abstract:
In this work, we compare five deep learning solutions to automatically segment the resection cavity in postoperative MRI. The proposed methods are based on the same 3D U-Net architecture. We use a dataset of postoperative MRI volumes, each including four MRI sequences and the ground truth of the corresponding resection cavity. Four solutions are trained with a different MRI sequence. Besides, a me…
▽ More
In this work, we compare five deep learning solutions to automatically segment the resection cavity in postoperative MRI. The proposed methods are based on the same 3D U-Net architecture. We use a dataset of postoperative MRI volumes, each including four MRI sequences and the ground truth of the corresponding resection cavity. Four solutions are trained with a different MRI sequence. Besides, a method designed with all the available sequences is also presented. Our experiments show that the method trained only with the T1 weighted contrast-enhanced MRI sequence achieves the best results, with a median DICE index of 0.81.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients
Authors:
Bhakti Baheti,
Satrajit Chakrabarty,
Hamed Akbari,
Michel Bilello,
Benedikt Wiestler,
Julian Schwarting,
Evan Calabrese,
Jeffrey Rudie,
Syed Abidi,
Mina Mousa,
Javier Villanueva-Meyer,
Brandon K. K. Fields,
Florian Kofler,
Russell Takeshi Shinohara,
Juan Eugenio Iglesias,
Tony C. W. Mok,
Albert C. S. Chung,
Marek Wodzinski,
Artur Jurgas,
Niccolo Marini,
Manfredo Atzori,
Henning Muller,
Christoph Grobroehmer,
Hanna Siebert,
Lasse Hansen
, et al. (48 additional authors not shown)
Abstract:
Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registr…
▽ More
Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/.
△ Less
Submitted 17 April, 2024; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Robust Segmentation Models using an Uncertainty Slice Sampling Based Annotation Workflow
Authors:
Grzegorz Chlebus,
Andrea Schenk,
Horst K. Hahn,
Bram van Ginneken,
Hans Meine
Abstract:
Semantic segmentation neural networks require pixel-level annotations in large quantities to achieve a good performance. In the medical domain, such annotations are expensive, because they are time-consuming and require expert knowledge. Active learning optimizes the annotation effort by devising strategies to select cases for labeling that are most informative to the model. In this work, we propo…
▽ More
Semantic segmentation neural networks require pixel-level annotations in large quantities to achieve a good performance. In the medical domain, such annotations are expensive, because they are time-consuming and require expert knowledge. Active learning optimizes the annotation effort by devising strategies to select cases for labeling that are most informative to the model. In this work, we propose an uncertainty slice sampling (USS) strategy for semantic segmentation of 3D medical volumes that selects 2D image slices for annotation and compare it with various other strategies. We demonstrate the efficiency of USS on a CT liver segmentation task using multi-site data. After five iterations, the training data resulting from USS consisted of 2410 slices (4% of all slices in the data pool) compared to 8121 (13%), 8641 (14%), and 3730 (6%) for uncertainty volume (UVS), random volume (RVS), and random slice (RSS) sampling, respectively. Despite being trained on the smallest amount of data, the model based on the USS strategy evaluated on 234 test volumes significantly outperformed models trained according to other strategies and achieved a mean Dice index of 0.964, a relative volume error of 4.2%, a mean surface distance of 1.35 mm, and a Hausdorff distance of 23.4 mm. This was only slightly inferior to 0.967, 3.8%, 1.18 mm, and 22.9 mm achieved by a model trained on all available data, but the robustness analysis using the 5th percentile of Dice and the 95th percentile of the remaining metrics demonstrated that USS resulted not only in the most robust model compared to other sampling schemes, but also outperformed the model trained on all data according to Dice (0.946 vs. 0.945) and mean surface distance (1.92 mm vs. 2.03 mm).
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Anisotropic 3D Multi-Stream CNN for Accurate Prostate Segmentation from Multi-Planar MRI
Authors:
Anneke Meyer,
Grzegorz Chlebus,
Marko Rak,
Daniel Schindele,
Martin Schostak,
Bram van Ginneken,
Andrea Schenk,
Hans Meine,
Horst K. Hahn,
Andreas Schreiber,
Christian Hansen
Abstract:
Background and Objective: Accurate and reliable segmentation of the prostate gland in MR images can support the clinical assessment of prostate cancer, as well as the planning and monitoring of focal and loco-regional therapeutic interventions. Despite the availability of multi-planar MR scans due to standardized protocols, the majority of segmentation approaches presented in the literature consid…
▽ More
Background and Objective: Accurate and reliable segmentation of the prostate gland in MR images can support the clinical assessment of prostate cancer, as well as the planning and monitoring of focal and loco-regional therapeutic interventions. Despite the availability of multi-planar MR scans due to standardized protocols, the majority of segmentation approaches presented in the literature consider the axial scans only. Methods: We propose an anisotropic 3D multi-stream CNN architecture, which processes additional scan directions to produce a higher-resolution isotropic prostate segmentation. We investigate two variants of our architecture, which work on two (dual-plane) and three (triple-plane) image orientations, respectively. We compare them with the standard baseline (single-plane) used in literature, i.e., plain axial segmentation. To realize a fair comparison, we employ a hyperparameter optimization strategy to select optimal configurations for the individual approaches. Results: Training and evaluation on two datasets spanning multiple sites obtain statistical significant improvement over the plain axial segmentation ($p<0.05$ on the Dice similarity coefficient). The improvement can be observed especially at the base ($0.898$ single-plane vs. $0.906$ triple-plane) and apex ($0.888$ single-plane vs. $0.901$ dual-plane). Conclusion: This study indicates that models employing two or three scan directions are superior to plain axial segmentation. The knowledge of precise boundaries of the prostate is crucial for the conservation of risk structures. Thus, the proposed models have the potential to improve the outcome of prostate cancer diagnosis and therapies.
△ Less
Submitted 2 December, 2020; v1 submitted 23 September, 2020;
originally announced September 2020.
-
A Ray-based Approach for Boundary Estimation of Fiber Bundles Derived from Diffusion Tensor Imaging
Authors:
Miriam H. A. Bauer,
Sebastiano Barbieri,
Jan Klein,
Jan Egger,
Daniela Kuhnt,
Bernd Freisleben,
Horst K. Hahn,
Christopher Nimsky
Abstract:
Diffusion Tensor Imaging (DTI) is a non-invasive imaging technique that allows estimation of the location of white matter tracts in-vivo, based on the measurement of water diffusion properties. For each voxel, a second-order tensor can be calculated by using diffusion-weighted sequences (DWI) that are sensitive to the random motion of water molecules. Given at least 6 diffusion-weighted images wit…
▽ More
Diffusion Tensor Imaging (DTI) is a non-invasive imaging technique that allows estimation of the location of white matter tracts in-vivo, based on the measurement of water diffusion properties. For each voxel, a second-order tensor can be calculated by using diffusion-weighted sequences (DWI) that are sensitive to the random motion of water molecules. Given at least 6 diffusion-weighted images with different gradients and one unweighted image, the coefficients of the symmetric diffusion tensor matrix can be calculated. Deriving the eigensystem of the tensor, the eigenvectors and eigenvalues can be calculated to describe the three main directions of diffusion and its magnitude. Using DTI data, fiber bundles can be determined, to gain information about eloquent brain structures. Especially in neurosurgery, information about location and dimension of eloquent structures like the corticospinal tract or the visual pathways is of major interest. Therefore, the fiber bundle boundary has to be determined. In this paper, a novel ray-based approach for boundary estimation of tubular structures is presented.
△ Less
Submitted 22 October, 2013;
originally announced October 2013.
-
Benchmarking the Quality of Diffusion-Weighted Images
Authors:
Jan Klein,
Sebastiano Barbieri,
Miriam H. A. Bauer,
Christopher Nimsky,
Horst K. Hahn
Abstract:
We present a novel method that allows for measuring the quality of diffusion-weighted MR images dependent on the image resolution and the image noise. For this purpose, we introduce a new thresholding technique so that noise and the signal can automatically be estimated from a single data set. Thus, no user interaction as well as no double acquisition technique, which requires a time-consuming pro…
▽ More
We present a novel method that allows for measuring the quality of diffusion-weighted MR images dependent on the image resolution and the image noise. For this purpose, we introduce a new thresholding technique so that noise and the signal can automatically be estimated from a single data set. Thus, no user interaction as well as no double acquisition technique, which requires a time-consuming proper geometrical registration, is needed. As a coarser image resolution or slice thickness leads to a higher signal-to-noise ratio (SNR), our benchmark determines a resolution-independent quality measure so that images with different resolutions can be adequately compared. To evaluate our method, a set of diffusion-weighted images from different vendors is used. It is shown that the quality can efficiently be determined and that the automatically computed SNR is comparable to the SNR which is measured manually in a manually selected region of interest.
△ Less
Submitted 9 May, 2011; v1 submitted 8 April, 2011;
originally announced April 2011.
-
Ray-Based and Graph-Based Methods for Fiber Bundle Boundary Estimation
Authors:
Miriam H. A. Bauer,
Jan Egger,
Daniela Kuhnt,
Sebastiano Barbieri,
Jan Klein,
Horst K. Hahn,
Bernd Freisleben,
Christopher Nimsky
Abstract:
Diffusion Tensor Imaging (DTI) provides the possibility of estimating the location and course of eloquent structures in the human brain. Knowledge about this is of high importance for preoperative planning of neurosurgical interventions and for intraoperative guidance by neuronavigation in order to minimize postoperative neurological deficits. Therefore, the segmentation of these structures as clo…
▽ More
Diffusion Tensor Imaging (DTI) provides the possibility of estimating the location and course of eloquent structures in the human brain. Knowledge about this is of high importance for preoperative planning of neurosurgical interventions and for intraoperative guidance by neuronavigation in order to minimize postoperative neurological deficits. Therefore, the segmentation of these structures as closed, three-dimensional object is necessary. In this contribution, two methods for fiber bundle segmentation between two defined regions are compared using software phantoms (abstract model and anatomical phantom modeling the right corticospinal tract). One method uses evaluation points from sampled rays as candidates for boundary points, the other method sets up a directed and weighted (depending on a scalar measure) graph and performs a min-cut for optimal segmentation results. Comparison is done by using the Dice Similarity Coefficient (DSC), a measure for spatial overlap of different segmentation results.
△ Less
Submitted 10 March, 2011;
originally announced March 2011.
-
A Semi-Automatic Graph-Based Approach for Determining the Boundary of Eloquent Fiber Bundles in the Human Brain
Authors:
Miriam H. A. Bauer,
Jan Egger,
Daniela Kuhnt,
Sebastiano Barbieri,
Jan Klein,
Horst K. Hahn,
Bernd Freisleben,
Christopher Nimsky
Abstract:
Diffusion Tensor Imaging (DTI) allows estimating the position, orientation and dimension of bundles of nerve pathways. This non-invasive imaging technique takes advantage of the diffusion of water molecules and determines the diffusion coefficients for every voxel of the data set. The identification of the diffusion coefficients and the derivation of information about fiber bundles is of major int…
▽ More
Diffusion Tensor Imaging (DTI) allows estimating the position, orientation and dimension of bundles of nerve pathways. This non-invasive imaging technique takes advantage of the diffusion of water molecules and determines the diffusion coefficients for every voxel of the data set. The identification of the diffusion coefficients and the derivation of information about fiber bundles is of major interest for planning and performing neurosurgical interventions. To minimize the risk of neural deficits during brain surgery as tumor resection (e.g. glioma), the segmentation and integration of the results in the operating room is of prime importance. In this contribution, a robust and efficient graph-based approach for segmentating tubular fiber bundles in the human brain is presented. To define a cost function, the fractional anisotropy (FA) is used, derived from the DTI data, but this value may differ from patient to patient. Besides manually definining seed regions describing the structure of interest, additionally a manual definition of the cost function by the user is necessary. To improve the approach the contribution introduces a solution for automatically determining the cost function by using different 3D masks for each individual data set.
△ Less
Submitted 8 March, 2011;
originally announced March 2011.
