-
Symmetry constrained neural networks for detection and localization of damage in metal plates
Authors:
James Amarel,
Christopher Rudolf,
Athanasios Iliopoulos,
John Michopoulos,
Leslie N. Smith
Abstract:
The present paper is concerned with deep learning techniques applied to detection and localization of damage in a thin aluminum plate. We used data collected on a tabletop apparatus by mounting to the plate four piezoelectric transducers, each of which took turn to generate a Lamb wave that then traversed the region of interest before being received by the remaining three sensors. On training a ne…
▽ More
The present paper is concerned with deep learning techniques applied to detection and localization of damage in a thin aluminum plate. We used data collected on a tabletop apparatus by mounting to the plate four piezoelectric transducers, each of which took turn to generate a Lamb wave that then traversed the region of interest before being received by the remaining three sensors. On training a neural network to analyze time-series data of the material response, which displayed damage-reflective features whenever the plate guided waves interacted with a contact load, we achieved a model that detected with greater than $99\%$ accuracy in addition to a model that localized with $2.58 \pm 0.12$ mm mean distance error. For each task, the best-performing model was designed according to the inductive bias that our transducers were both similar and arranged in a square pattern on a nearly uniform plate.
△ Less
Submitted 26 May, 2025; v1 submitted 9 September, 2024;
originally announced September 2024.
-
Detection of Anomalies in Multivariate Time Series Using Ensemble Techniques
Authors:
Anastasios Iliopoulos,
John Violos,
Christos Diou,
Iraklis Varlamis
Abstract:
Anomaly Detection in multivariate time series is a major problem in many fields. Due to their nature, anomalies sparsely occur in real data, thus making the task of anomaly detection a challenging problem for classification algorithms to solve. Methods that are based on Deep Neural Networks such as LSTM, Autoencoders, Convolutional Autoencoders etc., have shown positive results in such imbalanced…
▽ More
Anomaly Detection in multivariate time series is a major problem in many fields. Due to their nature, anomalies sparsely occur in real data, thus making the task of anomaly detection a challenging problem for classification algorithms to solve. Methods that are based on Deep Neural Networks such as LSTM, Autoencoders, Convolutional Autoencoders etc., have shown positive results in such imbalanced data. However, the major challenge that algorithms face when applied to multivariate time series is that the anomaly can arise from a small subset of the feature set. To boost the performance of these base models, we propose a feature-bagging technique that considers only a subset of features at a time, and we further apply a transformation that is based on nested rotation computed from Principal Component Analysis (PCA) to improve the effectiveness and generalization of the approach. To further enhance the prediction performance, we propose an ensemble technique that combines multiple base models toward the final decision. In addition, a semi-supervised approach using a Logistic Regressor to combine the base models' outputs is proposed. The proposed methodology is applied to the Skoltech Anomaly Benchmark (SKAB) dataset, which contains time series data related to the flow of water in a closed circuit, and the experimental results show that the proposed ensemble technique outperforms the basic algorithms. More specifically, the performance improvement in terms of anomaly detection accuracy reaches 2% for the unsupervised and at least 10% for the semi-supervised models.
△ Less
Submitted 6 August, 2023;
originally announced August 2023.
-
Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Caching
Authors:
Tim Kaler,
Alexandros-Stavros Iliopoulos,
Philip Murzynowski,
Tao B. Schardl,
Charles E. Leiserson,
Jie Chen
Abstract:
Training and inference with graph neural networks (GNNs) on massive graphs has been actively studied since the inception of GNNs, owing to the widespread use and success of GNNs in applications such as recommendation systems and financial forensics. This paper is concerned with minibatch training and inference with GNNs that employ node-wise sampling in distributed settings, where the necessary pa…
▽ More
Training and inference with graph neural networks (GNNs) on massive graphs has been actively studied since the inception of GNNs, owing to the widespread use and success of GNNs in applications such as recommendation systems and financial forensics. This paper is concerned with minibatch training and inference with GNNs that employ node-wise sampling in distributed settings, where the necessary partitioning of vertex features across distributed storage causes feature communication to become a major bottleneck that hampers scalability. To significantly reduce the communication volume without compromising prediction accuracy, we propose a policy for caching data associated with frequently accessed vertices in remote partitions. The proposed policy is based on an analysis of vertex-wise inclusion probabilities (VIP) during multi-hop neighborhood sampling, which may expand the neighborhood far beyond the partition boundaries of the graph. VIP analysis not only enables the elimination of the communication bottleneck, but it also offers a means to organize in-memory data by prioritizing GPU storage for the most frequently accessed vertex features. We present SALIENT++, which extends the prior state-of-the-art SALIENT system to work with partitioned feature data and leverages the VIP-driven caching policy. SALIENT++ retains the local training efficiency and scalability of SALIENT by using a deep pipeline and drastically reducing communication volume while consuming only a fraction of the storage required by SALIENT. We provide experimental results with the Open Graph Benchmark data sets and demonstrate that training a 3-layer GraphSAGE model with SALIENT++ on 8 single-GPU machines is 7.1 faster than with SALIENT on 1 single-GPU machine, and 12.7 faster than with DistDGL on 8 single-GPU machines.
△ Less
Submitted 4 May, 2023;
originally announced May 2023.
-
Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining
Authors:
Tim Kaler,
Nickolas Stathas,
Anne Ouyang,
Alexandros-Stavros Iliopoulos,
Tao B. Schardl,
Charles E. Leiserson,
Jie Chen
Abstract:
Improving the training and inference performance of graph neural networks (GNNs) is faced with a challenge uncommon in general neural networks: creating mini-batches requires a lot of computation and data movement due to the exponential growth of multi-hop graph neighborhoods along network layers. Such a unique challenge gives rise to a diverse set of system design choices. We argue in favor of pe…
▽ More
Improving the training and inference performance of graph neural networks (GNNs) is faced with a challenge uncommon in general neural networks: creating mini-batches requires a lot of computation and data movement due to the exponential growth of multi-hop graph neighborhoods along network layers. Such a unique challenge gives rise to a diverse set of system design choices. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment, under which we identify major performance bottlenecks hitherto under-explored by developers: mini-batch preparation and transfer. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler, a shared-memory parallelization strategy, and the pipelining of batch transfer with GPU computation. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised. Such an observation unifies training and inference, simplifying model implementation. We report comprehensive experimental results with several benchmark data sets and GNN architectures, including a demonstration that, for the ogbn-papers100M data set, our system SALIENT achieves a speedup of 3x over a standard PyTorch-Geometric implementation with a single GPU and a further 8x parallel speedup with 16 GPUs. Therein, training a 3-layer GraphSAGE model with sampling fanout (15, 10, 5) takes 2.0 seconds per epoch and inference with fanout (20, 20, 20) takes 2.4 seconds, attaining test accuracy 64.58%.
△ Less
Submitted 16 March, 2022; v1 submitted 15 October, 2021;
originally announced October 2021.
-
Projection-tree reduced order modeling for fast N-body computations
Authors:
Steven N. Rodriguez,
Athanasios P. Iliopoulos,
Kevin T. Carlberg,
Steven L. Brunton,
John C. Steuben,
John G. Michopoulos
Abstract:
This work presents a data-driven reduced-order modeling framework to accelerate the computations of $N$-body dynamical systems and their pair-wise interactions. The proposed framework differs from traditional acceleration methods, like the Barnes-Hut method, which requires online tree building of the state space, or the fast-multipole method, which requires rigorous $a$ $priori$ analysis of govern…
▽ More
This work presents a data-driven reduced-order modeling framework to accelerate the computations of $N$-body dynamical systems and their pair-wise interactions. The proposed framework differs from traditional acceleration methods, like the Barnes-Hut method, which requires online tree building of the state space, or the fast-multipole method, which requires rigorous $a$ $priori$ analysis of governing kernels and online tree building. Our approach combines Barnes-Hut hierarchical decomposition, dimensional compression via the least-squares Petrov-Galerkin (LSPG) projection, and hyper-reduction by way of the Gauss-Newton with approximated tensor (GNAT) approach. The resulting $projection-tree$ reduced order model (PTROM) enables a drastic reduction in operational count complexity by constructing sparse hyper-reduced pairwise interactions of the $N$-body dynamical system. As a result, the presented framework is capable of achieving an operational count complexity that is independent of $N$, the number of bodies in the numerical domain. Capabilities of the PTROM method are demonstrated on the two-dimensional fluid-dynamic Biot-Savart kernel within a parametric and reproductive setting. Results show the PTROM is capable of achieving over 2000$\times$ wall-time speed-up with respect to the full-order model, where the speed-up increases with $N$. The resulting solution delivers quantities of interest with errors that are less than 0.1$\%$ with respect to full-order model.
△ Less
Submitted 17 May, 2021; v1 submitted 2 March, 2021;
originally announced March 2021.
-
Challenges in biomarker discovery and biorepository for Gulf-war-disease studies: a novel data platform solution
Authors:
Dimitris Floros,
Mulugu V. Brahmajothi,
Alexandros-Stavros Iliopoulos,
Nikos Pitsianis,
Xiaobai Sun
Abstract:
Aims: Our Gulf War Illness (GWI) study conducts combinatorial screening of many interactive neural and humoral biomarkers in order to establish predictive, diagnostic, and therapeutic targets. We encounter obstacles at every stage of the biomarker discovery process, from sample acquisition, bio-marker extraction to multi-aspect, multi-way interaction analysis, due to the study complexity and lack…
▽ More
Aims: Our Gulf War Illness (GWI) study conducts combinatorial screening of many interactive neural and humoral biomarkers in order to establish predictive, diagnostic, and therapeutic targets. We encounter obstacles at every stage of the biomarker discovery process, from sample acquisition, bio-marker extraction to multi-aspect, multi-way interaction analysis, due to the study complexity and lack of support for complex data problem solutions. We introduce a novel data platform, named ROSALIND, to overcome the challenges, foster healthy and vital collaborations and advance scientific inquiries.
Main methods: ROSALIND is a researcher-centered, study-specific data platform. It provides vital support of individual creativity and effort in collaborative research. We follow the principles etched in the platform name - ROSALIND stands for resource organisms with self-governed accessibility, linkability, integrability, neutrality, and dependability. We translate, encode and implement the principles in the platform with novel use of advanced concepts and techniques to ensure and protect data integrity and research integrity. From a researcher's vantage point, ROSALIND embodies nuance utilities and advanced functionalities in one system, beyond conventional storage, archive and data management.
Key findings: The deployment of ROSALIND in our GWI study in recent 12 months has accelerated the pace of data experiment and analysis, removed numerous error sources, and increased research quality and productivity.
Significance: ROSALIND seems the first to address data integrity and research integrity in tandem with digital measures and means. It also promises a new type of distributed research networks with individualized data platforms connected in various self-organized collaboration configurations.
△ Less
Submitted 17 February, 2021; v1 submitted 4 February, 2021;
originally announced February 2021.
-
iPhantom: a framework for automated creation of individualized computational phantoms and its application to CT organ dosimetry
Authors:
Wanyi Fu,
Shobhit Sharma,
Ehsan Abadi,
Alexandros-Stavros Iliopoulos,
Qi Wang,
Joseph Y. Lo,
Xiaobai Sun,
William P. Segars,
Ehsan Samei
Abstract:
Objective: This study aims to develop and validate a novel framework, iPhantom, for automated creation of patient-specific phantoms or digital-twins (DT) using patient medical images. The framework is applied to assess radiation dose to radiosensitive organs in CT imaging of individual patients. Method: From patient CT images, iPhantom segments selected anchor organs (e.g. liver, bones, pancreas)…
▽ More
Objective: This study aims to develop and validate a novel framework, iPhantom, for automated creation of patient-specific phantoms or digital-twins (DT) using patient medical images. The framework is applied to assess radiation dose to radiosensitive organs in CT imaging of individual patients. Method: From patient CT images, iPhantom segments selected anchor organs (e.g. liver, bones, pancreas) using a learning-based model developed for multi-organ CT segmentation. Organs challenging to segment (e.g. intestines) are incorporated from a matched phantom template, using a diffeomorphic registration model developed for multi-organ phantom-voxels. The resulting full-patient phantoms are used to assess organ doses during routine CT exams. Result: iPhantom was validated on both the XCAT (n=50) and an independent clinical (n=10) dataset with similar accuracy. iPhantom precisely predicted all organ locations with good accuracy of Dice Similarity Coefficients (DSC) >0.6 for anchor organs and DSC of 0.3-0.9 for all other organs. iPhantom showed less than 10% dose errors for the majority of organs, which was notably superior to the state-of-the-art baseline method (20-35% dose errors). Conclusion: iPhantom enables automated and accurate creation of patient-specific phantoms and, for the first time, provides sufficient and automated patient-specific dose estimates for CT dosimetry. Significance: The new framework brings the creation and application of CHPs to the level of individual CHPs through automation, achieving a wider and precise organ localization, paving the way for clinical monitoring, and personalized optimization, and large-scale research.
△ Less
Submitted 19 August, 2020;
originally announced August 2020.
-
Spaceland Embedding of Sparse Stochastic Graphs
Authors:
Nikos Pitsianis,
Alexandros-Stavros Iliopoulos,
Dimitris Floros,
Xiaobai Sun
Abstract:
We introduce a nonlinear method for directly embedding large, sparse, stochastic graphs into low-dimensional spaces, without requiring vertex features to reside in, or be transformed into, a metric space. Graph data and models are prevalent in real-world applications. Direct graph embedding is fundamental to many graph analysis tasks, in addition to graph visualization. We name the novel approach…
▽ More
We introduce a nonlinear method for directly embedding large, sparse, stochastic graphs into low-dimensional spaces, without requiring vertex features to reside in, or be transformed into, a metric space. Graph data and models are prevalent in real-world applications. Direct graph embedding is fundamental to many graph analysis tasks, in addition to graph visualization. We name the novel approach SG-t-SNE, as it is inspired by and builds upon the core principle of t-SNE, a widely used method for nonlinear dimensionality reduction and data visualization. We also introduce t-SNE-$Π$, a high-performance software for 2D, 3D embedding of large sparse graphs on personal computers with superior efficiency. It empowers SG-t-SNE with modern computing techniques for exploiting in tandem both matrix structures and memory architectures. We present elucidating embedding results on one synthetic graph and four real-world networks.
△ Less
Submitted 13 June, 2019;
originally announced June 2019.
-
Rapid Near-Neighbor Interaction of High-dimensional Data via Hierarchical Clustering
Authors:
Nikos Pitsianis,
Dimitris Floros,
Alexandros-Stavros Iliopoulos,
Kostas Mylonakis,
Nikos Sismanis,
Xiaobai Sun
Abstract:
Calculation of near-neighbor interactions among high dimensional, irregularly distributed data points is a fundamental task to many graph-based or kernel-based machine learning algorithms and applications. Such calculations, involving large, sparse interaction matrices, expose the limitation of conventional data-and-computation reordering techniques for improving space and time locality on modern…
▽ More
Calculation of near-neighbor interactions among high dimensional, irregularly distributed data points is a fundamental task to many graph-based or kernel-based machine learning algorithms and applications. Such calculations, involving large, sparse interaction matrices, expose the limitation of conventional data-and-computation reordering techniques for improving space and time locality on modern computer memory hierarchies. We introduce a novel method for obtaining a matrix permutation that renders a desirable sparsity profile. The method is distinguished by the guiding principle to obtain a profile that is block-sparse with dense blocks. Our profile model and measure capture the essential properties affecting space and time locality, and permit variation in sparsity profile without imposing a restriction to a fixed pattern. The second distinction lies in an efficient algorithm for obtaining a desirable profile, via exploring and exploiting multi-scale cluster structure hidden in but intrinsic to the data. The algorithm accomplishes its task with key components for lower-dimensional embedding with data-specific principal feature axes, hierarchical data clustering, multi-level matrix compression storage, and multi-level interaction computations. We provide experimental results from case studies with two important data analysis algorithms. The resulting performance is remarkably comparable to the BLAS performance for the best-case interaction governed by a regularly banded matrix with the same sparsity.
△ Less
Submitted 11 September, 2017;
originally announced September 2017.
-
Iterative Inversion of Deformation Vector Fields with Feedback Control
Authors:
Abhishek Kumar Dubey,
Alexandros-Stavros Iliopoulos,
Xiaobai Sun,
Fang-Fang Yin,
Lei Ren
Abstract:
Purpose: Often, the inverse deformation vector field (DVF) is needed together with the corresponding forward DVF in 4D reconstruction and dose calculation, adaptive radiation therapy, and simultaneous deformable registration. This study aims at improving both accuracy and efficiency of iterative algorithms for DVF inversion, and advancing our understanding of divergence and latency conditions. Met…
▽ More
Purpose: Often, the inverse deformation vector field (DVF) is needed together with the corresponding forward DVF in 4D reconstruction and dose calculation, adaptive radiation therapy, and simultaneous deformable registration. This study aims at improving both accuracy and efficiency of iterative algorithms for DVF inversion, and advancing our understanding of divergence and latency conditions. Method: We introduce a framework of fixed-point iteration algorithms with active feedback control for DVF inversion. Based on rigorous convergence analysis, we design control mechanisms for modulating the inverse consistency (IC) residual of the current iterate, to be used as feedback into the next iterate. The control is designed adaptively to the input DVF with the objective to enlarge the convergence area and expedite convergence. Three particular settings of feedback control are introduced: constant value over the domain throughout the iteration; alternating values between iteration steps; and spatially variant values. We also introduce three spectral measures of the displacement Jacobian for characterizing a DVF. These measures reveal the critical role of what we term the non-translational displacement component (NTDC) of the DVF. We carry out inversion experiments with an analytical DVF pair, and with DVFs associated with thoracic CT images of 6 patients at end of expiration and end of inspiration. Results: NTDC-adaptive iterations are shown to attain a larger convergence region at a faster pace compared to previous non-adaptive DVF inversion iteration algorithms. By our numerical experiments, alternating control yields smaller IC residuals and inversion errors than constant control. Spatially variant control renders smaller residuals and errors by at least an order of magnitude, compared to other schemes, in no more than 10 steps. Inversion results also show remarkable quantitative agreement with analysis-based predictions. Conclusion: Our analysis captures properties of DVF data associated with clinical CT images, and provides new understanding of iterative DVF inversion algorithms with a simple residual feedback control. Adaptive control is necessary and highly effective in the presence of non-small NTDCs. The adaptive iterations or the spectral measures, or both, may potentially be incorporated into deformable image registration methods.
△ Less
Submitted 28 March, 2018; v1 submitted 26 October, 2016;
originally announced October 2016.
-
Hyperspectral Image Classification and Clutter Detection via Multiple Structural Embeddings and Dimension Reductions
Authors:
Alexandros-Stavros Iliopoulos,
Tiancheng Liu,
Xiaobai Sun
Abstract:
We present a new and effective approach for Hyperspectral Image (HSI) classification and clutter detection, overcoming a few long-standing challenges presented by HSI data characteristics. Residing in a high-dimensional spectral attribute space, HSI data samples are known to be strongly correlated in their spectral signatures, exhibit nonlinear structure due to several physical laws, and contain u…
▽ More
We present a new and effective approach for Hyperspectral Image (HSI) classification and clutter detection, overcoming a few long-standing challenges presented by HSI data characteristics. Residing in a high-dimensional spectral attribute space, HSI data samples are known to be strongly correlated in their spectral signatures, exhibit nonlinear structure due to several physical laws, and contain uncertainty and noise from multiple sources. In the presented approach, we generate an adaptive, structurally enriched representation environment, and employ the locally linear embedding (LLE) in it. There are two structure layers external to LLE. One is feature space embedding: the HSI data attributes are embedded into a discriminatory feature space where spatio-spectral coherence and distinctive structures are distilled and exploited to mitigate various difficulties encountered in the native hyperspectral attribute space. The other structure layer encloses the ranges of algorithmic parameters for LLE and feature embedding, and supports a multiplexing and integrating scheme for contending with multi-source uncertainty. Experiments on two commonly used HSI datasets with a small number of learning samples have rendered remarkably high-accuracy classification results, as well as distinctive maps of detected clutter regions.
△ Less
Submitted 3 June, 2015;
originally announced June 2015.
