Showing 1–50 of 72 results for author: Huerta, A

Noise-robust multi-fidelity surrogate modelling for parametric partial differential equations

Authors: Benjamin M. Kent, Lorenzo Tamellini, Matteo Giacomini, Antonio Huerta

Abstract: We address the challenge of constructing noise-robust surrogate models for quantities of interest (QoIs) arising from parametric partial differential equations (PDEs), using multi-fidelity collocation techniques; specifically, the Multi-Index Stochastic Collocation (MISC). In practical scenarios, the PDE evaluations used to build a response surface are often corrupted by numerical noise, especiall… ▽ More We address the challenge of constructing noise-robust surrogate models for quantities of interest (QoIs) arising from parametric partial differential equations (PDEs), using multi-fidelity collocation techniques; specifically, the Multi-Index Stochastic Collocation (MISC). In practical scenarios, the PDE evaluations used to build a response surface are often corrupted by numerical noise, especially for the low-fidelity models. This noise, which may originate from loose solver tolerances, coarse discretisations, or transient effects, can lead to overfitting in MISC, degrading surrogate quality through nonphysical oscillations and loss of convergence, thereby limiting its utility in downstream tasks like uncertainty quantification, optimisation, and control. To correct this behaviour, we propose an improved version of MISC that can automatically detect the presence of solver noise during the surrogate model construction and then ignore the exhausted fidelities. Our approach monitors the spectral decay of the surrogate at each iteration, identifying stagnation in the coefficient spectrum that signals the onset of noise. Once detected, the algorithm selectively halts the use of noisy fidelities, focusing computational resources on those fidelities that still provide meaningful information. The effectiveness of this approach is numerically validated on two challenging test cases: a parabolic advection--diffusion PDE with uncertain coefficients, and a parametric turbulent incompressible Navier--Stokes problem. The results showcase the accuracy and robustness of the resulting multi-fidelity surrogate and its capability to extract relevant information, even from under-resolved meshes not suitable for reliable single-fidelity computations. △ Less

Submitted 4 July, 2025; originally announced July 2025.

Comments: 35 pages, 31 figures

Resolving Turbulent Magnetohydrodynamics: A Hybrid Operator-Diffusion Framework

Authors: Semih Kacmaz, E. A. Huerta, Roland Haas

Abstract: We present a hybrid machine learning framework that combines Physics-Informed Neural Operators (PINOs) with score-based generative diffusion models to simulate the full spatio-temporal evolution of two-dimensional, incompressible, resistive magnetohydrodynamic (MHD) turbulence across a broad range of Reynolds numbers ($\mathrm{Re}$). The framework leverages the equation-constrained generalization… ▽ More We present a hybrid machine learning framework that combines Physics-Informed Neural Operators (PINOs) with score-based generative diffusion models to simulate the full spatio-temporal evolution of two-dimensional, incompressible, resistive magnetohydrodynamic (MHD) turbulence across a broad range of Reynolds numbers ($\mathrm{Re}$). The framework leverages the equation-constrained generalization capabilities of PINOs to predict coherent, low-frequency dynamics, while a conditional diffusion model stochastically corrects high-frequency residuals, enabling accurate modeling of fully developed turbulence. Trained on a comprehensive ensemble of high-fidelity simulations with $\mathrm{Re} \in \{100, 250, 500, 750, 1000, 3000, 10000\}$, the approach achieves state-of-the-art accuracy in regimes previously inaccessible to deterministic surrogates. At $\mathrm{Re}=1000$ and $3000$, the model faithfully reconstructs the full spectral energy distributions of both velocity and magnetic fields late into the simulation, capturing non-Gaussian statistics, intermittent structures, and cross-field correlations with high fidelity. At extreme turbulence levels ($\mathrm{Re}=10000$), it remains the first surrogate capable of recovering the high-wavenumber evolution of the magnetic field, preserving large-scale morphology and enabling statistically meaningful predictions. △ Less

Submitted 2 July, 2025; originally announced July 2025.

ACM Class: J.2; I.2

Characteristic boundary conditions for Hybridizable Discontinuous Galerkin methods

Authors: Jan Ellmenreich, Matteo Giacomini, Antonio Huerta, Philip L. Lederer

Abstract: In this work we introduce the concept of characteristic boundary conditions (CBCs) within the framework of Hybridizable Discontinuous Galerkin (HDG) methods, including both the Navier-Stokes characteristic boundary conditions (NSCBCs) and a novel approach to generalized characteristic relaxation boundary conditions (GRCBCs). CBCs are based on the characteristic decomposition of the compressible Eu… ▽ More In this work we introduce the concept of characteristic boundary conditions (CBCs) within the framework of Hybridizable Discontinuous Galerkin (HDG) methods, including both the Navier-Stokes characteristic boundary conditions (NSCBCs) and a novel approach to generalized characteristic relaxation boundary conditions (GRCBCs). CBCs are based on the characteristic decomposition of the compressible Euler equations and are designed to prevent the reflection of waves at the domain boundaries. We show the effectiveness of the proposed method for weakly compressible flows through a series of numerical experiments by comparing the results with common boundary conditions in the HDG setting and reference solutions available in the literature. In particular, HDG with CBCs show superior performance minimizing the reflection of vortices at artificial boundaries, for both inviscid and viscous flows. △ Less

Submitted 25 March, 2025; originally announced March 2025.

MOFA: Discovering Materials for Carbon Capture with a GenAI- and Simulation-Based Workflow

Authors: Xiaoli Yan, Nathaniel Hudson, Hyun Park, Daniel Grzenda, J. Gregory Pauloski, Marcus Schwarting, Haochen Pan, Hassan Harb, Samuel Foreman, Chris Knight, Tom Gibbs, Kyle Chard, Santanu Chaudhuri, Emad Tajkhorshid, Ian Foster, Mohamad Moosavi, Logan Ward, E. A. Huerta

Abstract: We present MOFA, an open-source generative AI (GenAI) plus simulation workflow for high-throughput generation of metal-organic frameworks (MOFs) on large-scale high-performance computing (HPC) systems. MOFA addresses key challenges in integrating GPU-accelerated computing for GPU-intensive GenAI tasks, including distributed training and inference, alongside CPU- and GPU-optimized tasks for screeni… ▽ More We present MOFA, an open-source generative AI (GenAI) plus simulation workflow for high-throughput generation of metal-organic frameworks (MOFs) on large-scale high-performance computing (HPC) systems. MOFA addresses key challenges in integrating GPU-accelerated computing for GPU-intensive GenAI tasks, including distributed training and inference, alongside CPU- and GPU-optimized tasks for screening and filtering AI-generated MOFs using molecular dynamics, density functional theory, and Monte Carlo simulations. These heterogeneous tasks are unified within an online learning framework that optimizes the utilization of available CPU and GPU resources across HPC systems. Performance metrics from a 450-node (14,400 AMD Zen 3 CPUs + 1800 NVIDIA A100 GPUs) supercomputer run demonstrate that MOFA achieves high-throughput generation of novel MOF structures, with CO$_2$ adsorption capacities ranking among the top 10 in the hypothetical MOF (hMOF) dataset. Furthermore, the production of high-quality MOFs exhibits a linear relationship with the number of nodes utilized. The modular architecture of MOFA will facilitate its integration into other scientific applications that dynamically combine GenAI with large-scale simulations. △ Less

Submitted 17 January, 2025; originally announced January 2025.

Comments: 13 pages, 10 figures

A hybrid pressure formulation of the face-centred finite volume method for viscous laminar incompressible flows

Authors: Matteo Giacomini, Davide Cortellessa, Luan M. Vieira, Ruben Sevilla, Antonio Huerta

Abstract: This work presents a hybrid pressure face-centred finite volume (FCFV) solver to simulate steady-state incompressible Navier-Stokes flows. The method leverages the robustness, in the incompressible limit, of the hybridisable discontinuous Galerkin paradigm for compressible and weakly compressible flows to derive the formulation of a novel, low-order face-based discretisation. The incompressibility… ▽ More This work presents a hybrid pressure face-centred finite volume (FCFV) solver to simulate steady-state incompressible Navier-Stokes flows. The method leverages the robustness, in the incompressible limit, of the hybridisable discontinuous Galerkin paradigm for compressible and weakly compressible flows to derive the formulation of a novel, low-order face-based discretisation. The incompressibility constraint is enforced in a weak sense, by introducing an inter-cell mass flux defined in terms of a new, hybrid variable, representing the pressure at the cell faces. This results in a new hybridisation strategy where cell variables (velocity, pressure and deviatoric strain rate tensor) are expressed as a function of velocity and pressure at the barycentre of the cell faces. The hybrid pressure formulation provides first-order convergence of all variables, including the stress, without the need for gradient reconstruction, thus being less sensitive to cell type, stretching, distortion, and skewness than traditional low-order finite volume solvers. Numerical benchmarks of Navier-Stokes flows at low and moderate Reynolds numbers, in two and three dimensions, are presented to evaluate accuracy and robustness of the method. In particular, the hybrid pressure formulation outperforms the FCFV method when convective effects are relevant, achieving accurate predictions on significantly coarser meshes. △ Less

Submitted 1 April, 2025; v1 submitted 8 January, 2025; originally announced January 2025.

Comments: 47 pages, 18 figures, 7 tables

MSC Class: 76M12; 65N30; 65N12; 76D05; 76D07

An OpenFOAM face-centred solver for incompressible flows robust to mesh distortion

Authors: Davide Cortellessa, Matteo Giacomini, Antonio Huerta

Abstract: This work presents an overview of mesh-induced errors commonly experienced by cell-centred finite volumes (CCFV), for which the face-centred finite volume (FCFV) paradigm offers competitive solutions. In particular, a robust FCFV solver for incompressible laminar flows is integrated in OpenFOAM and tested on a set of steady-state and transient benchmarks. The method outperforms standard simpleFoam… ▽ More This work presents an overview of mesh-induced errors commonly experienced by cell-centred finite volumes (CCFV), for which the face-centred finite volume (FCFV) paradigm offers competitive solutions. In particular, a robust FCFV solver for incompressible laminar flows is integrated in OpenFOAM and tested on a set of steady-state and transient benchmarks. The method outperforms standard simpleFoam and pimpleFoam algorithms in terms of optimal convergence, accuracy, stability, and robustness. Special attention is devoted to motivate and numerically demonstrate the ability of the FCFV method to treat non-orthogonal, stretched, and skewed meshes, where CCFV schemes exhibit shortcomings. △ Less

Submitted 8 January, 2025; v1 submitted 31 December, 2024; originally announced January 2025.

Comments: 18 pages, 12 figures, 2 tables

MSC Class: 76M12; 76D05; 65M12

Machine learning-driven conservative-to-primitive conversion in hybrid piecewise polytropic and tabulated equations of state

Authors: Semih Kacmaz, Roland Haas, E. A. Huerta

Abstract: We present a novel machine learning (ML) method to accelerate conservative-to-primitive inversion, focusing on hybrid piecewise polytropic and tabulated equations of state. Traditional root-finding techniques are computationally expensive, particularly for large-scale relativistic hydrodynamics simulations. To address this, we employ feedforward neural networks (NNC2PS and NNC2PL), trained in PyTo… ▽ More We present a novel machine learning (ML) method to accelerate conservative-to-primitive inversion, focusing on hybrid piecewise polytropic and tabulated equations of state. Traditional root-finding techniques are computationally expensive, particularly for large-scale relativistic hydrodynamics simulations. To address this, we employ feedforward neural networks (NNC2PS and NNC2PL), trained in PyTorch and optimized for GPU inference using NVIDIA TensorRT, achieving significant speedups with minimal accuracy loss. The NNC2PS model achieves $ L_1 $ and $ L_\infty $ errors of $ 4.54 \times 10^{-7} $ and $ 3.44 \times 10^{-6} $, respectively, while the NNC2PL model exhibits even lower error values. TensorRT optimization with mixed-precision deployment substantially accelerates performance compared to traditional root-finding methods. Specifically, the mixed-precision TensorRT engine for NNC2PS achieves inference speeds approximately 400 times faster than a traditional single-threaded CPU implementation for a dataset size of 1,000,000 points. Ideal parallelization across an entire compute node in the Delta supercomputer (Dual AMD 64 core 2.45 GHz Milan processors; and 8 NVIDIA A100 GPUs with 40 GB HBM2 RAM and NVLink) predicts a 25-fold speedup for TensorRT over an optimally-parallelized numerical method when processing 8 million data points. Moreover, the ML method exhibits sub-linear scaling with increasing dataset sizes. We release the scientific software developed, enabling further validation and extension of our findings. This work underscores the potential of ML, combined with GPU optimization and model quantization, to accelerate conservative-to-primitive inversion in relativistic hydrodynamics simulations. △ Less

Submitted 29 January, 2025; v1 submitted 10 December, 2024; originally announced December 2024.

Comments: 17 pages, 4 figures, 1 table New results added

ACM Class: J.2; I.2

A face-centred finite volume method for laminar and turbulent incompressible flows

Authors: Luan M. Vieira, Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: This work develops, for the first time, a face-centred finite volume (FCFV) solver for the simulation of laminar and turbulent viscous incompressible flows. The formulation relies on the Reynolds-averaged Navier-Stokes (RANS) equations coupled with the negative Spalart-Allmaras (SA) model and three novel convective stabilisations, inspired by Riemann solvers, are derived and compared numerically.… ▽ More This work develops, for the first time, a face-centred finite volume (FCFV) solver for the simulation of laminar and turbulent viscous incompressible flows. The formulation relies on the Reynolds-averaged Navier-Stokes (RANS) equations coupled with the negative Spalart-Allmaras (SA) model and three novel convective stabilisations, inspired by Riemann solvers, are derived and compared numerically. The resulting method achieves first-order convergence of the velocity, the velocity-gradient tensor and the pressure. FCFV accurately predicts engineering quantities of interest, such as drag and lift, on unstructured meshes and, by avoiding gradient reconstruction, the method is less sensitive to mesh quality than other FV methods, even in the presence of highly distorted and stretched cells. A monolithic and a staggered solution strategies for the RANS-SA system are derived and compared numerically. Numerical benchmarks, involving laminar and turbulent, steady and transient cases are used to assess the performance, accuracy and robustness of the proposed FCFV method. △ Less

Submitted 11 June, 2024; v1 submitted 3 March, 2024; originally announced March 2024.

Comments: 42 pages, 25 figures, 4 tables

MSC Class: 76M12; 76D05; 65M12; 76F99

Journal ref: Computers and Fluids, 279:106339, 2024

Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources -- A Case Study on Federated Fine-tuning of LLaMA 2

Authors: Zilinghan Li, Shilan He, Pranshu Chaturvedi, Volodymyr Kindratenko, Eliu A Huerta, Kibaek Kim, Ravi Madduri

Abstract: Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. In this paper, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learn… ▽ More Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. In this paper, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning a LLaMA 2 7B model using several cloud resources and supercomputers. △ Less

Submitted 19 February, 2024; originally announced February 2024.

An unfitted high-order HDG method for two-fluid Stokes flow with exact NURBS geometries

Authors: Stefano Piccardo, Matteo Giacomini, Antonio Huerta

Abstract: A high-order, degree-adaptive hybridizable discontinuous Galerkin (HDG) method is presented for two-fluid incompressible Stokes flows, with boundaries and interfaces described using NURBS. The NURBS curves are embedded in a fixed Cartesian grid, yielding an unfitted HDG scheme capable of treating the exact geometry of the boundaries/interfaces, circumventing the need for fitted, high-order, curved… ▽ More A high-order, degree-adaptive hybridizable discontinuous Galerkin (HDG) method is presented for two-fluid incompressible Stokes flows, with boundaries and interfaces described using NURBS. The NURBS curves are embedded in a fixed Cartesian grid, yielding an unfitted HDG scheme capable of treating the exact geometry of the boundaries/interfaces, circumventing the need for fitted, high-order, curved meshes. The framework of the NURBS-enhanced finite element method (NEFEM) is employed for accurate quadrature along immersed NURBS and in elements cut by NURBS curves. A Nitsche's formulation is used to enforce Dirichlet conditions on embedded surfaces, yielding unknowns only on the mesh skeleton as in standard HDG, without introducing any additional degree of freedom on non-matching boundaries/interfaces. The resulting unfitted HDG-NEFEM method combines non-conforming meshes, exact NURBS geometry and high-order approximations to provide high-fidelity results on coarse meshes, independent of the geometric features of the domain. Numerical examples illustrate the optimal accuracy and robustness of the method, even in the presence of badly cut cells or faces, and its suitability to simulate microfluidic systems from CAD geometries. △ Less

Submitted 23 May, 2024; v1 submitted 21 December, 2023; originally announced December 2023.

Comments: 47 pages, 23 figures, 4 tables

MSC Class: 65M60; 76D07; 76M10

Journal ref: Journal of Computational Physics, 512, 113143 (2024)

Enabling End-to-End Secure Federated Learning in Biomedical Research on Heterogeneous Computing Environments with APPFLx

Authors: Trung-Hieu Hoang, Jordan Fuhrman, Ravi Madduri, Miao Li, Pranshu Chaturvedi, Zilinghan Li, Kibaek Kim, Minseok Ryu, Ryan Chard, E. A. Huerta, Maryellen Giger

Abstract: Facilitating large-scale, cross-institutional collaboration in biomedical machine learning projects requires a trustworthy and resilient federated learning (FL) environment to ensure that sensitive information such as protected health information is kept confidential. In this work, we introduce APPFLx, a low-code FL framework that enables the easy setup, configuration, and running of FL experiment… ▽ More Facilitating large-scale, cross-institutional collaboration in biomedical machine learning projects requires a trustworthy and resilient federated learning (FL) environment to ensure that sensitive information such as protected health information is kept confidential. In this work, we introduce APPFLx, a low-code FL framework that enables the easy setup, configuration, and running of FL experiments across organizational and administrative boundaries while providing secure end-to-end communication, privacy-preserving functionality, and identity management. APPFLx is completely agnostic to the underlying computational infrastructure of participating clients. We demonstrate the capability of APPFLx as an easy-to-use framework for accelerating biomedical studies across institutions and healthcare systems while maintaining the protection of private medical data in two case studies: (1) predicting participant age from electrocardiogram (ECG) waveforms, and (2) detecting COVID-19 disease from chest radiographs. These experiments were performed securely across heterogeneous compute resources, including a mixture of on-premise high-performance computing and cloud computing, and highlight the role of federated learning in improving model generalizability and performance when aggregating data from multiple healthcare systems. Finally, we demonstrate that APPFLx serves as a convenient and easy-to-use framework for accelerating biomedical studies across institutions and healthcare system while maintaining the protection of private medical data. △ Less

Submitted 14 December, 2023; originally announced December 2023.

AI ensemble for signal detection of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

Authors: Minyang Tian, E. A. Huerta, Huihuo Zheng

Abstract: We introduce spatiotemporal-graph models that concurrently process data from the twin advanced LIGO detectors and the advanced Virgo detector. We trained these AI classifiers with 2.4 million IMRPhenomXPHM waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers with component masses $m_{\{1,2\}}\in[3M_\odot, 50 M_\odot]$, and individual spins… ▽ More We introduce spatiotemporal-graph models that concurrently process data from the twin advanced LIGO detectors and the advanced Virgo detector. We trained these AI classifiers with 2.4 million IMRPhenomXPHM waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers with component masses $m_{\{1,2\}}\in[3M_\odot, 50 M_\odot]$, and individual spins $s^z_{\{1,2\}}\in[-0.9, 0.9]$; and which include the $(\ell, |m|) = \{(2, 2), (2, 1), (3, 3), (3, 2), (4, 4)\}$ modes, and mode mixing effects in the $\ell = 3, |m| = 2$ harmonics. We trained these AI classifiers within 22 hours using distributed training over 96 NVIDIA V100 GPUs in the Summit supercomputer. We then used transfer learning to create AI predictors that estimate the total mass of potential binary black holes identified by all AI classifiers in the ensemble. We used this ensemble, 3 classifiers for signal detection and 2 total mass predictors, to process a year-long test set in which we injected 300,000 signals. This year-long test set was processed within 5.19 minutes using 1024 NVIDIA A100 GPUs in the Polaris supercomputer (for AI inference) and 128 CPU nodes in the ThetaKNL supercomputer (for post-processing of noise triggers), housed at the Argonne Leadership Computing Facility. These studies indicate that our AI ensemble provides state-of-the-art signal detection accuracy, and reports 2 misclassifications for every year of searched data. This is the first AI ensemble designed to search for and find higher order gravitational wave mode signals. △ Less

Submitted 4 December, 2023; v1 submitted 29 September, 2023; originally announced October 2023.

Comments: 4 pages, 2 figures, 1 table; v2: 5 pages, 2 figures, 1 table, accepted to NeurIPS 2023 workshop on Machine Learning and the Physical Sciences

MSC Class: 68T01; 68T35; 83C35; 83C57

FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler

Authors: Zilinghan Li, Pranshu Chaturvedi, Shilan He, Han Chen, Gagandeep Singh, Volodymyr Kindratenko, E. A. Huerta, Kibaek Kim, Ravi Madduri

Abstract: Cross-silo federated learning offers a promising solution to collaboratively train robust and generalized AI models without compromising the privacy of local datasets, e.g., healthcare, financial, as well as scientific projects that lack a centralized data facility. Nonetheless, because of the disparity of computing resources among different clients (i.e., device heterogeneity), synchronous federa… ▽ More Cross-silo federated learning offers a promising solution to collaboratively train robust and generalized AI models without compromising the privacy of local datasets, e.g., healthcare, financial, as well as scientific projects that lack a centralized data facility. Nonetheless, because of the disparity of computing resources among different clients (i.e., device heterogeneity), synchronous federated learning algorithms suffer from degraded efficiency when waiting for straggler clients. Similarly, asynchronous federated learning algorithms experience degradation in the convergence rate and final model accuracy on non-identically and independently distributed (non-IID) heterogeneous datasets due to stale local models and client drift. To address these limitations in cross-silo federated learning with heterogeneous clients and data, we propose FedCompass, an innovative semi-asynchronous federated learning algorithm with a computing power-aware scheduler on the server side, which adaptively assigns varying amounts of training tasks to different clients using the knowledge of the computing power of individual clients. FedCompass ensures that multiple locally trained models from clients are received almost simultaneously as a group for aggregation, effectively reducing the staleness of local models. At the same time, the overall training process remains asynchronous, eliminating prolonged waiting periods from straggler clients. Using diverse non-IID heterogeneous distributed datasets, we demonstrate that FedCompass achieves faster convergence and higher accuracy than other asynchronous algorithms while remaining more efficient than synchronous algorithms when performing federated learning on heterogeneous clients. The source code for FedCompass is available at https://github.com/APPFL/FedCompass. △ Less

Submitted 11 March, 2024; v1 submitted 26 September, 2023; originally announced September 2023.

Comments: Accepted as poster at The Twelfth International Conference on Learning Representations (ICLR 2024)

APPFLx: Providing Privacy-Preserving Cross-Silo Federated Learning as a Service

Authors: Zilinghan Li, Shilan He, Pranshu Chaturvedi, Trung-Hieu Hoang, Minseok Ryu, E. A. Huerta, Volodymyr Kindratenko, Jordan Fuhrman, Maryellen Giger, Ryan Chard, Kibaek Kim, Ravi Madduri

Abstract: Cross-silo privacy-preserving federated learning (PPFL) is a powerful tool to collaboratively train robust and generalized machine learning (ML) models without sharing sensitive (e.g., healthcare of financial) local data. To ease and accelerate the adoption of PPFL, we introduce APPFLx, a ready-to-use platform that provides privacy-preserving cross-silo federated learning as a service. APPFLx empl… ▽ More Cross-silo privacy-preserving federated learning (PPFL) is a powerful tool to collaboratively train robust and generalized machine learning (ML) models without sharing sensitive (e.g., healthcare of financial) local data. To ease and accelerate the adoption of PPFL, we introduce APPFLx, a ready-to-use platform that provides privacy-preserving cross-silo federated learning as a service. APPFLx employs Globus authentication to allow users to easily and securely invite trustworthy collaborators for PPFL, implements several synchronous and asynchronous FL algorithms, streamlines the FL experiment launch process, and enables tracking and visualizing the life cycle of FL experiments, allowing domain experts and ML practitioners to easily orchestrate and evaluate cross-silo FL under one platform. APPFLx is available online at https://appflx.link △ Less

Submitted 17 August, 2023; originally announced August 2023.

APACE: AlphaFold2 and advanced computing as a service for accelerated discovery in biophysics

Authors: Hyun Park, Parth Patel, Roland Haas, E. A. Huerta

Abstract: The prediction of protein 3D structure from amino acid sequence is a computational grand challenge in biophysics, and plays a key role in robust protein structure prediction algorithms, from drug discovery to genome interpretation. The advent of AI models, such as AlphaFold, is revolutionizing applications that depend on robust protein structure prediction algorithms. To maximize the impact, and e… ▽ More The prediction of protein 3D structure from amino acid sequence is a computational grand challenge in biophysics, and plays a key role in robust protein structure prediction algorithms, from drug discovery to genome interpretation. The advent of AI models, such as AlphaFold, is revolutionizing applications that depend on robust protein structure prediction algorithms. To maximize the impact, and ease the usability, of these novel AI tools we introduce APACE, AlphaFold2 and advanced computing as a service, a novel computational framework that effectively handles this AI model and its TB-size database to conduct accelerated protein structure prediction analyses in modern supercomputing environments. We deployed APACE in the Delta and Polaris supercomputers, and quantified its performance for accurate protein structure predictions using four exemplar proteins: 6AWO, 6OAN, 7MEZ, and 6D6U. Using up to 300 ensembles, distributed across 200 NVIDIA A100 GPUs, we found that APACE is up to two orders of magnitude faster than off-the-self AlphaFold2 implementations, reducing time-to-solution from weeks to minutes. This computational approach may be readily linked with robotics laboratories to automate and accelerate scientific discovery. △ Less

Submitted 1 July, 2024; v1 submitted 15 August, 2023; originally announced August 2023.

Comments: 7 pages, 4 figures, 2 tables

ACM Class: I.2

Journal ref: Proceedings of the National Academy of Sciences, 121, 27, (2024)

Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers

Authors: Minyang Tian, E. A. Huerta, Huihuo Zheng, Prayush Kumar

Abstract: We present a new class of AI models for the detection of quasi-circular, spinning, non-precessing binary black hole mergers whose waveforms include the higher order gravitational wave modes $(l, |m|)=\{(2, 2), (2, 1), (3, 3), (3, 2), (4, 4)\}$, and mode mixing effects in the $l = 3, |m| = 2$ harmonics. These AI models combine hybrid dilated convolution neural networks to accurately model both shor… ▽ More We present a new class of AI models for the detection of quasi-circular, spinning, non-precessing binary black hole mergers whose waveforms include the higher order gravitational wave modes $(l, |m|)=\{(2, 2), (2, 1), (3, 3), (3, 2), (4, 4)\}$, and mode mixing effects in the $l = 3, |m| = 2$ harmonics. These AI models combine hybrid dilated convolution neural networks to accurately model both short- and long-range temporal sequential information of gravitational waves; and graph neural networks to capture spatial correlations among gravitational wave observatories to consistently describe and identify the presence of a signal in a three detector network encompassing the Advanced LIGO and Virgo detectors. We first trained these spatiotemporal-graph AI models using synthetic noise, using 1.2 million modeled waveforms to densely sample this signal manifold, within 1.7 hours using 256 A100 GPUs in the Polaris supercomputer at the ALCF. Our distributed training approach had optimal performance, and strong scaling up to 512 A100 GPUs. With these AI ensembles we processed data from a three detector network, and found that an ensemble of 4 AI models achieves state-of-the-art performance for signal detection, and reports two misclassifications for every decade of searched data. We distributed AI inference over 128 GPUs in the Polaris supercomputer and 128 nodes in the Theta supercomputer, and completed the processing of a decade of gravitational wave data from a three detector network within 3.5 hours. Finally, we fine-tuned these AI ensembles to process the entire month of February 2020, which is part of the O3b LIGO/Virgo observation run, and found 6 gravitational waves, concurrently identified in Advanced LIGO and Advanced Virgo data, and zero false positives. This analysis was completed in one hour using one A100 GPU. △ Less

Submitted 18 June, 2024; v1 submitted 27 June, 2023; originally announced June 2023.

Comments: 14 pages, 6 figures, and 3 tables

MSC Class: 68T01; 68T35; 83C35; 83C57

Journal ref: Mach. Learn.: Sci. Technol. 5 (2024) 025056

A generative artificial intelligence framework based on a molecular diffusion model for the design of metal-organic frameworks for carbon capture

Authors: Hyun Park, Xiaoli Yan, Ruijie Zhu, E. A. Huerta, Santanu Chaudhuri, Donny Cooper, Ian Foster, Emad Tajkhorshid

Abstract: Metal-organic frameworks (MOFs) exhibit great promise for CO2 capture. However, finding the best performing materials poses computational and experimental grand challenges in view of the vast chemical space of potential building blocks. Here, we introduce GHP-MOFassemble, a generative artificial intelligence (AI), high performance framework for the rational and accelerated design of MOFs with high… ▽ More Metal-organic frameworks (MOFs) exhibit great promise for CO2 capture. However, finding the best performing materials poses computational and experimental grand challenges in view of the vast chemical space of potential building blocks. Here, we introduce GHP-MOFassemble, a generative artificial intelligence (AI), high performance framework for the rational and accelerated design of MOFs with high CO2 adsorption capacity and synthesizable linkers. GHP-MOFassemble generates novel linkers, assembled with one of three pre-selected metal nodes (Cu paddlewheel, Zn paddlewheel, Zn tetramer) into MOFs in a primitive cubic topology. GHP-MOFassemble screens and validates AI-generated MOFs for uniqueness, synthesizability, structural validity, uses molecular dynamics simulations to study their stability and chemical consistency, and crystal graph neural networks and Grand Canonical Monte Carlo simulations to quantify their CO2 adsorption capacities. We present the top six AI-generated MOFs with CO2 capacities greater than 2 $m mol/g$, i.e., higher than 96.9% of structures in the hypothetical MOF dataset. △ Less

Submitted 12 March, 2024; v1 submitted 14 June, 2023; originally announced June 2023.

Comments: 25 pages, 17 figures, 6 tables, accepted to Nature Communications Chemistry. This work was awarded the HPCwire 2023 Editors' Choice Awards for Best Use of High Performance Data Analytics \& Artificial Intelligence see https://www.hpcwire.com/2023-readers-editors-choice-data-analytics-ai/

ACM Class: I.2

Journal ref: Commun Chem 7, 21 (2024)

Magnetohydrodynamics with Physics Informed Neural Operators

Authors: Shawn G. Rosofsky, E. A. Huerta

Abstract: The modeling of multi-scale and multi-physics complex systems typically involves the use of scientific software that can optimally leverage extreme scale computing. Despite major developments in recent years, these simulations continue to be computationally intensive and time consuming. Here we explore the use of AI to accelerate the modeling of complex systems at a fraction of the computational c… ▽ More The modeling of multi-scale and multi-physics complex systems typically involves the use of scientific software that can optimally leverage extreme scale computing. Despite major developments in recent years, these simulations continue to be computationally intensive and time consuming. Here we explore the use of AI to accelerate the modeling of complex systems at a fraction of the computational cost of classical methods, and present the first application of physics informed neural operators to model 2D incompressible magnetohydrodynamics simulations. Our AI models incorporate tensor Fourier neural operators as their backbone, which we implemented with the TensorLY package. Our results indicate that physics informed neural operators can accurately capture the physics of magnetohydrodynamics simulations that describe laminar flows with Reynolds numbers $Re\leq250$. We also explore the applicability of our AI surrogates for turbulent flows, and discuss a variety of methodologies that may be incorporated in future work to create AI models that provide a computationally efficient and high fidelity description of magnetohydrodynamics simulations for a broad range of Reynolds numbers. The scientific software developed in this project is released with this manuscript. △ Less

Submitted 7 July, 2023; v1 submitted 13 February, 2023; originally announced February 2023.

Comments: 32 pages, 24 figures, 1 table. First application of physics informed neural operators to solve magnetohydrodynamics equations, v2: Accepted to Machine Learning: Science and Technology

MSC Class: 35-04; ACM Class: I.2.0; J.2

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 035002

End-to-end AI framework for interpretable prediction of molecular and crystal properties

Authors: Hyun Park, Ruijie Zhu, E. A. Huerta, Santanu Chaudhuri, Emad Tajkhorshid, Donny Cooper

Abstract: We introduce an end-to-end computational framework that allows for hyperparameter optimization using the DeepHyper library, accelerated model training, and interpretable AI inference. The framework is based on state-of-the-art AI models including CGCNN, PhysNet, SchNet, MPNN, MPNN-transformer, and TorchMD-NET. We employ these AI models along with the benchmark QM9, hMOF, and MD17 datasets to showc… ▽ More We introduce an end-to-end computational framework that allows for hyperparameter optimization using the DeepHyper library, accelerated model training, and interpretable AI inference. The framework is based on state-of-the-art AI models including CGCNN, PhysNet, SchNet, MPNN, MPNN-transformer, and TorchMD-NET. We employ these AI models along with the benchmark QM9, hMOF, and MD17 datasets to showcase how the models can predict user-specified material properties within modern computing environments. We demonstrate transferable applications in the modeling of small molecules, inorganic crystals and nanoporous metal organic frameworks with a unified, standalone framework. We have deployed and tested this framework in the ThetaGPU supercomputer at the Argonne Leadership Computing Facility, and in the Delta supercomputer at the National Center for Supercomputing Applications to provide researchers with modern tools to conduct accelerated AI-driven discovery in leadership-class computing environments. We release these digital assets as open source scientific software in GitLab, and ready-to-use Jupyter notebooks in Google Colab. △ Less

Submitted 14 August, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Comments: 20 pages, 10 images, 6 tables; v2: accepted to Machine Learning: Science and Technology

ACM Class: I.2

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 025036

FAIR AI Models in High Energy Physics

Authors: Javier Duarte, Haoyang Li, Avik Roy, Ruike Zhu, E. A. Huerta, Daniel Diaz, Philip Harris, Raghav Kansal, Daniel S. Katz, Ishaan H. Kavoori, Volodymyr V. Kindratenko, Farouk Mokhtar, Mark S. Neubauer, Sang Eon Park, Melissa Quinnan, Roger Rusack, Zhizhen Zhao

Abstract: The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning (ML) models -- algorithms that have been trained on data without being explicitly… ▽ More The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning (ML) models -- algorithms that have been trained on data without being explicitly programmed -- and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template's use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability. △ Less

Submitted 29 December, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: 34 pages, 9 figures, 10 tables

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 045062

FAIR for AI: An interdisciplinary and international community building perspective

Authors: E. A. Huerta, Ben Blaiszik, L. Catherine Brinson, Kristofer E. Bouchard, Daniel Diaz, Caterina Doglioni, Javier M. Duarte, Murali Emani, Ian Foster, Geoffrey Fox, Philip Harris, Lukas Heinrich, Shantenu Jha, Daniel S. Katz, Volodymyr Kindratenko, Christine R. Kirkpatrick, Kati Lassila-Perini, Ravi K. Madduri, Mark S. Neubauer, Fotis E. Psomopoulos, Avik Roy, Oliver Rübel, Zhizhen Zhao, Ruike Zhu

Abstract: A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to i… ▽ More A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022. △ Less

Submitted 1 August, 2023; v1 submitted 30 September, 2022; originally announced October 2022.

Comments: 10 pages, comments welcome!; v2: 12 pages, accepted to Scientific Data

ACM Class: I.2.0; E.0

Journal ref: Scientific Data 10, 487 (2023)

MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge

Authors: Marlin B. Schäfer, Ondřej Zelenka, Alexander H. Nitz, He Wang, Shichao Wu, Zong-Kuan Guo, Zhoujian Cao, Zhixiang Ren, Paraskevi Nousi, Nikolaos Stergioulas, Panagiotis Iosif, Alexandra E. Koloniari, Anastasios Tefas, Nikolaos Passalis, Francesco Salemi, Gabriele Vedovato, Sergey Klimenko, Tanmaya Mishra, Bernd Brügmann, Elena Cuoco, E. A. Huerta, Chris Messenger, Frank Ohme

Abstract: We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. The final of the 4 provided datasets contained real noise from the O3a observing run and… ▽ More We present the results of the first Machine Learning Gravitational-Wave Search Mock Data Challenge (MLGWSC-1). For this challenge, participating groups had to identify gravitational-wave signals from binary black hole mergers of increasing complexity and duration embedded in progressively more realistic noise. The final of the 4 provided datasets contained real noise from the O3a observing run and signals up to a duration of 20 seconds with the inclusion of precession effects and higher order modes. We present the average sensitivity distance and runtime for the 6 entered algorithms derived from 1 month of test data unknown to the participants prior to submission. Of these, 4 are machine learning algorithms. We find that the best machine learning based algorithms are able to achieve up to 95% of the sensitive distance of matched-filtering based production analyses for simulated Gaussian noise at a false-alarm rate (FAR) of one per month. In contrast, for real noise, the leading machine learning search achieved 70%. For higher FARs the differences in sensitive distance shrink to the point where select machine learning submissions outperform traditional search algorithms at FARs $\geq 200$ per month on some datasets. Our results show that current machine learning search algorithms may already be sensitive enough in limited parameter regions to be useful for some production settings. To improve the state-of-the-art, machine learning algorithms need to reduce the false-alarm rates at which they are capable of detecting signals and extend their validity to regions of parameter space where modeled searches are computationally expensive to run. Based on our findings we compile a list of research areas that we believe are the most important to elevate machine learning searches to an invaluable tool in gravitational-wave signal detection. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Comments: 25 pages, 6 figures, 4 tables, additional material available at https://github.com/gwastro/ml-mock-data-challenge-1

Benchmarking the face-centred finite volume method for compressible laminar flows

Authors: Jordi Vila-Pérez, Matteo Giacomini, Antonio Huerta

Abstract: Purpose: This study aims to assess the robustness and accuracy of the face-centred finite volume (FCFV) method for the simulation of compressible laminar flows in different regimes, using numerical benchmarks. Design/methodology/approach: The work presents a detailed comparison with reference solutions published in the literature -- when available -- and numerical results computed using a commer… ▽ More Purpose: This study aims to assess the robustness and accuracy of the face-centred finite volume (FCFV) method for the simulation of compressible laminar flows in different regimes, using numerical benchmarks. Design/methodology/approach: The work presents a detailed comparison with reference solutions published in the literature -- when available -- and numerical results computed using a commercial cell-centred finite volume software. Findings: The FCFV scheme provides first-order accurate approximations of the viscous stress tensor and the heat flux, insensitively to cell distortion or stretching. The strategy demonstrates its efficiency in inviscid and viscous flows, for a wide range of Mach numbers, also in the incompressible limit. In purely inviscid flows, non-oscillatory approximations are obtained in the presence of shock waves. In the incompressible limit, accurate solutions are computed without pressure correction algorithms. The method shows its superior performance for viscous high Mach number flows, achieving physically admissible solutions without carbuncle effect and predictions of quantities of interest with errors below 5%. Originality/value: The FCFV method accurately evaluates, for a wide range of compressible laminar flows, quantities of engineering interest, such as drag, lift and heat transfer coefficients, on unstructured meshes featuring distorted and highly stretched cells, with an aspect ratio up to ten thousand. The method is suitable to simulate industrial flows on complex geometries, relaxing the requirements on mesh quality introduced by existing finite volume solvers and alleviating the need for time-consuming manual procedures for mesh generation to be performed by specialised technicians. △ Less

Submitted 11 December, 2022; v1 submitted 4 August, 2022; originally announced August 2022.

Comments: 39 pages, 18 figures, 12 tables

MSC Class: 76M12; 76Nxx; 65M12; 76G25; 76H05; 76J20

Journal ref: International Journal of Numerical Methods for Heat & Fluid Flow, Vol. 33, Issue 6, pp. 2198-2231, 2023

FAIR principles for AI models with a practical application for accelerated high energy diffraction microscopy

Authors: Nikil Ravi, Pranshu Chaturvedi, E. A. Huerta, Zhengchun Liu, Ryan Chard, Aristana Scourtas, K. J. Schmidt, Kyle Chard, Ben Blaiszik, Ian Foster

Abstract: A concise and measurable set of FAIR (Findable, Accessible, Interoperable and Reusable) principles for scientific data is transforming the state-of-practice for data management and stewardship, supporting and enabling discovery and innovation. Learning from this initiative, and acknowledging the impact of artificial intelligence (AI) in the practice of science and engineering, we introduce a set o… ▽ More A concise and measurable set of FAIR (Findable, Accessible, Interoperable and Reusable) principles for scientific data is transforming the state-of-practice for data management and stewardship, supporting and enabling discovery and innovation. Learning from this initiative, and acknowledging the impact of artificial intelligence (AI) in the practice of science and engineering, we introduce a set of practical, concise, and measurable FAIR principles for AI models. We showcase how to create and share FAIR data and AI models within a unified computational framework combining the following elements: the Advanced Photon Source at Argonne National Laboratory, the Materials Data Facility, the Data and Learning Hub for Science, and funcX, and the Argonne Leadership Computing Facility (ALCF), in particular the ThetaGPU supercomputer and the SambaNova DataScale system at the ALCF AI Testbed. We describe how this domain-agnostic computational framework may be harnessed to enable autonomous AI-driven discovery. △ Less

Submitted 21 December, 2022; v1 submitted 1 July, 2022; originally announced July 2022.

Comments: 11 pages, 3 figures; Accepted to Scientific Data; for press release see https://www.anl.gov/article/argonne-scientists-promote-fair-standards-for-managing-artificial-intelligence-models and https://www.ncsa.illinois.edu/ncsa-student-researchers-lead-authors-on-award-winning-paper; Received 2022 HPCwire Readers' Choice Award on Best Use of High Performance Data Analytics & Artificial Intelligence

MSC Class: 68T01; 68T05 ACM Class: I.2; J.2

Journal ref: Scientific Data 9, 657 (2022)

Applications of physics informed neural operators

Authors: Shawn G. Rosofsky, Hani Al Majed, E. A. Huerta

Abstract: We present an end-to-end framework to learn partial differential equations that brings together initial data production, selection of boundary conditions, and the use of physics-informed neural operators to solve partial differential equations that are ubiquitous in the study and modeling of physics phenomena. We first demonstrate that our methods reproduce the accuracy and performance of other ne… ▽ More We present an end-to-end framework to learn partial differential equations that brings together initial data production, selection of boundary conditions, and the use of physics-informed neural operators to solve partial differential equations that are ubiquitous in the study and modeling of physics phenomena. We first demonstrate that our methods reproduce the accuracy and performance of other neural operators published elsewhere in the literature to learn the 1D wave equation and the 1D Burgers equation. Thereafter, we apply our physics-informed neural operators to learn new types of equations, including the 2D Burgers equation in the scalar, inviscid and vector types. Finally, we show that our approach is also applicable to learn the physics of the 2D linear and nonlinear shallow water equations, which involve three coupled partial differential equations. We release our artificial intelligence surrogates and scientific software to produce initial data and boundary conditions to study a broad range of physically motivated scenarios. We provide the source code, an interactive website to visualize the predictions of our physics informed neural operators, and a tutorial for their use at the Data and Learning Hub for Science. △ Less

Submitted 8 December, 2022; v1 submitted 23 March, 2022; originally announced March 2022.

Comments: 15 pages, 12 figures

Journal ref: Mach. Learn.: Sci. Technol. 4 (2023) 025022

Interpreting a Machine Learning Model for Detecting Gravitational Waves

Authors: Mohammadtaher Safarzadeh, Asad Khan, E. A. Huerta, Martin Wattenberg

Abstract: We describe a case study of translational research, applying interpretability techniques developed for computer vision to machine learning models used to search for and find gravitational waves. The models we study are trained to detect black hole merger events in non-Gaussian and non-stationary advanced Laser Interferometer Gravitational-wave Observatory (LIGO) data. We produced visualizations of… ▽ More We describe a case study of translational research, applying interpretability techniques developed for computer vision to machine learning models used to search for and find gravitational waves. The models we study are trained to detect black hole merger events in non-Gaussian and non-stationary advanced Laser Interferometer Gravitational-wave Observatory (LIGO) data. We produced visualizations of the response of machine learning models when they process advanced LIGO data that contains real gravitational wave signals, noise anomalies, and pure advanced LIGO noise. Our findings shed light on the responses of individual neurons in these machine learning models. Further analysis suggests that different parts of the network appear to specialize in local versus global features, and that this difference appears to be rooted in the branched architecture of the network as well as noise characteristics of the LIGO detectors. We believe efforts to whiten these "black box" models can suggest future avenues for research and help inform the design of interpretable machine learning models for gravitational wave astrophysics. △ Less

Submitted 15 February, 2022; originally announced February 2022.

Comments: 19 pages, to be submitted, comments are welcome. Movies based on this work can be accessed via: https://www.youtube.com/watch?v=SXFGMOtJwn0 https://www.youtube.com/watch?v=itVCj9gpmAs

Inference-optimized AI and high performance computing for gravitational wave detection at scale

Authors: Pranshu Chaturvedi, Asad Khan, Minyang Tian, E. A. Huerta, Huihuo Zheng

Abstract: We introduce an ensemble of artificial intelligence models for gravitational wave detection that we trained in the Summit supercomputer using 32 nodes, equivalent to 192 NVIDIA V100 GPUs, within 2 hours. Once fully trained, we optimized these models for accelerated inference using NVIDIA TensorRT. We deployed our inference-optimized AI ensemble in the ThetaGPU supercomputer at Argonne Leadership C… ▽ More We introduce an ensemble of artificial intelligence models for gravitational wave detection that we trained in the Summit supercomputer using 32 nodes, equivalent to 192 NVIDIA V100 GPUs, within 2 hours. Once fully trained, we optimized these models for accelerated inference using NVIDIA TensorRT. We deployed our inference-optimized AI ensemble in the ThetaGPU supercomputer at Argonne Leadership Computer Facility to conduct distributed inference. Using the entire ThetaGPU supercomputer, consisting of 20 nodes each of which has 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, our NVIDIA TensorRT-optimized AI ensemble processed an entire month of advanced LIGO data (including Hanford and Livingston data streams) within 50 seconds. Our inference-optimized AI ensemble retains the same sensitivity of traditional AI models, namely, it identifies all known binary black hole mergers previously identified in this advanced LIGO dataset and reports no misclassifications, while also providing a 3X inference speedup compared to traditional artificial intelligence models. We used time slides to quantify the performance of our AI ensemble to process up to 5 years worth of advanced LIGO data. In this synthetically enhanced dataset, our AI ensemble reports an average of one misclassification for every month of searched advanced LIGO data. We also present the receiver operating characteristic curve of our AI ensemble using this 5 year long advanced LIGO dataset. This approach provides the required tools to conduct accelerated, AI-driven gravitational wave detection at scale. △ Less

Submitted 17 February, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

Comments: 19 pages, 8 figures; v2. Accepted to Frontiers in Artificial Intelligence, Special Issue: Efficient AI in Particle Physics and Astrophysics

MSC Class: 68T10; 85-08; 83C35; 83C57 ACM Class: I.2

Journal ref: Front. Artif. Intell. 5:828672 (2022)

AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

Authors: Asad Khan, E. A. Huerta, Prayush Kumar

Abstract: We use artificial intelligence (AI) to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non precessing binary black hole mergers. We trained AI models using 14 million waveforms, produced with the surrogate model NRHybSur3dq8, that include modes up to $\ell \leq 4$ and $(5,5)$, except for $(4,0)$ and $(4,1)$, that describe binaries with mass-ratios… ▽ More We use artificial intelligence (AI) to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non precessing binary black hole mergers. We trained AI models using 14 million waveforms, produced with the surrogate model NRHybSur3dq8, that include modes up to $\ell \leq 4$ and $(5,5)$, except for $(4,0)$ and $(4,1)$, that describe binaries with mass-ratios $q\leq8$, individual spins $s^z_{\{1,2\}}\in[-0.8, 0.8]$, and inclination angle $θ\in[0,π]$.Our probabilistic AI surrogates can accurately constrain the mass-ratio, individual spins, effective spin, and inclination angle of numerical relativity waveforms that describe such signal manifold. We compared the predictions of our AI models with Gaussian process regression, random forest, k-nearest neighbors, and linear regression, and with traditional Bayesian inference methods through the PyCBC Inference toolkit, finding that AI outperforms all these approaches in terms of accuracy, and are between three to four orders of magnitude faster than traditional Bayesian inference methods. Our AI surrogates were trained within 3.4 hours using distributed training on 1,536 NVIDIA V100 GPUs in the Summit supercomputer. △ Less

Submitted 26 October, 2022; v1 submitted 13 December, 2021; originally announced December 2021.

Comments: 22 pages, 12 figures

MSC Class: 68T10; 85-08; 83C35; 83C57 ACM Class: I.2

Journal ref: Physics Letters B, Volume 835, 10 December 2022, 137505

Interpretable AI forecasting for numerical relativity waveforms of quasi-circular, spinning, non-precessing binary black hole mergers

Authors: Asad Khan, E. A. Huerta, Huihuo Zheng

Abstract: We present a deep-learning artificial intelligence model that is capable of learning and forecasting the late-inspiral, merger and ringdown of numerical relativity waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers. We used the NRHybSur3dq8 surrogate model to produce train, validation and test sets of $\ell=|m|=2$ waveforms that cover the parameter space of… ▽ More We present a deep-learning artificial intelligence model that is capable of learning and forecasting the late-inspiral, merger and ringdown of numerical relativity waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers. We used the NRHybSur3dq8 surrogate model to produce train, validation and test sets of $\ell=|m|=2$ waveforms that cover the parameter space of binary black hole mergers with mass-ratios $q\leq8$ and individual spins $|s^z_{\{1,2\}}| \leq 0.8$. These waveforms cover the time range $t\in[-5000\textrm{M}, 130\textrm{M}]$, where $t=0M$ marks the merger event, defined as the maximum value of the waveform amplitude. We harnessed the ThetaGPU supercomputer at the Argonne Leadership Computing Facility to train our AI model using a training set of 1.5 million waveforms. We used 16 NVIDIA DGX A100 nodes, each consisting of 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, to fully train our model within 3.5 hours. Our findings show that artificial intelligence can accurately forecast the dynamical evolution of numerical relativity waveforms in the time range $t\in[-100\textrm{M}, 130\textrm{M}]$. Sampling a test set of 190,000 waveforms, we find that the average overlap between target and predicted waveforms is $\gtrsim99\%$ over the entire parameter space under consideration. We also combined scientific visualization and accelerated computing to identify what components of our model take in knowledge from the early and late-time waveform evolution to accurately forecast the latter part of numerical relativity waveforms. This work aims to accelerate the creation of scalable, computationally efficient and interpretable artificial intelligence models for gravitational wave astrophysics. △ Less

Submitted 17 January, 2022; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: 15 pages, 7 figures, 2 appendices, 1 interactive visualization at https://khanx169.github.io/gw_forecasting/interactive_results.html

MSC Class: 68T10; 85-08; 83C35; 83C57 ACM Class: I.2

Journal ref: Phys. Rev. D 105, 024024 (2022)

A FAIR and AI-ready Higgs boson decay dataset

Authors: Yifan Chen, E. A. Huerta, Javier Duarte, Philip Harris, Daniel S. Katz, Mark S. Neubauer, Daniel Diaz, Farouk Mokhtar, Raghav Kansal, Sang Eon Park, Volodymyr V. Kindratenko, Zhizhen Zhao, Roger Rusack

Abstract: To enable the reusability of massive scientific datasets by humans and machines, researchers aim to adhere to the principles of findability, accessibility, interoperability, and reusability (FAIR) for data and artificial intelligence (AI) models. This article provides a domain-agnostic, step-by-step assessment guide to evaluate whether or not a given dataset meets these principles. We demonstrate… ▽ More To enable the reusability of massive scientific datasets by humans and machines, researchers aim to adhere to the principles of findability, accessibility, interoperability, and reusability (FAIR) for data and artificial intelligence (AI) models. This article provides a domain-agnostic, step-by-step assessment guide to evaluate whether or not a given dataset meets these principles. We demonstrate how to use this guide to evaluate the FAIRness of an open simulated dataset produced by the CMS Collaboration at the CERN Large Hadron Collider. This dataset consists of Higgs boson decays and quark and gluon background, and is available through the CERN Open Data Portal. We use additional available tools to assess the FAIRness of this dataset, and incorporate feedback from members of the FAIR community to validate our results. This article is accompanied by a Jupyter notebook to visualize and explore this dataset. This study marks the first in a planned series of articles that will guide scientists in the creation of FAIR AI models and datasets in high energy particle physics. △ Less

Submitted 16 February, 2022; v1 submitted 4 August, 2021; originally announced August 2021.

Comments: 13 pages, 3 figures. v2: Accepted to Nature Scientific Data. Learn about the FAIR4HEP project at https://fair4hep.github.io. See our invited Behind the Paper Blog in Springer Nature Research Data Community at https://go.nature.com/3oMVYxo

ACM Class: I.2; J.2

Journal ref: Scientific Data volume 9, Article number: 31 (2022)

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

Authors: E. A. Huerta, Zhizhen Zhao

Abstract: We live in momentous times. The science community is empowered with an arsenal of cosmic messengers to study the Universe in unprecedented detail. Gravitational waves, electromagnetic waves, neutrinos and cosmic rays cover a wide range of wavelengths and time scales. Combining and processing these datasets that vary in volume, speed and dimensionality requires new modes of instrument coordination,… ▽ More We live in momentous times. The science community is empowered with an arsenal of cosmic messengers to study the Universe in unprecedented detail. Gravitational waves, electromagnetic waves, neutrinos and cosmic rays cover a wide range of wavelengths and time scales. Combining and processing these datasets that vary in volume, speed and dimensionality requires new modes of instrument coordination, funding and international collaboration with a specialized human and technological infrastructure. In tandem with the advent of large-scale scientific facilities, the last decade has experienced an unprecedented transformation in computing and signal processing algorithms. The combination of graphics processing units, deep learning, and the availability of open source, high-quality datasets, have powered the rise of artificial intelligence. This digital revolution now powers a multi-billion dollar industry, with far-reaching implications in technology and society. In this chapter we describe pioneering efforts to adapt artificial intelligence algorithms to address computational grand challenges in Multi-Messenger Astrophysics. We review the rapid evolution of these disruptive algorithms, from the first class of algorithms introduced in early 2017, to the sophisticated algorithms that now incorporate domain expertise in their architectural design and optimization schemes. We discuss the importance of scientific visualization and extreme-scale computing in reducing time-to-insight and obtaining new knowledge from the interplay between models and data. △ Less

Submitted 1 October, 2021; v1 submitted 13 May, 2021; originally announced May 2021.

Comments: 30 pages, 11 figures. Invited chapter for "Handbook of Gravitational Wave Astronomy"; v2: updated to reflect published version

MSC Class: 83-02; 83-04; 83-08; 85-08; 85-10; 68T07; 68T20; ACM Class: I.2; I.3; I.5; J.2

Accelerated, Scalable and Reproducible AI-driven Gravitational Wave Detection

Authors: E. A. Huerta, Asad Khan, Xiaobo Huang, Minyang Tian, Maksim Levental, Ryan Chard, Wei Wei, Maeve Heflin, Daniel S. Katz, Volodymyr Kindratenko, Dawei Mu, Ben Blaiszik, Ian Foster

Abstract: The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astrophysics. Here we develop a workflow that connects the Data and Learning Hub for Science, a repository for publishing AI models, with the Hardware Accelerated Learning (HAL) cluster, using funcX as a universal distributed… ▽ More The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astrophysics. Here we develop a workflow that connects the Data and Learning Hub for Science, a repository for publishing AI models, with the Hardware Accelerated Learning (HAL) cluster, using funcX as a universal distributed computing service. Using this workflow, an ensemble of four openly available AI models can be run on HAL to process an entire month's worth (August 2017) of advanced Laser Interferometer Gravitational-Wave Observatory data in just seven minutes, identifying all four all four binary black hole mergers previously identified in this dataset and reporting no misclassifications. This approach combines advances in AI, distributed computing, and scientific data infrastructure to open new pathways to conduct reproducible, accelerated, data-driven discovery. △ Less

Submitted 9 July, 2021; v1 submitted 15 December, 2020; originally announced December 2020.

Comments: 17 pages, 5 figures; v2: 12 pages, 6 figures. Accepted to Nature Astronomy. See also the Behind the Paper blog in Nature Astronomy "https://astronomycommunity.nature.com/posts/from-disruption-to-sustained-innovation-artificial-intelligence-for-gravitational-wave-astrophysics"

MSC Class: 68T01; 68T35; 83C35; 83C57

Journal ref: Nat Astron 5, 1062-1068 (2021)

A non-oscillatory face-centred finite volume method for compressible flows

Authors: Jordi Vila-Pérez, Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: This work presents the face-centred finite volume (FCFV) paradigm for the simulation of compressible flows. The FCFV method defines the unknowns at the face barycentre and uses a hybridisation procedure to eliminate all the degrees of freedom inside the cells. In addition, Riemann solvers are defined implicitly within the expressions of the numerical fluxes. The resulting methodology provides firs… ▽ More This work presents the face-centred finite volume (FCFV) paradigm for the simulation of compressible flows. The FCFV method defines the unknowns at the face barycentre and uses a hybridisation procedure to eliminate all the degrees of freedom inside the cells. In addition, Riemann solvers are defined implicitly within the expressions of the numerical fluxes. The resulting methodology provides first-order accurate approximations of the conservative quantities, i.e. density, momentum and energy, as well as of the viscous stress tensor and of the heat flux, without the need of any gradient reconstruction procedure. Hence, the FCFV solver preserves the accuracy of the approximation in presence of distorted and highly stretched cells, providing a solver insensitive to mesh quality. In addition, FCFV is capable of constructing non-oscillatory approximations of sharp discontinuities without resorting to shock capturing or limiting techniques. For flows at low Mach number, the method is robust and is capable of computing accurate solutions in the incompressible limit without the need of introducing specific pressure correction strategies. A set of 2D and 3D benchmarks of external flows is presented to validate the methodology in different flow regimes, from inviscid to viscous laminar flows, from transonic to subsonic incompressible flows, demonstrating its potential to handle compressible flows in realistic scenarios. △ Less

Submitted 6 January, 2022; v1 submitted 26 November, 2020; originally announced November 2020.

Comments: 38 pages, 22 figures

MSC Class: 76M12; 76Nxx; 65M08; 65M12; 76G25; 76H05

Journal ref: Computers & Fluids, 235, 105272, 2022

HDGlab: An open-source implementation of the hybridisable discontinuous Galerkin method in MATLAB

Authors: Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: This paper presents HDGlab, an open source MATLAB implementation of the hybridisable discontinuous Galerkin (HDG) method. The main goal is to provide a detailed description of both the HDG method for elliptic problems and its implementation available in HDGlab. Ultimately, this is expected to make this relatively new advanced discretisation method more accessible to the computational engineering c… ▽ More This paper presents HDGlab, an open source MATLAB implementation of the hybridisable discontinuous Galerkin (HDG) method. The main goal is to provide a detailed description of both the HDG method for elliptic problems and its implementation available in HDGlab. Ultimately, this is expected to make this relatively new advanced discretisation method more accessible to the computational engineering community. HDGlab presents some features not available in other implementations of the HDG method that can be found in the free domain. First, it implements high-order polynomial shape functions up to degree nine, with both equally-spaced and Fekete nodal distributions. Second, it supports curved isoparametric simplicial elements in two and three dimensions. Third, it supports non-uniform degree polynomial approximations and it provides a flexible structure to devise degree adaptivity strategies. Finally, an interface with the open-source high-order mesh generator Gmsh is provided to facilitate its application to practical engineering problems. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Comments: 90 pages, 51 figures

MSC Class: 65-04; 35-04; 76-04; 65N30; 76M10

Journal ref: Archives of Computational Methods in Engineering, Vol. 28, Issue 3, pp. 1941-1986, 2021

Hybridisable discontinuous Galerkin formulation of compressible flows

Authors: Jordi Vila-Pérez, Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: This work presents a review of high-order hybridisable discontinuous Galerkin (HDG) methods in the context of compressible flows. Moreover, an original unified framework for the derivation of Riemann solvers in hybridised formulations is proposed. This framework includes, for the first time in an HDG context, the HLL and HLLEM Riemann solvers as well as the traditional Lax-Friedrichs and Roe solve… ▽ More This work presents a review of high-order hybridisable discontinuous Galerkin (HDG) methods in the context of compressible flows. Moreover, an original unified framework for the derivation of Riemann solvers in hybridised formulations is proposed. This framework includes, for the first time in an HDG context, the HLL and HLLEM Riemann solvers as well as the traditional Lax-Friedrichs and Roe solvers. HLL-type Riemann solvers demonstrate their superiority with respect to Roe in supersonic cases due to their positivity preserving properties. In addition, HLLEM specifically outstands in the approximation of boundary layers because of its shear preservation, which confers it an increased accuracy with respect to HLL and Lax-Friedrichs. A comprehensive set of relevant numerical benchmarks of viscous and inviscid compressible flows is presented. The test cases are used to evaluate the competitiveness of the resulting high-order HDG scheme with the aforementioned Riemann solvers and equipped with a shock treatment technique based on artificial viscosity. △ Less

Submitted 10 September, 2020; originally announced September 2020.

Comments: 60 pages, 31 figures. arXiv admin note: substantial text overlap with arXiv:1912.00044

MSC Class: 65M12; 65M22; 65M60; 76N15; 76N17

Journal ref: Archives of Computational Methods in Engineering, Vol. 28, Issue 2, pp. 753--784, 2021

A weakly compressible hybridizable discontinuous Galerkin formulation for fluid-structure interaction problems

Authors: Andrea La Spina, Martin Kronbichler, Matteo Giacomini, Wolfgang A. Wall, Antonio Huerta

Abstract: A scheme for the solution of fluid-structure interaction (FSI) problems with weakly compressible flows is proposed in this work. A novel hybridizable discontinuous Galerkin (HDG) method is derived for the discretization of the fluid equations, while the standard continuous Galerkin (CG) approach is adopted for the structural problem. The chosen HDG solver combines robustness of discontinuous Galer… ▽ More A scheme for the solution of fluid-structure interaction (FSI) problems with weakly compressible flows is proposed in this work. A novel hybridizable discontinuous Galerkin (HDG) method is derived for the discretization of the fluid equations, while the standard continuous Galerkin (CG) approach is adopted for the structural problem. The chosen HDG solver combines robustness of discontinuous Galerkin (DG) approaches in advection-dominated flows with higher order accuracy and efficient implementations. Two coupling strategies are examined in this contribution, namely a partitioned Dirichlet-Neumann scheme in the context of hybrid HDG-CG discretizations and a monolithic approach based on Nitsche's method, exploiting the definition of the numerical flux and the trace of the solution to impose the coupling conditions. Numerical experiments show optimal convergence of the HDG and CG primal and mixed variables and superconvergence of the postprocessed fluid velocity. The robustness and the efficiency of the proposed weakly compressible formulation, in comparison to a fully incompressible one, are also highlighted on a selection of two and three dimensional FSI benchmark problems. △ Less

Submitted 9 September, 2020; originally announced September 2020.

Comments: 49 pages, 20 figures, 2 tables

MSC Class: 65M60; 76D07; 76M10

Journal ref: Comput. Methods Appl. Mech. Eng., Volume 372, Article 113392 (2020)

Separated response surfaces for flows in parametrised domains: comparison of a priori and a posteriori PGD algorithms

Authors: Matteo Giacomini, Luca Borchini, Ruben Sevilla, Antonio Huerta

Abstract: Reduced order models (ROM) are commonly employed to solve parametric problems and to devise inexpensive response surfaces to evaluate quantities of interest in real-time. There are many families of ROMs in the literature and choosing among them is not always a trivial task. This work presents a comparison of the performance of a priori and a posteriori proper generalised decomposition (PGD) algori… ▽ More Reduced order models (ROM) are commonly employed to solve parametric problems and to devise inexpensive response surfaces to evaluate quantities of interest in real-time. There are many families of ROMs in the literature and choosing among them is not always a trivial task. This work presents a comparison of the performance of a priori and a posteriori proper generalised decomposition (PGD) algorithms for an incompressible Stokes flow problem in a geometrically parametrised domain. This problem is particularly challenging as the geometric parameters affect both the solution manifold and the computational spatial domain. The difficulty is further increased because multiple geometric parameters are considered and extended ranges of values are analysed for the parameters and this leads to significant variations in the flow features. Using a set of numerical experiments involving geometrically parametrised microswimmers, the two PGD algorithms are extensively compared in terms of their accuracy and their computational cost, expressed as a function of the number of full-order solves required. △ Less

Submitted 25 June, 2021; v1 submitted 4 September, 2020; originally announced September 2020.

Comments: 58 pages, 31 figures

MSC Class: 65M60; 76D07; 76M10

Journal ref: Finite Elements in Analysis and Design, Vol. 196, 103530, 2021

Hybridisable discontinuous Galerkin solution of geometrically parametrised Stokes flows

Authors: Ruben Sevilla, Luca Borchini, Matteo Giacomini, Antonio Huerta

Abstract: This paper proposes a novel computational framework for the solution of geometrically parametrised flow problems governed by the Stokes equation. The proposed method uses a high-order hybridisable discontinuous Galerkin formulation and the proper generalised decomposition rationale to construct an off-line solution for a given set of geometric parameters. The generalised solution contains the info… ▽ More This paper proposes a novel computational framework for the solution of geometrically parametrised flow problems governed by the Stokes equation. The proposed method uses a high-order hybridisable discontinuous Galerkin formulation and the proper generalised decomposition rationale to construct an off-line solution for a given set of geometric parameters. The generalised solution contains the information for all the geometric parameters in a user-defined range and it can be used to compute sensitivities. The proposed approach circumvents many of the weaknesses of other approaches based on the proper generalised decomposition for computing generalised solutions of geometrically parametrised problems. Four numerical examples show the optimal approximation properties of the proposed method and demonstrate its applicability in two and three dimensions. △ Less

Submitted 21 June, 2020; originally announced June 2020.

Comments: 51 pages, 30 figures

MSC Class: 65M60; 76D07; 76M10

Journal ref: Comput. Methods Appl. Mech. Eng., Volume 372, Article 113397 (2020)

Parametric solutions of turbulent incompressible flows in OpenFOAM via the proper generalised decomposition

Authors: Vasileios Tsiolakis, Matteo Giacomini, Ruben Sevilla, Carsten Othmer, Antonio Huerta

Abstract: An a priori reduced order method based on the proper generalised decomposition (PGD) is proposed to compute parametric solutions involving turbulent incompressible flows of interest in an industrial context, using OpenFOAM. The PGD framework is applied for the first time to the incompressible Navier-Stokes equations in the turbulent regime, to compute a generalised solution for velocity, pressure… ▽ More An a priori reduced order method based on the proper generalised decomposition (PGD) is proposed to compute parametric solutions involving turbulent incompressible flows of interest in an industrial context, using OpenFOAM. The PGD framework is applied for the first time to the incompressible Navier-Stokes equations in the turbulent regime, to compute a generalised solution for velocity, pressure and turbulent viscosity, explicitly depending on the design parameters of the problem. In order to simulate flows of industrial interest, a minimally intrusive implementation based on OpenFOAM SIMPLE algorithm applied to the Reynolds-averaged Navier-Stokes equations with the Spalart-Allmaras turbulence model is devised. The resulting PGD strategy is applied to parametric flow control problems and achieves both qualitative and quantitative agreement with the full order OpenFOAM solution for convection-dominated fully-developed turbulent incompressible flows, with Reynolds number up to one million. △ Less

Submitted 22 October, 2021; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: 44 pages, 13 figures, 2 tables

MSC Class: 65N08; 76D05; 76D55; 62P30

Journal ref: Journal of Computational Physics, 449, 110802, 2022

Physics-inspired deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers

Authors: Asad Khan, E. A. Huerta, Arnav Das

Abstract: The spin distribution of binary black hole mergers contains key information concerning the formation channels of these objects, and the astrophysical environments where they form, evolve and coalesce. To quantify the suitability of deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers, we introduce a modified version of WaveNet trai… ▽ More The spin distribution of binary black hole mergers contains key information concerning the formation channels of these objects, and the astrophysical environments where they form, evolve and coalesce. To quantify the suitability of deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers, we introduce a modified version of WaveNet trained with a novel optimization scheme that incorporates general relativistic constraints of the spin properties of astrophysical black holes. The neural network model is trained, validated and tested with 1.5 million $\ell=|m|=2$ waveforms generated within the regime of validity of NRHybSur3dq8, i.e., mass-ratios $q\leq8$ and individual black hole spins $ | s^z_{\{1,\,2\}} | \leq 0.8$. Using this neural network model, we quantify how accurately we can infer the astrophysical parameters of black hole mergers in the absence of noise. We do this by computing the overlap between waveforms in the testing data set and the corresponding signals whose mass-ratio and individual spins are predicted by our neural network. We find that the convergence of high performance computing and physics-inspired optimization algorithms enable an accurate reconstruction of the mass-ratio and individual spins of binary black hole mergers across the parameter space under consideration. This is a significant step towards an informed utilization of physics-inspired deep learning models to reconstruct the spin distribution of binary black hole mergers in realistic detection scenarios. △ Less

Submitted 25 August, 2020; v1 submitted 20 April, 2020; originally announced April 2020.

Comments: 25 pages, 12 figures, 1 appendix, 1 Interactive visualization at https://khanx169.github.io/smr_bbm_v2/interactive_results.html

MSC Class: 68T10; 85-08; 83C35; 83C57 ACM Class: I.2

Journal ref: Physics Letters B 808 (2020) 0370-2693

Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure

Authors: E. A. Huerta, Asad Khan, Edward Davis, Colleen Bushell, William D. Gropp, Daniel S. Katz, Volodymyr Kindratenko, Seid Koric, William T. C. Kramer, Brendan McGinty, Kenton McHenry, Aaron Saxton

Abstract: Significant investments to upgrade and construct large-scale scientific facilities demand commensurate investments in R&D to design algorithms and computing approaches to enable scientific and engineering breakthroughs in the big data era. Innovative Artificial Intelligence (AI) applications have powered transformational solutions for big data challenges in industry and technology that now drive a… ▽ More Significant investments to upgrade and construct large-scale scientific facilities demand commensurate investments in R&D to design algorithms and computing approaches to enable scientific and engineering breakthroughs in the big data era. Innovative Artificial Intelligence (AI) applications have powered transformational solutions for big data challenges in industry and technology that now drive a multi-billion dollar industry, and which play an ever increasing role shaping human social patterns. As AI continues to evolve into a computing paradigm endowed with statistical and mathematical rigor, it has become apparent that single-GPU solutions for training, validation, and testing are no longer sufficient for computational grand challenges brought about by scientific facilities that produce data at a rate and volume that outstrip the computing capabilities of available cyberinfrastructure platforms. This realization has been driving the confluence of AI and high performance computing (HPC) to reduce time-to-insight, and to enable a systematic study of domain-inspired AI architectures and optimization schemes to enable data-driven discovery. In this article we present a summary of recent developments in this field, and describe specific advances that authors in this article are spearheading to accelerate and streamline the use of HPC platforms to design and apply accelerated AI algorithms in academia and industry. △ Less

Submitted 19 October, 2020; v1 submitted 18 March, 2020; originally announced March 2020.

Comments: White paper accepted to the NSF Workshop on Smart Cyberinfrastructure, February 25-27, 2020 http://smartci.sci.utah.edu/. v2: Survey paper accepted to Journal of Big Data

MSC Class: 68T35; 68M14; 68N15; 68N30 ACM Class: I.2; I.6

Journal ref: Journal of Big Data volume 7, Article number: 88 (2020)

A kernel Principal Component Analysis (kPCA) digest with a new backward mapping (pre-image reconstruction) strategy

Authors: Alberto García-González, Antonio Huerta, Sergio Zlotnik, Pedro Díez

Abstract: Methodologies for multidimensionality reduction aim at discovering low-dimensional manifolds where data ranges. Principal Component Analysis (PCA) is very effective if data have linear structure. But fails in identifying a possible dimensionality reduction if data belong to a nonlinear low-dimensional manifold. For nonlinear dimensionality reduction, kernel Principal Component Analysis (kPCA) is a… ▽ More Methodologies for multidimensionality reduction aim at discovering low-dimensional manifolds where data ranges. Principal Component Analysis (PCA) is very effective if data have linear structure. But fails in identifying a possible dimensionality reduction if data belong to a nonlinear low-dimensional manifold. For nonlinear dimensionality reduction, kernel Principal Component Analysis (kPCA) is appreciated because of its simplicity and ease implementation. The paper provides a concise review of PCA and kPCA main ideas, trying to collect in a single document aspects that are often dispersed. Moreover, a strategy to map back the reduced dimension into the original high dimensional space is also devised, based on the minimization of a discrepancy functional. △ Less

Submitted 13 January, 2021; v1 submitted 7 January, 2020; originally announced January 2020.

Deep Learning for Cardiologist-level Myocardial Infarction Detection in Electrocardiograms

Authors: Arjun Gupta, E. A. Huerta, Zhizhen Zhao, Issam Moussa

Abstract: Myocardial infarction is the leading cause of death worldwide. In this paper, we design domain-inspired neural network models to detect myocardial infarction. First, we study the contribution of various leads. This systematic analysis, first of its kind in the literature, indicates that out of 15 ECG leads, data from the v6, vz, and ii leads are critical to correctly identify myocardial infarction… ▽ More Myocardial infarction is the leading cause of death worldwide. In this paper, we design domain-inspired neural network models to detect myocardial infarction. First, we study the contribution of various leads. This systematic analysis, first of its kind in the literature, indicates that out of 15 ECG leads, data from the v6, vz, and ii leads are critical to correctly identify myocardial infarction. Second, we use this finding and adapt the ConvNetQuake neural network model--originally designed to identify earthquakes--to attain state-of-the-art classification results for myocardial infarction, achieving $99.43\%$ classification accuracy on a record-wise split, and $97.83\%$ classification accuracy on a patient-wise split. These two results represent cardiologist-level performance level for myocardial infarction detection after feeding only 10 seconds of raw ECG data into our model. Third, we show that our multi-ECG-channel neural network achieves cardiologist-level performance without the need of any kind of manual feature extraction or data pre-processing. △ Less

Submitted 21 September, 2020; v1 submitted 16 December, 2019; originally announced December 2019.

Comments: Accepted to the European Medical and Biological Engineering Conference (EMBEC) 2020

MSC Class: 97R40; 68Txx; 92C50 ACM Class: I.2; I.5; J.3

An HLL Riemann solver for the hybridised discontinuous Galerkin formulation of compressible flows

Authors: Jordi Vila-Pérez, Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: This work proposes a high-order hybridised discontinuous Galerkin (HDG) formulation of the Harten-Lax-Van Leer (HLL) Riemann solver for compressible flows. A unified framework is introduced to present Lax-Friedrichs, Roe and HLL Riemann solvers via appropriate definitions of the HDG numerical fluxes. The resulting high-order HDG method with HLL Riemann solver is evaluated through a set of numerica… ▽ More This work proposes a high-order hybridised discontinuous Galerkin (HDG) formulation of the Harten-Lax-Van Leer (HLL) Riemann solver for compressible flows. A unified framework is introduced to present Lax-Friedrichs, Roe and HLL Riemann solvers via appropriate definitions of the HDG numerical fluxes. The resulting high-order HDG method with HLL Riemann solver is evaluated through a set of numerical simulations of inviscid compressible flows in different regimes, from subsonic isentropic flows to transonic and supersonic problems with shocks. The accuracy of the proposed method is comparable with the one of Lax-Friedrichs and Roe numerical fluxes in subsonic and transonic flows. The superior performance of HLL is highlighted in supersonic cases, where the method provides extra robustness, being able to produce positivity preserving approximations without the need of any user-defined entropy fix. △ Less

Submitted 25 September, 2020; v1 submitted 29 November, 2019; originally announced December 2019.

Comments: An extended and updated version of the paper covering both inviscid and viscous compressible flows is available at arXiv:2009.06396

MSC Class: 65M60; 76M10; 76H05; 76J20; 76N15

Enabling real-time multi-messenger astrophysics discoveries with deep learning

Authors: E. A. Huerta, Gabrielle Allen, Igor Andreoni, Javier M. Antelis, Etienne Bachelet, Bruce Berriman, Federica Bianco, Rahul Biswas, Matias Carrasco, Kyle Chard, Minsik Cho, Philip S. Cowperthwaite, Zachariah B. Etienne, Maya Fishbach, Francisco Förster, Daniel George, Tom Gibbs, Matthew Graham, William Gropp, Robert Gruendl, Anushri Gupta, Roland Haas, Sarah Habib, Elise Jennings, Margaret W. G. Johnson , et al. (35 additional authors not shown)

Abstract: Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravit… ▽ More Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and the need to build a community of experts to realize the goals of multi-messenger astrophysics. △ Less

Submitted 26 November, 2019; originally announced November 2019.

Comments: Invited Expert Recommendation for Nature Reviews Physics. The art work produced by E. A. Huerta and Shawn Rosofsky for this article was used by Carl Conway to design the cover of the October 2019 issue of Nature Reviews Physics

Journal ref: Nature Reviews Physics volume 1, pages 600-608 (2019)

Tutorial on Hybridizable Discontinuous Galerkin (HDG) Formulation for Incompressible Flow Problems

Authors: Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: A hybridizable discontinuous Galerkin (HDG) formulation of the linearized incompressible Navier-Stokes equations, known as Oseen equations, is presented. The Cauchy stress formulation is considered and the symmetry of the stress tensor and the mixed variable, namely the scaled strain-rate tensor, is enforced pointwise via Voigt notation. Using equal-order polynomial approximations of degree k for… ▽ More A hybridizable discontinuous Galerkin (HDG) formulation of the linearized incompressible Navier-Stokes equations, known as Oseen equations, is presented. The Cauchy stress formulation is considered and the symmetry of the stress tensor and the mixed variable, namely the scaled strain-rate tensor, is enforced pointwise via Voigt notation. Using equal-order polynomial approximations of degree k for all variables, HDG provides a stable discretization. Moreover, owing to Voigt notation, optimal convergence of order k+1 is obtained for velocity, pressure and strain-rate tensor and a local postprocessing strategy is devised to construct an approximation of the velocity superconverging with order k+2, even for low-order polynomial approximations. A tutorial for the numerical solution of incompressible flow problems using HDG is presented, with special emphasis on the technical details required for its implementation. △ Less

Submitted 12 November, 2019; originally announced November 2019.

Comments: 35 pages, 13 figures

MSC Class: 65M60; 76D07; 76M10

Journal ref: In: Modeling in Engineering Using Innovative Numerical Methods for Solids and Fluids. Eds: L. De Lorenzis and A. Düster. CISM International Centre for Mechanical Sciences. Volume 599, pp. 163--201. Springer International Publishing (2020)

A second-order face-centred finite volume method for elliptic problems

Authors: Luan M Vieira, Matteo Giacomini, Ruben Sevilla, Antonio Huerta

Abstract: A second-order face-centred finite volume method (FCFV) is proposed. Contrary to the more popular cell-centred and vertex-centred finite volume (FV) techniques, the proposed method defines the solution on the faces of the mesh (edges in two dimensions). The method is based on a mixed formulation and therefore considers the solution and its gradient as independent unknowns. They are computed solvin… ▽ More A second-order face-centred finite volume method (FCFV) is proposed. Contrary to the more popular cell-centred and vertex-centred finite volume (FV) techniques, the proposed method defines the solution on the faces of the mesh (edges in two dimensions). The method is based on a mixed formulation and therefore considers the solution and its gradient as independent unknowns. They are computed solving an element-by-element problem after the solution at the faces is determined. The proposed approach avoids the need of reconstructing the solution gradient, as required by cell-centred and vertex-centred FV methods. This strategy leads to a method that is insensitive to mesh distortion and stretching. The current method is second-order and requires the solution of a global system of equations of identical size and identical number of non-zero elements when compared to the recently proposed first-order FCFV. The formulation is presented for Poisson and Stokes problems. Numerical examples are used to illustrate the approximation properties of the method as well as to demonstrate its potential in three dimensional problems with complex geometries. The integration of a mesh adaptive procedure in the FCFV solution algorithm is also presented. △ Less

Submitted 8 August, 2019; originally announced August 2019.

Comments: 31 pages, 24 figures

MSC Class: 65N08; 65N30; 65N12

Journal ref: Comput. Methods Appl. Mech. Eng., Volume 358, Article 112655 (2020)

Hybrid coupling of CG and HDG discretizations based on Nitsche's method

Authors: Andrea La Spina, Matteo Giacomini, Antonio Huerta

Abstract: A strategy to couple continuous Galerkin (CG) and hybridizable discontinuous Galerkin (HDG) discretizations based only on the HDG hybrid variable is presented for linear thermal and elastic problems. The hybrid CG-HDG coupling exploits the definition of the numerical flux and the trace of the solution on the mesh faces to impose the transmission conditions between the CG and HDG subdomains. The co… ▽ More A strategy to couple continuous Galerkin (CG) and hybridizable discontinuous Galerkin (HDG) discretizations based only on the HDG hybrid variable is presented for linear thermal and elastic problems. The hybrid CG-HDG coupling exploits the definition of the numerical flux and the trace of the solution on the mesh faces to impose the transmission conditions between the CG and HDG subdomains. The continuity of the solution is imposed in the CG problem via Nitsche's method, whereas the equilibrium of the flux at the interface is naturally enforced as a Neumann condition in the HDG global problem. The proposed strategy does not affect the core structure of CG and HDG discretizations. In fact, the resulting formulation leads to a minimally-intrusive coupling, suitable to be integrated in existing CG and HDG libraries. Numerical experiments in two and three dimensions show optimal global convergence of the stress and superconvergence of the displacement field, locking-free approximation, as well as the potential to treat structural problems of engineering interest featuring multiple materials with compressible and nearly incompressible behaviors. △ Less

Submitted 25 June, 2019; originally announced June 2019.

Comments: 28 pages, 14 figures

MSC Class: 65N30; 74S05; 74B05; 65N12

Journal ref: Comput. Mech., 65, 311--330 (2020)

Nonintrusive proper generalised decomposition for parametrised incompressible flow problems in OpenFOAM

Authors: Vasileios Tsiolakis, Matteo Giacomini, Ruben Sevilla, Carsten Othmer, Antonio Huerta

Abstract: The computational cost of parametric studies currently represents the major limitation to the application of simulation-based engineering techniques in a daily industrial environment. This work presents the first nonintrusive implementation of the proper generalised decomposition (PGD) in OpenFOAM, for the approximation of parametrised laminar incompressible Navier-Stokes equations. The key featur… ▽ More The computational cost of parametric studies currently represents the major limitation to the application of simulation-based engineering techniques in a daily industrial environment. This work presents the first nonintrusive implementation of the proper generalised decomposition (PGD) in OpenFOAM, for the approximation of parametrised laminar incompressible Navier-Stokes equations. The key feature of this approach is the seamless integration of a reduced order model (ROM) in the framework of an industrially validated computational fluid dynamics software. This is of special importance in an industrial environment because in the online phase of the PGD ROM the description of the flow for a specific set of parameters is obtained simply via interpolation of the generalised solution, without the need of any extra solution step. On the one hand, the spatial problems arising from the PGD separation of the unknowns are treated using the classical solution strategies of OpenFOAM, namely the semi-implicit method for pressure linked equations (SIMPLE) algorithm. On the other hand, the parametric iteration is solved via a collocation approach. The resulting ROM is applied to several benchmark tests of laminar incompressible Navier-Stokes flows, in two and three dimensions, with different parameters affecting the flow features. Eventually, the capability of the proposed strategy to treat industrial problems is verified by applying the methodology to a parametrised flow control in a realistic geometry of interest for the automotive industry. △ Less

Submitted 12 June, 2019; originally announced June 2019.

Comments: 35 pages, 17 figures

MSC Class: 65N08; 76D05; 76D55; 62P30

Journal ref: Comput. Phys. Commun., 249:107013 (2020)

Denoising Gravitational Waves with Enhanced Deep Recurrent Denoising Auto-Encoders

Authors: Hongyu Shen, Daniel George, E. A. Huerta, Zhizhen Zhao

Abstract: Denoising of time domain data is a crucial task for many applications such as communication, translation, virtual assistants etc. For this task, a combination of a recurrent neural net (RNNs) with a Denoising Auto-Encoder (DAEs) has shown promising results. However, this combined model is challenged when operating with low signal-to-noise ratio (SNR) data embedded in non-Gaussian and non-stationar… ▽ More Denoising of time domain data is a crucial task for many applications such as communication, translation, virtual assistants etc. For this task, a combination of a recurrent neural net (RNNs) with a Denoising Auto-Encoder (DAEs) has shown promising results. However, this combined model is challenged when operating with low signal-to-noise ratio (SNR) data embedded in non-Gaussian and non-stationary noise. To address this issue, we design a novel model, referred to as 'Enhanced Deep Recurrent Denoising Auto-Encoder' (EDRDAE), that incorporates a signal amplifier layer, and applies curriculum learning by first denoising high SNR signals, before gradually decreasing the SNR until the signals become noise dominated. We showcase the performance of EDRDAE using time-series data that describes gravitational waves embedded in very noisy backgrounds. In addition, we show that EDRDAE can accurately denoise signals whose topology is significantly more complex than those used for training, demonstrating that our model generalizes to new classes of gravitational waves that are beyond the scope of established denoising algorithms. △ Less

Submitted 6 March, 2019; originally announced March 2019.

Comments: 5 pages, 11 figures and 3 tables, accepted to ICASSP 2019

MSC Class: 97R40 ACM Class: I.2

Journal ref: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)