-
Constrained Bayesian optimization with merit functions
Authors:
J. Wang,
C. G. Petra,
J. L. Peterson
Abstract:
Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions…
▽ More
Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions, such as the penalty merit function, in acquisition functions, inspired by nonlinear optimization methods, e.g., sequential quadratic programming. Merit functions measure the potential progress of both the objective and constraint functions, thus increasing algorithmic efficiency and allowing infeasible initial samples. The acquisition functions with merit functions are relaxed to have closed forms, making its implementation readily available wherever Bayesian optimization is. We further propose a unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach. We demonstrate the effectiveness and efficiency of the proposed algorithms through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
A multifidelity Bayesian optimization method for inertial confinement fusion design
Authors:
J. Wang,
N. Chiang,
A. Gillette,
J. L. Peterson
Abstract:
Due to their cost, experiments for inertial confinement fusion (ICF) heavily rely on numerical simulations to guide design. As simulation technology progresses, so too can the fidelity of models used to plan for new experiments. However, these high-fidelity models are by themselves insufficient for optimal experimental design, because their computational cost remains too high to efficiently and ef…
▽ More
Due to their cost, experiments for inertial confinement fusion (ICF) heavily rely on numerical simulations to guide design. As simulation technology progresses, so too can the fidelity of models used to plan for new experiments. However, these high-fidelity models are by themselves insufficient for optimal experimental design, because their computational cost remains too high to efficiently and effectively explore the numerous parameters required to describe a typical experiment. Traditionally, ICF design has relied on low-fidelity modeling to initially identify potentially interesting design regions, which are then subsequently explored via selected high-fidelity modeling. In this paper, we demonstrate that this two-step approach can be insufficient: even for simple design problems, a two-step optimization strategy can lead high-fidelity searching towards incorrect regions and consequently waste computational resources on parameter regimes far away from the true optimal solution. We reveal that a primary cause of this behavior in ICF design problems is the presence of low-fidelity optima in distinct regions of the parameter space from high-fidelity optima. To address this issue, we propose an iterative multifidelity Bayesian optimization method based on Gaussian Process Regression that leverages both low- and high-fidelity modelings. We demonstrate, using both two- and eight-dimensional ICF test problems, that our algorithm can effectively utilize low-fidelity modeling for exploration, while automatically refining promising designs with high-fidelity models. This approach proves to be more efficient than relying solely on high-fidelity modeling for optimization.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
General Relativistic Implicit Monte Carlo Radiation-Hydrodynamics
Authors:
Nathaniel Roth,
Peter Anninos,
Peter B. Robinson,
J. Luc Peterson,
Brooke Polak,
Tymothy K. Mangan,
Kyle Beyer
Abstract:
We report on a new capability added to our general relativistic radiation-magnetohydrodynamics code, Cosmos++: an implicit Monte Carlo (IMC) treatment for radiation transport. The method is based on a Fleck-type implicit discretization of the radiation-hydrodynamics equations, but generalized for both Newtonian and relativistic regimes. A multiple reference frame approach is used to geodesically t…
▽ More
We report on a new capability added to our general relativistic radiation-magnetohydrodynamics code, Cosmos++: an implicit Monte Carlo (IMC) treatment for radiation transport. The method is based on a Fleck-type implicit discretization of the radiation-hydrodynamics equations, but generalized for both Newtonian and relativistic regimes. A multiple reference frame approach is used to geodesically transport photon packets (and solve the hydrodynamics equations) in the coordinate frame, while radiation-matter interactions are handled either in the fluid or electron frames then communicated via Lorentz boosts and orthonormal tetrad bases attached to the fluid. We describe a method for constructing estimators of radiation moments using path-weighting that generalizes to arbitrary coordinate systems in flat or curved spacetime. Absorption, emission, scattering, and relativistic Comptonization are among the matter interactions considered in this report. We discuss our formulations and numerical methods, and validate our models against a suite of radiation and coupled radiation-hydrodynamics test problems in both flat and curved spacetimes.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
2022 Review of Data-Driven Plasma Science
Authors:
Rushil Anirudh,
Rick Archibald,
M. Salman Asif,
Markus M. Becker,
Sadruddin Benkadda,
Peer-Timo Bremer,
Rick H. S. Budé,
C. S. Chang,
Lei Chen,
R. M. Churchill,
Jonathan Citrin,
Jim A Gaffney,
Ana Gainaru,
Walter Gekelman,
Tom Gibbs,
Satoshi Hamaguchi,
Christian Hill,
Kelli Humbird,
Sören Jalas,
Satoru Kawaguchi,
Gon-Ho Kim,
Manuel Kirchen,
Scott Klasky,
John L. Kline,
Karl Krushelnick
, et al. (38 additional authors not shown)
Abstract:
Data science and technology offer transformative tools and methods to science. This review article highlights latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS). A large amount of data and machine learning algorithms go hand in hand. Most plasma data, whether experimental, observational or computational, are generated or collected by machines today.…
▽ More
Data science and technology offer transformative tools and methods to science. This review article highlights latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS). A large amount of data and machine learning algorithms go hand in hand. Most plasma data, whether experimental, observational or computational, are generated or collected by machines today. It is now becoming impractical for humans to analyze all the data manually. Therefore, it is imperative to train machines to analyze and interpret (eventually) such data as intelligently as humans but far more efficiently in quantity. Despite the recent impressive progress in applications of data science to plasma science and technology, the emerging field of DDPS is still in its infancy. Fueled by some of the most challenging problems such as fusion energy, plasma processing of materials, and fundamental understanding of the universe through observable plasma phenomena, it is expected that DDPS continues to benefit significantly from the interdisciplinary marriage between plasma science and data science into the foreseeable future.
△ Less
Submitted 31 May, 2022;
originally announced May 2022.
-
Transfer learning driven design optimization for inertial confinement fusion
Authors:
K. D. Humbird,
J. L. Peterson
Abstract:
Transfer learning is a promising approach to creating predictive models that incorporate simulation and experimental data into a common framework. In this technique, a neural network is first trained on a large database of simulations, then partially retrained on sparse sets of experimental data to adjust predictions to be more consistent with reality. Previously, this technique has been used to c…
▽ More
Transfer learning is a promising approach to creating predictive models that incorporate simulation and experimental data into a common framework. In this technique, a neural network is first trained on a large database of simulations, then partially retrained on sparse sets of experimental data to adjust predictions to be more consistent with reality. Previously, this technique has been used to create predictive models of Omega and NIF inertial confinement fusion (ICF) experiments that are more accurate than simulations alone. In this work, we conduct a transfer learning driven hypothetical ICF campaign in which the goal is to maximize experimental neutron yield via Bayesian optimization. The transfer learning model achieves yields within 5% of the maximum achievable yield in a modest-sized design space in fewer than 20 experiments. Furthermore, we demonstrate that this method is more efficient at optimizing designs than traditional model calibration techniques commonly employed in ICF design. Such an approach to ICF design could enable robust optimization of experimental performance under uncertainty.
△ Less
Submitted 26 May, 2022;
originally announced May 2022.
-
The data-driven future of high energy density physics
Authors:
Peter W. Hatfield,
Jim A. Gaffney,
Gemma J. Anderson,
Suzanne Ali,
Luca Antonelli,
Suzan Başeğmez du Pree,
Jonathan Citrin,
Marta Fajardo,
Patrick Knapp,
Brendan Kettle,
Bogdan Kustowski,
Michael J. MacDonald,
Derek Mariscal,
Madison E. Martin,
Taisuke Nagayama,
Charlotte A. J. Palmer,
J. Luc Peterson,
Steven Rose,
J J Ruby,
Carl Shneider,
Matt J. V. Streeter,
Will Trickey,
Ben Williams
Abstract:
The study of plasma physics under conditions of extreme temperatures, densities and electromagnetic field strengths is significant for our understanding of astrophysics, nuclear fusion and fundamental physics. These extreme physical systems are strongly non-linear and very difficult to understand theoretically or optimize experimentally. Here, we argue that machine learning models and data-driven…
▽ More
The study of plasma physics under conditions of extreme temperatures, densities and electromagnetic field strengths is significant for our understanding of astrophysics, nuclear fusion and fundamental physics. These extreme physical systems are strongly non-linear and very difficult to understand theoretically or optimize experimentally. Here, we argue that machine learning models and data-driven methods are in the process of reshaping our exploration of these extreme systems that have hitherto proven far too non-linear for human researchers. From a fundamental perspective, our understanding can be helped by the way in which machine learning models can rapidly discover complex interactions in large data sets. From a practical point of view, the newest generation of extreme physics facilities can perform experiments multiple times a second (as opposed to ~daily), moving away from human-based control towards automatic control based on real-time interpretation of diagnostic data and updates of the physics model. To make the most of these emerging opportunities, we advance proposals for the community in terms of research design, training, best practices, and support for synthetic diagnostics and data analysis.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
Experiments conducted in the burning plasma regime with inertial fusion implosions
Authors:
J. S. Ross,
J. E. Ralph,
A. B. Zylstra,
A. L. Kritcher,
H. F. Robey,
C. V. Young,
O. A. Hurricane,
D. A. Callahan,
K. L. Baker,
D. T. Casey,
T. Doeppner,
L. Divol,
M. Hohenberger,
S. Le Pape,
A. Pak,
P. K. Patel,
R. Tommasini,
S. J. Ali,
P. A. Amendt,
L. J. Atherton,
B. Bachmann,
D. Bailey,
L. R. Benedetti,
L. Berzak Hopkins,
R. Betti
, et al. (127 additional authors not shown)
Abstract:
An experimental program is currently underway at the National Ignition Facility (NIF) to compress deuterium and tritium (DT) fuel to densities and temperatures sufficient to achieve fusion and energy gain. The primary approach being investigated is indirect drive inertial confinement fusion (ICF), where a high-Z radiation cavity (a hohlraum) is heated by lasers, converting the incident energy into…
▽ More
An experimental program is currently underway at the National Ignition Facility (NIF) to compress deuterium and tritium (DT) fuel to densities and temperatures sufficient to achieve fusion and energy gain. The primary approach being investigated is indirect drive inertial confinement fusion (ICF), where a high-Z radiation cavity (a hohlraum) is heated by lasers, converting the incident energy into x-ray radiation which in turn drives the DT fuel filled capsule causing it to implode. Previous experiments reported DT fuel gain exceeding unity [O.A. Hurricane et al., Nature 506, 343 (2014)] and then exceeding the kinetic energy of the imploding fuel [S. Le Pape et al., Phys. Rev. Lett. 120, 245003 (2018)]. We report on recent experiments that have achieved record fusion neutron yields on NIF, greater than 100 kJ with momentary fusion powers exceeding 1PW, and have for the first time entered the burning plasma regime where fusion alpha-heating of the fuel exceeds the energy delivered to the fuel via compression. This was accomplished by increasing the size of the high-density carbon (HDC) capsule, increasing energy coupling, while controlling symmetry and implosion design parameters. Two tactics were successful in controlling the radiation flux symmetry and therefore the implosion symmetry: transferring energy between laser cones via plasma waves, and changing the shape of the hohlraum. In conducting these experiments, we controlled for known sources of degradation. Herein we show how these experiments were performed to produce record performance, and demonstrate the data fidelity leading us to conclude that these shots have entered the burning plasma regime.
△ Less
Submitted 8 November, 2021;
originally announced November 2021.
-
A Community Roadmap for Scientific Workflows Research and Development
Authors:
Rafael Ferreira da Silva,
Henri Casanova,
Kyle Chard,
Ilkay Altintas,
Rosa M Badia,
Bartosz Balis,
Tainã Coleman,
Frederik Coppens,
Frank Di Natale,
Bjoern Enders,
Thomas Fahringer,
Rosa Filgueira,
Grigori Fursin,
Daniel Garijo,
Carole Goble,
Dorran Howell,
Shantenu Jha,
Daniel S. Katz,
Daniel Laney,
Ulf Leser,
Maciej Malawski,
Kshitij Mehta,
Loïc Pottier,
Jonathan Ozik,
J. Luc Peterson
, et al. (4 additional authors not shown)
Abstract:
The landscape of workflow systems for scientific applications is notoriously convoluted with hundreds of seemingly equivalent workflow systems, many isolated research claims, and a steep learning curve. To address some of these challenges and lay the groundwork for transforming workflows research and development, the WorkflowsRI and ExaWorks projects partnered to bring the international workflows…
▽ More
The landscape of workflow systems for scientific applications is notoriously convoluted with hundreds of seemingly equivalent workflow systems, many isolated research claims, and a steep learning curve. To address some of these challenges and lay the groundwork for transforming workflows research and development, the WorkflowsRI and ExaWorks projects partnered to bring the international workflows community together. This paper reports on discussions and findings from two virtual "Workflows Community Summits" (January and April, 2021). The overarching goals of these workshops were to develop a view of the state of the art, identify crucial research challenges in the workflows community, articulate a vision for potential community efforts, and discuss technical approaches for realizing this vision. To this end, participants identified six broad themes: FAIR computational workflows; AI workflows; exascale challenges; APIs, interoperability, reuse, and standards; training and education; and building a workflows community. We summarize discussions and recommendations for each of these themes.
△ Less
Submitted 8 October, 2021; v1 submitted 5 October, 2021;
originally announced October 2021.
-
Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data
Authors:
K. D. Humbird,
J. L. Peterson,
J. Salmonson,
B. K. Spears
Abstract:
The design space for inertial confinement fusion (ICF) experiments is vast and experiments are extremely expensive. Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. However, ICF multiphysics codes must make simplifying assumptions, and thus deviate from experimental measurements for complex implosions. For more effective design a…
▽ More
The design space for inertial confinement fusion (ICF) experiments is vast and experiments are extremely expensive. Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. However, ICF multiphysics codes must make simplifying assumptions, and thus deviate from experimental measurements for complex implosions. For more effective design and investigation, simulations require input from past experimental data to better predict future performance. In this work, we describe a cognitive simulation method for combining simulation and experimental data into a common, predictive model. This method leverages a machine learning technique called transfer learning, the process of taking a model trained to solve one task, and partially retraining it on a sparse dataset to solve a different, but related task. In the context of ICF design, neural network models trained on large simulation databases and partially retrained on experimental data, producing models that are far more accurate than simulations alone. We demonstrate improved model performance for a range of ICF experiments at the National Ignition Facility, and predict the outcome of recent experiments with less than ten percent error for several key observables. We discuss how the methods might be used to carry out a data-driven experimental campaign to optimize performance, illustrating the key product -- models that become increasingly accurate as data is acquired.
△ Less
Submitted 18 March, 2021;
originally announced March 2021.
-
Enabling Machine Learning-Ready HPC Ensembles with Merlin
Authors:
J. Luc Peterson,
Ben Bay,
Joe Koning,
Peter Robinson,
Jessica Semler,
Jeremy White,
Rushil Anirudh,
Kevin Athey,
Peer-Timo Bremer,
Francesco Di Natale,
David Fox,
Jim A. Gaffney,
Sam A. Jacobs,
Bhavya Kailkhura,
Bogdan Kustowski,
Steven Langer,
Brian Spears,
Jayaraman Thiagarajan,
Brian Van Essen,
Jae-Seung Yeom
Abstract:
With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computin…
▽ More
With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. In addition to its design, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.
△ Less
Submitted 1 July, 2021; v1 submitted 5 December, 2019;
originally announced December 2019.
-
Transfer learning to model inertial confinement fusion experiments
Authors:
K. D. Humbird,
J. L. Peterson,
R. G. McClarren
Abstract:
Inertial confinement fusion (ICF) experiments are designed using computer simulations that are approximations of reality, and therefore must be calibrated to accurately predict experimental observations. In this work, we propose a novel nonlinear technique for calibrating from simulations to experiments, or from low fidelity simulations to high fidelity simulations, via "transfer learning". Transf…
▽ More
Inertial confinement fusion (ICF) experiments are designed using computer simulations that are approximations of reality, and therefore must be calibrated to accurately predict experimental observations. In this work, we propose a novel nonlinear technique for calibrating from simulations to experiments, or from low fidelity simulations to high fidelity simulations, via "transfer learning". Transfer learning is a commonly used technique in the machine learning community, in which models trained on one task are partially retrained to solve a separate, but related task, for which there is a limited quantity of data. We introduce the idea of hierarchical transfer learning, in which neural networks trained on low fidelity models are calibrated to high fidelity models, then to experimental data. This technique essentially bootstraps the calibration process, enabling the creation of models which predict high fidelity simulations or experiments with minimal computational cost. We apply this technique to a database of ICF simulations and experiments carried out at the Omega laser facility. Transfer learning with deep neural networks enables the creation of models that are more predictive of Omega experiments than simulations alone. The calibrated models accurately predict future Omega experiments, and are used to search for new, optimal implosion designs.
△ Less
Submitted 14 December, 2018;
originally announced December 2018.
-
Predicting the time-evolution of multi-physics systems with sequence-to-sequence models
Authors:
K. D. Humbird,
J. L. Peterson,
R. G. McClarren
Abstract:
In this work, sequence-to-sequence (seq2seq) models, originally developed for language translation, are used to predict the temporal evolution of complex, multi-physics computer simulations. The predictive performance of seq2seq models is compared to state transition models for datasets generated with multi-physics codes with varying levels of complexity - from simple 1D diffusion calculations to…
▽ More
In this work, sequence-to-sequence (seq2seq) models, originally developed for language translation, are used to predict the temporal evolution of complex, multi-physics computer simulations. The predictive performance of seq2seq models is compared to state transition models for datasets generated with multi-physics codes with varying levels of complexity - from simple 1D diffusion calculations to simulations of inertial confinement fusion implosions. Seq2seq models demonstrate the ability to accurately emulate complex systems, enabling the rapid estimation of the evolution of quantities of interest in computationally expensive simulations.
△ Less
Submitted 14 November, 2018;
originally announced November 2018.
-
Deep neural network initialization with decision trees
Authors:
K. D. Humbird,
J. L. Peterson,
R. G. McClarren
Abstract:
In this work a novel, automated process for constructing and initializing deep feed-forward neural networks based on decision trees is presented. The proposed algorithm maps a collection of decision trees trained on the data into a collection of initialized neural networks, with the structures of the networks determined by the structures of the trees. The tree-informed initialization acts as a war…
▽ More
In this work a novel, automated process for constructing and initializing deep feed-forward neural networks based on decision trees is presented. The proposed algorithm maps a collection of decision trees trained on the data into a collection of initialized neural networks, with the structures of the networks determined by the structures of the trees. The tree-informed initialization acts as a warm-start to the neural network training process, resulting in efficiently trained, accurate networks. These models, referred to as "deep jointly-informed neural networks" (DJINN), demonstrate high predictive performance for a variety of regression and classification datasets, and display comparable performance to Bayesian hyper-parameter optimization at a lower computational cost. By combining the user-friendly features of decision tree models with the flexibility and scalability of deep neural networks, DJINN is an attractive algorithm for training predictive models on a wide range of complex datasets.
△ Less
Submitted 2 July, 2018; v1 submitted 3 July, 2017;
originally announced July 2017.
-
An Enhanced Nonlinear Critical Gradient for Electron Turbulent Transport due to Reversed Magnetic Shear
Authors:
J. L. Peterson,
G. W. Hammett,
D. R. Mikkelsen,
H. Y. Yuh,
J. Candy,
W. Guttenfelder,
S. M. Kaye,
B. LeBlanc
Abstract:
The first nonlinear gyrokinetic simulations of electron internal transport barriers (e-ITBs) in the National Spherical Torus Experiment show that reversed magnetic shear can suppress thermal transport by increasing the nonlinear critical gradient for electron-temperature-gradient-driven turbulence to three times its linear critical value. An interesting feature of this turbulence is nonlinearly dr…
▽ More
The first nonlinear gyrokinetic simulations of electron internal transport barriers (e-ITBs) in the National Spherical Torus Experiment show that reversed magnetic shear can suppress thermal transport by increasing the nonlinear critical gradient for electron-temperature-gradient-driven turbulence to three times its linear critical value. An interesting feature of this turbulence is nonlinearly driven off-midplane radial streamers. This work reinforces the experimental observation that magnetic shear is likely an effective way of triggering and sustaining e-ITBs in magnetic fusion devices.
△ Less
Submitted 11 May, 2011;
originally announced May 2011.
