Differentiable modeling to unify machine learning and physical models and advance Geosciences
Authors:
Chaopeng Shen,
Alison P. Appling,
Pierre Gentine,
Toshiyuki Bandai,
Hoshin Gupta,
Alexandre Tartakovsky,
Marco Baity-Jesi,
Fabrizio Fenicia,
Daniel Kifer,
Li Li,
Xiaofeng Liu,
Wei Ren,
Yi Zheng,
Ciaran J. Harman,
Martyn Clark,
Matthew Farthing,
Dapeng Feng,
Praveen Kumar,
Doaa Aboelyazeed,
Farshid Rahmani,
Hylke E. Beck,
Tadd Bindas,
Dipankar Dwivedi,
Kuai Fang,
Marvin Höge
, et al. (5 additional authors not shown)
Abstract:
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage lar…
▽ More
Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.
△ Less
Submitted 26 December, 2023; v1 submitted 10 January, 2023;
originally announced January 2023.
Physics-Guided Recurrent Graph Networks for Predicting Flow and Temperature in River Networks
Authors:
Xiaowei Jia,
Jacob Zwart,
Jeffrey Sadler,
Alison Appling,
Samantha Oliver,
Steven Markstrom,
Jared Willard,
Shaoming Xu,
Michael Steinbach,
Jordan Read,
Vipin Kumar
Abstract:
This paper proposes a physics-guided machine learning approach that combines advanced machine learning models and physics-based models to improve the prediction of water flow and temperature in river networks. We first build a recurrent graph network model to capture the interactions among multiple segments in the river network. Then we present a pre-training technique which transfers knowledge fr…
▽ More
This paper proposes a physics-guided machine learning approach that combines advanced machine learning models and physics-based models to improve the prediction of water flow and temperature in river networks. We first build a recurrent graph network model to capture the interactions among multiple segments in the river network. Then we present a pre-training technique which transfers knowledge from physics-based models to initialize the machine learning model and learn the physics of streamflow and thermodynamics. We also propose a new loss function that balances the performance over different river segments. We demonstrate the effectiveness of the proposed method in predicting temperature and streamflow in a subset of the Delaware River Basin. In particular, we show that the proposed method brings a 33\%/14\% improvement over the state-of-the-art physics-based model and 24\%/14\% over traditional machine learning models (e.g., Long-Short Term Memory Neural Network) in temperature/streamflow prediction using very sparse (0.1\%) observation data for training. The proposed method has also been shown to produce better performance when generalized to different seasons or river segments with different streamflow ranges.
△ Less
Submitted 8 December, 2020; v1 submitted 26 September, 2020;
originally announced September 2020.
Physics-Guided Architecture (PGA) of Neural Networks for Quantifying Uncertainty in Lake Temperature Modeling
Authors:
Arka Daw,
R. Quinn Thomas,
Cayelan C. Carey,
Jordan S. Read,
Alison P. Appling,
Anuj Karpatne
Abstract:
To simultaneously address the rising need of expressing uncertainties in deep learning models along with producing model outputs which are consistent with the known scientific knowledge, we propose a novel physics-guided architecture (PGA) of neural networks in the context of lake temperature modeling where the physical constraints are hard coded in the neural network architecture. This allows us…
▽ More
To simultaneously address the rising need of expressing uncertainties in deep learning models along with producing model outputs which are consistent with the known scientific knowledge, we propose a novel physics-guided architecture (PGA) of neural networks in the context of lake temperature modeling where the physical constraints are hard coded in the neural network architecture. This allows us to integrate such models with state of the art uncertainty estimation approaches such as Monte Carlo (MC) Dropout without sacrificing the physical consistency of our results. We demonstrate the effectiveness of our approach in ensuring better generalizability as well as physical consistency in MC estimates over data collected from Lake Mendota in Wisconsin and Falling Creek Reservoir in Virginia, even with limited training data. We further show that our MC estimates correctly match the distribution of ground-truth observations, thus making the PGA paradigm amenable to physically grounded uncertainty quantification.
△ Less
Submitted 6 November, 2019;
originally announced November 2019.
