-
Matrix-Free Finite Volume Kernels on a Dataflow Architecture
Authors:
Ryuichi Sai,
Francois P. Hamon,
John Mellor-Crummey,
Mauricio Araya-Polo
Abstract:
Fast and accurate numerical simulations are crucial for designing large-scale geological carbon storage projects ensuring safe long-term CO2 containment as a climate change mitigation strategy. These simulations involve solving numerous large and complex linear systems arising from the implicit Finite Volume (FV) discretization of PDEs governing subsurface fluid flow. Compounded with highly detail…
▽ More
Fast and accurate numerical simulations are crucial for designing large-scale geological carbon storage projects ensuring safe long-term CO2 containment as a climate change mitigation strategy. These simulations involve solving numerous large and complex linear systems arising from the implicit Finite Volume (FV) discretization of PDEs governing subsurface fluid flow. Compounded with highly detailed geomodels, solving linear systems is computationally and memory expensive, and accounts for the majority of the simulation time. Modern memory hierarchies are insufficient to meet the latency and bandwidth needs of large-scale numerical simulations. Therefore, exploring algorithms that can leverage alternative and balanced paradigms, such as dataflow and in-memory computing is crucial. This work introduces a matrix-free algorithm to solve FV-based linear systems using a dataflow architecture to significantly minimize memory latency and bandwidth bottlenecks. Our implementation achieves two orders of magnitude speedup compared to a GPGPU-based reference implementation, and up to 1.2 PFlops on a single dataflow device.
△ Less
Submitted 6 August, 2024;
originally announced August 2024.
-
Constrained Pressure-Temperature Residual (CPTR) Preconditioner Performance for Large-Scale Thermal CO2 Injection Simulation
Authors:
Matthias A. Cremon,
Jacques Franc,
Francois P. Hamon
Abstract:
This work studies the performance of a novel preconditioner, designed for thermal reservoir simulation cases and recently introduced in Roy et al. (2020) and Cremon et al. (2020), on large-scale thermal CO2 injection cases. For Carbon Capture and Sequestration (CCS) projects, injecting CO2 under supercritical conditions is typically tens of degrees colder than the reservoir temperature. Thermal ef…
▽ More
This work studies the performance of a novel preconditioner, designed for thermal reservoir simulation cases and recently introduced in Roy et al. (2020) and Cremon et al. (2020), on large-scale thermal CO2 injection cases. For Carbon Capture and Sequestration (CCS) projects, injecting CO2 under supercritical conditions is typically tens of degrees colder than the reservoir temperature. Thermal effects can have a significant impact on the simulation results, but they also add many challenges for the solvers. More specifically, the usual combination of an iterative linear solver (such as GMRES) and the Constrained Pressure Residual (CPR) physics-based block-preconditioner is known to perform rather poorly or fail to converge when thermal effects play a significant role. The Constrained Pressure-Temperature Residual (CPTR) preconditioner retains the 2x2 block structure (elliptic/hyperbolic) of CPR but includes the temperature in the elliptic subsystem. The elliptic subsystem is now formed by two equations, and is dealt with by the system-solver of BoomerAMG (from the HYPRE library). Then a global smoother, ILU(0), is applied to the full system to handle the local, hyperbolic temperature fronts. We implemented CPTR in the multi-physics solver GEOS and present results on various large-scale thermal CCS simulation cases, including both Cartesian and fully unstructured meshes, up to tens of millions of degrees of freedom. The CPTR preconditioner severely reduces the number of GMRES iterations and the runtime, with cases timing out in 24h with CPR now requiring a few hours with CPTR. We present strong scaling results using hundreds of CPU cores for multiple cases, and show close to linear scaling. CPTR is also virtually insensitive to the thermal Peclet number (which compares advection and diffusion effects) and is suitable to any thermal regime.
△ Less
Submitted 13 April, 2024; v1 submitted 22 August, 2023;
originally announced August 2023.
-
Surrogate Model for Geological CO2 Storage and Its Use in Hierarchical MCMC History Matching
Authors:
Yifu Han,
Francois P. Hamon,
Su Jiang,
Louis J. Durlofsky
Abstract:
Deep-learning-based surrogate models show great promise for use in geological carbon storage operations. In this work we target an important application - the history matching of storage systems characterized by a high degree of (prior) geological uncertainty. Toward this goal, we extend the recently introduced recurrent R-U-Net surrogate model to treat geomodel realizations drawn from a wide rang…
▽ More
Deep-learning-based surrogate models show great promise for use in geological carbon storage operations. In this work we target an important application - the history matching of storage systems characterized by a high degree of (prior) geological uncertainty. Toward this goal, we extend the recently introduced recurrent R-U-Net surrogate model to treat geomodel realizations drawn from a wide range of geological scenarios. These scenarios are defined by a set of metaparameters, which include the horizontal correlation length, mean and standard deviation of log-permeability, permeability anisotropy ratio, and constants in the porosity-permeability relationship. An infinite number of realizations can be generated for each set of metaparameters, so the range of prior uncertainty is large. The surrogate model is trained with flow simulation results, generated using the open-source simulator GEOS, for 2000 random realizations. The flow problems involve four wells, each injecting 1 Mt CO2/year, for 30 years. The trained surrogate model is shown to provide accurate predictions for new realizations over the full range of geological scenarios, with median relative error of 1.3% in pressure and 4.5% in saturation. The surrogate model is incorporated into a hierarchical Markov chain Monte Carlo history matching workflow, where the goal is to generate history matched geomodel realizations and posterior estimates of the metaparameters. We show that, using observed data from monitoring wells in synthetic `true' models, geological uncertainty is reduced substantially. This leads to posterior 3D pressure and saturation fields that display much closer agreement with the true-model responses than do prior predictions.
△ Less
Submitted 2 February, 2024; v1 submitted 11 August, 2023;
originally announced August 2023.
-
Learning CO$_2$ plume migration in faulted reservoirs with Graph Neural Networks
Authors:
Xin Ju,
François P. Hamon,
Gege Wen,
Rayan Kanfar,
Mauricio Araya-Polo,
Hamdi A. Tchelepi
Abstract:
Deep-learning-based surrogate models provide an efficient complement to numerical simulations for subsurface flow problems such as CO$_2$ geological storage. Accurately capturing the impact of faults on CO$_2$ plume migration remains a challenge for many existing deep learning surrogate models based on Convolutional Neural Networks (CNNs) or Neural Operators. We address this challenge with a graph…
▽ More
Deep-learning-based surrogate models provide an efficient complement to numerical simulations for subsurface flow problems such as CO$_2$ geological storage. Accurately capturing the impact of faults on CO$_2$ plume migration remains a challenge for many existing deep learning surrogate models based on Convolutional Neural Networks (CNNs) or Neural Operators. We address this challenge with a graph-based neural model leveraging recent developments in the field of Graph Neural Networks (GNNs). Our model combines graph-based convolution Long-Short-Term-Memory (GConvLSTM) with a one-step GNN model, MeshGraphNet (MGN), to operate on complex unstructured meshes and limit temporal error accumulation. We demonstrate that our approach can accurately predict the temporal evolution of gas saturation and pore pressure in a synthetic reservoir with impermeable faults. Our results exhibit a better accuracy and a reduced temporal error accumulation compared to the standard MGN model. We also show the excellent generalizability of our algorithm to mesh configurations, boundary conditions, and heterogeneous permeability fields not included in the training set. This work highlights the potential of GNN-based methods to accurately and rapidly model subsurface flow with complex faults and fractures.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
Smooth Implicit Hybrid Upwinding for Compositional Multiphase Flow in Porous Media
Authors:
Sebastian B. M. Bosma,
Francois P. Hamon,
Brad T. Mallison,
Hamdi A. Tchelepi
Abstract:
In subsurface multiphase flow simulations, poor nonlinear solver performance is a significant runtime sink. The system of fully implicit mass balance equations is highly nonlinear and often difficult to solve for the nonlinear solver, generally Newton(-Raphson). Strong nonlinearities can cause Newton iterations to converge very slowly. This frequently results in time step cuts, leading to computat…
▽ More
In subsurface multiphase flow simulations, poor nonlinear solver performance is a significant runtime sink. The system of fully implicit mass balance equations is highly nonlinear and often difficult to solve for the nonlinear solver, generally Newton(-Raphson). Strong nonlinearities can cause Newton iterations to converge very slowly. This frequently results in time step cuts, leading to computationally expensive simulations. Much literature has looked into how to improve the nonlinear solver through enhancements or safeguarding updates. In this work, we take a different approach; we aim to improve convergence with a smoother finite volume discretization scheme which is more suitable for the Newton solver.
Building on recent work, we propose a novel total velocity hybrid upwinding scheme with weighted average flow mobilities (WA-HU TV) that is unconditionally monotone and extends to compositional multiphase simulations. Analyzing the solution space of a one-cell problem, we demonstrate the improved properties of the scheme and explain how it leverages the advantages of both phase potential upwinding and arithmetic averaging. This results in a flow subproblem that is smooth with respect to changes in the sign of phase fluxes, and is well-behaved when phase velocities are large or when co-current viscous forces dominate. Additionally, we propose a WA-HU scheme with a total mass (WA-HU TM) formulation that includes phase densities in the weighted averaging.
The proposed WA-HU TV consistently outperforms existing schemes, yielding benefits from 5\% to over 50\% reduction in nonlinear iterations. The WA-HU TM scheme also shows promising results; in some cases leading to even more efficiency. However, WA-HU TM can occasionally also lead to convergence issues. Overall, based on the current results, we recommend the adoption of the WA-HU TV scheme as it is highly efficient and robust.
△ Less
Submitted 6 June, 2021;
originally announced June 2021.