-
Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression
Authors:
Masaki Morimoto,
Kai Fukami,
Romit Maulik,
Ricardo Vinuesa,
Koji Fukagata
Abstract:
We use Gaussian stochastic weight averaging (SWAG) to assess the model-form uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sample…
▽ More
We use Gaussian stochastic weight averaging (SWAG) to assess the model-form uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sampled weights, which can be used to obtain ensemble predictions. The average of such an ensemble can be regarded as the `mean estimation', whereas its standard deviation can be used to construct `confidence intervals', which enable us to perform uncertainty quantification (UQ) with regard to the training process of neural networks. We utilize representative neural-network-based function approximation tasks for the following cases: (i) a two-dimensional circular-cylinder wake; (ii) the DayMET dataset (maximum daily temperature in North America); (iii) a three-dimensional square-cylinder wake; and (iv) urban flow, to assess the generalizability of the present idea for a wide range of complex datasets. SWAG-based UQ can be applied regardless of the network architecture, and therefore, we demonstrate the applicability of the method for two types of neural networks: (i) global field reconstruction from sparse sensors by combining convolutional neural network (CNN) and multi-layer perceptron (MLP); and (ii) far-field state estimation from sectional data with two-dimensional CNN. We find that SWAG can obtain physically-interpretable confidence-interval estimates from the perspective of model-form uncertainty. This capability supports its use for a wide range of problems in science and engineering.
△ Less
Submitted 14 July, 2022; v1 submitted 16 September, 2021;
originally announced September 2021.
-
Inserting machine-learned virtual wall velocity for large-eddy simulation of turbulent channel flows
Authors:
Naoki Moriya,
Kai Fukami,
Yusuke Nabae,
Masaki Morimoto,
Taichi Nakamura,
Koji Fukagata
Abstract:
We propose a supervised-machine-learning-based wall model for coarse-grid wall-resolved large-eddy simulation (LES). Our consideration is made on LES of turbulent channel flows with a first grid point set relatively far from the wall ($\sim$ 10 wall units), while still resolving the near-wall region, to present a new path to save the computational cost. Convolutional neural network (CNN) is utiliz…
▽ More
We propose a supervised-machine-learning-based wall model for coarse-grid wall-resolved large-eddy simulation (LES). Our consideration is made on LES of turbulent channel flows with a first grid point set relatively far from the wall ($\sim$ 10 wall units), while still resolving the near-wall region, to present a new path to save the computational cost. Convolutional neural network (CNN) is utilized to estimate a virtual wall-surface velocity from $x-z$ sectional fields near the wall, whose training data are generated by a direct numerical simulation (DNS) at ${\rm Re}_τ=180$. The virtual wall-surface velocity is prepared with the extrapolation of the DNS data near the wall. This idea enables us to give a proper wall condition to correct a velocity gradient near the wall. The estimation ability of the model from near wall information is first investigated as a priori test. The estimated velocity fields by the present CNN model are in statistical agreement with the reference DNS data. The model trained in a priori test is then combined with the LES as a posteriori test. We find that the LES can successfully be augmented using the present model at both the friction Reynolds number ${\rm Re}_τ=180$ used for training and the unseen Reynolds number ${\rm Re}_τ=360$ even when the first grid point is located at 5 wall units off the wall. We also investigate the robustness of the present model for the choice of sub-grid scale model and the possibility of transfer learning in a local domain. The observations through the paper suggest that the present model is a promising tool for recovering the accuracy of LES with a coarse grid near the wall.
△ Less
Submitted 17 June, 2021;
originally announced June 2021.
-
Reconstructing three-dimensional bluff body wake from sectional flow fields with convolutional neural networks
Authors:
Mitsuaki Matsuo,
Kai Fukami,
Taichi Nakamura,
Masaki Morimoto,
Koji Fukagata
Abstract:
The recent development of high-performance computing enables us to generate spatio-temporal high-resolution data of nonlinear dynamical systems and to analyze them for a deeper understanding of their complex nature. This trend can be found in a wide range of science and engineering, which suggests that detailed investigations on efficient data handling in physical science must be required in the f…
▽ More
The recent development of high-performance computing enables us to generate spatio-temporal high-resolution data of nonlinear dynamical systems and to analyze them for a deeper understanding of their complex nature. This trend can be found in a wide range of science and engineering, which suggests that detailed investigations on efficient data handling in physical science must be required in the future. This study considers the use of convolutional neural networks (CNNs) to achieve efficient data storage and estimation of scientific big data derived from nonlinear dynamical systems. The CNN is used to reconstruct three-dimensional data from a few numbers of two-dimensional sections in a computationally friendly manner. The present model is a combination of two- and three-dimensional CNNs, which allows users to save only some of the two-dimensional sections to reconstruct the volumetric data. As examples, we consider a flow around a square cylinder at the diameter-based Reynolds number $Re_D = 300$. We demonstrate that volumetric fluid flow data can be reconstructed with the present method from as few as five sections. Furthermore, we propose a combination of the present CNN-based reconstruction with an adaptive sampling-based super-resolution analysis to augment the data compressibility. Our report can serve as a bridge toward practical data handling for not only fluid mechanics but also a broad range of physical sciences.
△ Less
Submitted 29 December, 2023; v1 submitted 16 March, 2021;
originally announced March 2021.
-
Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization
Authors:
Masaki Morimoto,
Kai Fukami,
Kai Zhang,
Aditya G. Nair,
Koji Fukagata
Abstract:
We focus on a convolutional neural network (CNN), which has recently been utilized for fluid flow analyses, from the perspective on the influence of various operations inside it by considering some canonical regression problems with fluid flow data. We consider two types of CNN-based fluid flow analyses; 1. CNN metamodeling and 2. CNN autoencoder. For the first type of CNN with additional scalar i…
▽ More
We focus on a convolutional neural network (CNN), which has recently been utilized for fluid flow analyses, from the perspective on the influence of various operations inside it by considering some canonical regression problems with fluid flow data. We consider two types of CNN-based fluid flow analyses; 1. CNN metamodeling and 2. CNN autoencoder. For the first type of CNN with additional scalar inputs, which is one of the common forms of CNN for fluid flow analysis, we investigate the influence of input placements in the CNN training pipeline. As an example, estimation of force coefficients of an inclined flat plate and two side-by-side cylinders in laminar flows is considered. We find that care should be taken for the placement of additional scalar inputs depending on the problem setting and the complexity of flows that users handle. We then discuss the influence of various parameters and operations on the CNN performance, with the utilization of autoencoder (AE). A two-dimensional turbulence is considered for the demonstration of AE. The results of AE highly rely on the decaying nature. Investigation on the influence of padding operation at a convolutional layer is also performed. The zero padding shows reasonable ability compared to other methods which account for the boundary conditions assumed in the numerical data. Moreover, the effect of the dimensional reduction/extension methods inside CNN is also examined. The CNN model is robust against the difference in dimension reduction operations, while it is sensitive to the dimensional extension methods. The findings of this paper will help us better design a CNN architecture for practical fluid flow analysis
△ Less
Submitted 22 July, 2021; v1 submitted 7 January, 2021;
originally announced January 2021.
-
Generalization techniques of neural networks for fluid flow estimation
Authors:
Masaki Morimoto,
Kai Fukami,
Kai Zhang,
Koji Fukagata
Abstract:
We demonstrate several techniques to encourage practical uses of neural networks for fluid flow estimation. In the present paper, three perspectives which are remaining challenges for applications of machine learning to fluid dynamics are considered: 1. interpretability of machine-learned results, 2. bulking out of training data, and 3. generalizability of neural networks. For the interpretability…
▽ More
We demonstrate several techniques to encourage practical uses of neural networks for fluid flow estimation. In the present paper, three perspectives which are remaining challenges for applications of machine learning to fluid dynamics are considered: 1. interpretability of machine-learned results, 2. bulking out of training data, and 3. generalizability of neural networks. For the interpretability, we first demonstrate two methods to observe the internal procedure of neural networks, i.e., visualization of hidden layers and application of gradient-weighted class activation mapping (Grad-CAM), applied to canonical fluid flow estimation problems -- $(1)$ drag coefficient estimation of a cylinder wake and $(2)$ velocity estimation from particle images. It is exemplified that both approaches can successfully tell us evidences of the great capability of machine learning-based estimations. We then utilize some techniques to bulk out training data for super-resolution analysis and temporal prediction for cylinder wake and NOAA sea surface temperature data to demonstrate that sufficient training of neural networks with limited amount of training data can be achieved for fluid flow problems. The generalizability of machine learning model is also discussed by accounting for the perspectives of inter/extrapolation of training data, considering super-resolution of wakes behind two parallel cylinders. We find that various flow patterns generated by complex interaction between two cylinders can be reconstructed well, even for the test configurations regarding the distance factor. The present paper can be a significant step toward practical uses of neural networks for both laminar and turbulent flow problems.
△ Less
Submitted 14 October, 2021; v1 submitted 24 November, 2020;
originally announced November 2020.
-
Model order reduction with neural networks: Application to laminar and turbulent flows
Authors:
Kai Fukami,
Kazuto Hasegawa,
Taichi Nakamura,
Masaki Morimoto,
Koji Fukagata
Abstract:
We investigate the capability of neural network-based model order reduction, i.e., autoencoder (AE), for fluid flows. As an example model, an AE which comprises of a convolutional neural network and multi-layer perceptrons is considered in this study. The AE model is assessed with four canonical fluid flows, namely: (1) two-dimensional cylinder wake, (2) its transient process, (3) NOAA sea surface…
▽ More
We investigate the capability of neural network-based model order reduction, i.e., autoencoder (AE), for fluid flows. As an example model, an AE which comprises of a convolutional neural network and multi-layer perceptrons is considered in this study. The AE model is assessed with four canonical fluid flows, namely: (1) two-dimensional cylinder wake, (2) its transient process, (3) NOAA sea surface temperature, and (4) $y-z$ sectional field of turbulent channel flow, in terms of a number of latent modes, a choice of nonlinear activation functions, and a number of weights contained in the AE model. We find that the AE models are sensitive against the choice of the aforementioned parameters depending on the target flows. Finally, we foresee the extensional applications and perspectives of machine learning based order reduction for numerical and experimental studies in fluid dynamics community.
△ Less
Submitted 7 September, 2021; v1 submitted 20 November, 2020;
originally announced November 2020.
-
Experimental velocity data estimation for imperfect particle images using machine learning
Authors:
Masaki Morimoto,
Kai Fukami,
Koji Fukagata
Abstract:
We propose a method using supervised machine learning to estimate velocity fields from particle images having missing regions due to experimental limitations. As a first example, a velocity field around a square cylinder at Reynolds number of ${\rm Re}_D=300$ is considered. To train machine learning models, we utilize artificial particle images (APIs) as the input data, which mimic the images of t…
▽ More
We propose a method using supervised machine learning to estimate velocity fields from particle images having missing regions due to experimental limitations. As a first example, a velocity field around a square cylinder at Reynolds number of ${\rm Re}_D=300$ is considered. To train machine learning models, we utilize artificial particle images (APIs) as the input data, which mimic the images of the particle image velocimetry (PIV). The output data are the velocity fields, and the correct answers for them are given by a direct numerical simulation (DNS). We examine two types of the input data: APIs without missing regions (i.e., full APIs) and APIs with missing regions (lacked APIs). The missing regions in the lacked APIs are assumed following the exact experimental situation in our wind tunnel setup. The velocity fields estimated from both full and lacked APIs are in great agreement with the reference DNS data in terms of various statistical assessments. We further apply these machine learned models trained with the DNS data to experimental particle images so that their applicability to the exact experimental situation can be investigated. The velocity fields estimated by the machine learned models contain approximately 40 folds denser data than that with the conventional cross-correlation method. This finding suggests that we may be able to obtain finer and hidden structures of the flow field which cannot be resolved with the conventional cross-correlation method. We also found that even the complex flow structures are hidden due to the alignment of two square cylinders, the machine learned model is able to estimate the field in the missing region reasonably well. The present results indicate a great potential of the proposed machine learning based method as a new data reconstruction method for PIV.
△ Less
Submitted 9 August, 2021; v1 submitted 2 May, 2020;
originally announced May 2020.
-
Consideration of Covariant Quantization of Electromagnetic Field
Authors:
Masahito Morimoto
Abstract:
We examine a covariant quantization of electromagnetic fields by using an operator derived from a constant scalar that can be called extended Lorentz gauge. The quantization can avoid an inconsistency between Lorentz gauge and a commutation relation, which can eliminate the need for introduction of physical state defined by a subsidiary condition and auxiliary field in Lagrangian density in Lorent…
▽ More
We examine a covariant quantization of electromagnetic fields by using an operator derived from a constant scalar that can be called extended Lorentz gauge. The quantization can avoid an inconsistency between Lorentz gauge and a commutation relation, which can eliminate the need for introduction of physical state defined by a subsidiary condition and auxiliary field in Lagrangian density in Lorentz gauge. By using this quantization and indefinite metric straightforwardly, all quantum phenomena can be provided without enigmatic and paradoxical "probability interpretation".
△ Less
Submitted 31 July, 2020; v1 submitted 4 June, 2018;
originally announced June 2018.
-
Unobservable Potentials to Explain a Quantum Eraser and a Delayed-Choice Experiment
Authors:
Masahito Morimoto
Abstract:
We present a new explanation for a quantum eraser. Mathematical description of the traditional explanation needs quantum-superposition states. However, the phenomenon can be explained without quantum-superposition states by introducing unobservable potentials which can be identified as an indefinite metric vector. In addition, a delayed choice experiment can also be explained by the interference b…
▽ More
We present a new explanation for a quantum eraser. Mathematical description of the traditional explanation needs quantum-superposition states. However, the phenomenon can be explained without quantum-superposition states by introducing unobservable potentials which can be identified as an indefinite metric vector. In addition, a delayed choice experiment can also be explained by the interference between the photons and unobservable potentials, which seems like an unreal long-range correlation beyond the causality.
△ Less
Submitted 20 November, 2017;
originally announced November 2017.
-
Unobservable Potentials to Explain Single Photon and Electron Interference
Authors:
Masahito Morimoto
Abstract:
We show single photon and electron interferences can be calculated without quantum-superposition states by using tensor form (covariant quantization). From the analysis results, the scalar potential which correspond to an indefinite metric vector forms an oscillatory field and causes the interferences. The results clarify the concept of quantum-superposition states is not required for the descript…
▽ More
We show single photon and electron interferences can be calculated without quantum-superposition states by using tensor form (covariant quantization). From the analysis results, the scalar potential which correspond to an indefinite metric vector forms an oscillatory field and causes the interferences. The results clarify the concept of quantum-superposition states is not required for the description of the interference, which leads to an improved understanding of the uncertainty principle and resolution of paradox of reduction of the wave packet, elimination of infinite zero-point energy and derivation of spontaneous symmetry breaking. The results conclude Quantum theory is a kind of deterministic physics without "probabilistic interpretation".
△ Less
Submitted 12 September, 2017;
originally announced September 2017.
