-
Matrix-by-matrix multiplication algorithm with $O(N^2log_2N)$ computational complexity for variable precision arithmetic
Authors:
Maciej Paszyński
Abstract:
We show that assuming the availability of the processor with variable precision arithmetic, we can compute matrix-by-matrix multiplications in $O(N^2log_2N)$ computational complexity. We replace the standard matrix-by-matrix multiplications algorithm…
▽ More
We show that assuming the availability of the processor with variable precision arithmetic, we can compute matrix-by-matrix multiplications in $O(N^2log_2N)$ computational complexity. We replace the standard matrix-by-matrix multiplications algorithm $\begin{bmatrix}A_{11}&A_{12}\\A_{21}&A_{22}\end{bmatrix}\begin{bmatrix}B_{11}&B_{12}\\B_{21}&B_{22}\end{bmatrix}=\begin{bmatrix}A_{11}B_{11}+A_{12}B_{21}&A_{11}B_{12}+A_{12}B_{22}\\A_{21}B_{11}+A_{22}B_{21}&A_{21}B_{12}+A_{22}B_{22}\end{bmatrix}$ by $\begin{bmatrix}A_{11}&A_{12}\\A_{21}&A_{22}\end{bmatrix}\begin{bmatrix}B_{11}&B_{12}\\B_{21}&B_{22}\end{bmatrix}=\Bigl\lfloor\begin{bmatrix} (A_{11}+εA_{12})(B_{11}+1/εB_{21})&(A_{11}+εA_{12})(B_{12}+1/εB_{22})\\(A_{21}+εA_{22})(B_{11}+1/εB_{21})&(A_{21}+εA_{22})(B_{12}+1/εB_{22})\end{bmatrix}\Bigr\rfloor \mod \frac{1}ε$. The resulting computational complexity for $N\times N$ matrices can be estimated from recursive equation $T(N)=4(N/2)^2$ (multiplication of a matrix by number)+$4(N/2)^2$ (additions of matrices)+$2N^2$ (floor and modulo)+$4T(N/2)$ (recursive calls) as $O(N^2log_2N)$. The novelty of the method lies in the observation, somehow ignored by other matrix-by-matrix multiplication algorithms, that we can multiply matrix entries by non-integer numbers to improve computational complexity. In other words, while having a processor that can compute multiplications, additions, modulo and floor operations with variable precision arithmetic in $O(1)$, we can obtain a matrix-by-matrix multiplication algorithm with $O(N^2log_2N)$ computational complexity. We also present a MATLAB code using VPA variable precision arithmetic emulator that can multiply matrices of size $N\times N$ using $(4log_2N+1)N^2$ variable precision arithmetic operations. This emulator uses $O(N)$ digits to run our algorithm.
△ Less
Submitted 28 October, 2024;
originally announced October 2024.
-
Simulating the aftermath of Northern European Enclosure Dam (NEED) break and flooding of European coast
Authors:
Paweł Maczuga,
Marcin Łoś,
Eirik Valseth,
Albert Oliver Serra,
Leszek Siwik,
Elisabede Alberdi Celaya,
Anna Paszyńska,
Maciej Paszyński
Abstract:
The Northern European Enclosure Dam (NEED) is a hypothetical project to prevent flooding in European countries following the rising ocean level due to melting arctic glaciers. This project involves the construction of two large dams between Scotland and Norway, as well as England and France. The anticipated cost of this project is 250 to 500 billion euros. In this paper, we present the simulation…
▽ More
The Northern European Enclosure Dam (NEED) is a hypothetical project to prevent flooding in European countries following the rising ocean level due to melting arctic glaciers. This project involves the construction of two large dams between Scotland and Norway, as well as England and France. The anticipated cost of this project is 250 to 500 billion euros. In this paper, we present the simulation of the aftermath of flooding on the European coastline caused by a catastrophic break of this hypothetical dam. From our simulation results, we can observe that there is a traveling wave after the accident, with a velocity of around 10 kilometers per hour, raising the sea level permanently inside the dammed region. This observation implies a need to construct additional dams or barriers protecting the northern coastline of the Netherlands and the interior of the Baltic Sea. Our simulations have been obtained using the following building blocks. First, a graph transformation model was applied to generate an adaptive mesh approximating the topography of the Earth. We employ the composition graph grammar model for breaking triangular elements in the mesh without the generation of hanging nodes. Second, the wave equation is formulated in a spherical latitude-longitude system of coordinates and solved by a high-order time integration scheme using the generalized $α$ method.
△ Less
Submitted 31 March, 2024;
originally announced May 2024.
-
Collocation-based Robust Variational Physics-Informed Neural Networks (CRVPINN)
Authors:
Marcin Łoś,
Tomasz Służalec,
Paweł Maczuga,
Askold Vilkha,
Carlos Uriarte,
Maciej Paszyński
Abstract:
Physics-Informed Neural Networks (PINNs) have been successfully applied to solve Partial Differential Equations (PDEs). Their loss function is founded on a strong residual minimization scheme. Variational Physics-Informed Neural Networks (VPINNs) are their natural extension to weak variational settings. In this context, the recent work of Robust Variational Physics-Informed Neural Networks (RVPINN…
▽ More
Physics-Informed Neural Networks (PINNs) have been successfully applied to solve Partial Differential Equations (PDEs). Their loss function is founded on a strong residual minimization scheme. Variational Physics-Informed Neural Networks (VPINNs) are their natural extension to weak variational settings. In this context, the recent work of Robust Variational Physics-Informed Neural Networks (RVPINNs) highlights the importance of conveniently translating the norms of the underlying continuum-level spaces to the discrete level. Otherwise, VPINNs might become unrobust, implying that residual minimization might be highly uncorrelated with a desired minimization of the error in the energy norm. However, applying this robustness to VPINNs typically entails dealing with the inverse of a Gram matrix, usually producing slow convergence speeds during training. In this work, we accelerate the implementation of RVPINN, establishing a LU factorization of sparse Gram matrix in a kind of point-collocation scheme with the same spirit as original PINNs. We call out method the Collocation-based Robust Variational Physics Informed Neural Networks (CRVPINN). We test our efficient CRVPINN algorithm on Laplace, advection-diffusion, and Stokes problems in two spatial dimensions.
△ Less
Submitted 16 October, 2024; v1 submitted 4 January, 2024;
originally announced January 2024.
-
Physics Informed Neural Network Code for 2D Transient Problems (PINN-2DT) Compatible with Google Colab
Authors:
Paweł Maczuga,
Maciej Sikora,
Maciej Skoczeń,
Przemysław Rożnawski,
Filip Tłuszcz,
Marcin Szubert,
Marcin Łoś,
Witold Dzwinel,
Keshav Pingali,
Maciej Paszyński
Abstract:
We present an open-source Physics Informed Neural Network environment for simulations of transient phenomena on two-dimensional rectangular domains, with the following features: (1) it is compatible with Google Colab which allows automatic execution on cloud environment; (2) it supports two dimensional time-dependent PDEs; (3) it provides simple interface for definition of the residual loss, bound…
▽ More
We present an open-source Physics Informed Neural Network environment for simulations of transient phenomena on two-dimensional rectangular domains, with the following features: (1) it is compatible with Google Colab which allows automatic execution on cloud environment; (2) it supports two dimensional time-dependent PDEs; (3) it provides simple interface for definition of the residual loss, boundary condition and initial loss, together with their weights; (4) it support Neumann and Dirichlet boundary conditions; (5) it allows for customizing the number of layers and neurons per layer, as well as for arbitrary activation function; (6) the learning rate and number of epochs are available as parameters; (7) it automatically differentiates PINN with respect to spatial and temporal variables; (8) it provides routines for plotting the convergence (with running average), initial conditions learnt, 2D and 3D snapshots from the simulation and movies (9) it includes a library of problems: (a) non-stationary heat transfer; (b) wave equation modeling a tsunami; (c) atmospheric simulations including thermal inversion; (d) tumor growth simulations.
△ Less
Submitted 19 February, 2024; v1 submitted 24 September, 2023;
originally announced October 2023.
-
Automatic stabilization of finite-element simulations using neural networks and hierarchical matrices
Authors:
Tomasz Sluzalec,
Mateusz Dobija,
Anna Paszynska,
Ignacio Muga,
Maciej Paszynski
Abstract:
Petrov-Galerkin formulations with optimal test functions allow for the stabilization of finite element simulations. In particular, given a discrete trial space, the optimal test space induces a numerical scheme delivering the best approximation in terms of a problem-dependent energy norm. This ideal approach has two shortcomings: first, we need to explicitly know the set of optimal test functions;…
▽ More
Petrov-Galerkin formulations with optimal test functions allow for the stabilization of finite element simulations. In particular, given a discrete trial space, the optimal test space induces a numerical scheme delivering the best approximation in terms of a problem-dependent energy norm. This ideal approach has two shortcomings: first, we need to explicitly know the set of optimal test functions; and second, the optimal test functions may have large supports inducing expensive dense linear systems. Nevertheless, parametric families of PDEs are an example where it is worth investing some (offline) computational effort to obtain stabilized linear systems that can be solved efficiently, for a given set of parameters, in an online stage. Therefore, as a remedy for the first shortcoming, we explicitly compute (offline) a function mapping any PDE-parameter, to the matrix of coefficients of optimal test functions (in a basis expansion) associated with that PDE-parameter. Next, as a remedy for the second shortcoming, we use the low-rank approximation to hierarchically compress the (non-square) matrix of coefficients of optimal test functions. In order to accelerate this process, we train a neural network to learn a critical bottleneck of the compression algorithm (for a given set of PDE-parameters). When solving online the resulting (compressed) Petrov-Galerkin formulation, we employ a GMRES iterative solver with inexpensive matrix-vector multiplications thanks to the low-rank features of the compressed matrix. We perform experiments showing that the full online procedure as fast as the original (unstable) Galerkin approach. In other words, we get the stabilization with hierarchical matrices and neural networks practically for free. We illustrate our findings by means of 2D Eriksson-Johnson and Hemholtz model problems.
△ Less
Submitted 24 December, 2022;
originally announced December 2022.
-
Quasi-optimal $hp$-finite element refinements towards singularities via deep neural network prediction
Authors:
Tomasz Sluzalec,
Rafal Grzeszczuk,
Sergio Rojas,
Witold Dzwinel,
Maciej Paszynski
Abstract:
We show how to construct the deep neural network (DNN) expert to predict quasi-optimal $hp$-refinements for a given computational problem. The main idea is to train the DNN expert during executing the self-adaptive $hp$-finite element method ($hp$-FEM) algorithm and use it later to predict further $hp$ refinements. For the training, we use a two-grid paradigm self-adaptive $hp$-FEM algorithm. It e…
▽ More
We show how to construct the deep neural network (DNN) expert to predict quasi-optimal $hp$-refinements for a given computational problem. The main idea is to train the DNN expert during executing the self-adaptive $hp$-finite element method ($hp$-FEM) algorithm and use it later to predict further $hp$ refinements. For the training, we use a two-grid paradigm self-adaptive $hp$-FEM algorithm. It employs the fine mesh to provide the optimal $hp$ refinements for coarse mesh elements. We aim to construct the DNN expert to identify quasi-optimal $hp$ refinements of the coarse mesh elements. During the training phase, we use the direct solver to obtain the solution for the fine mesh to guide the optimal refinements over the coarse mesh element. After training, we turn off the self-adaptive $hp$-FEM algorithm and continue with quasi-optimal refinements as proposed by the DNN expert trained. We test our method on three-dimensional Fichera and two-dimensional L-shaped domain problems. We verify the convergence of the numerical accuracy with respect to the mesh size. We show that the exponential convergence delivered by the self-adaptive $hp$-FEM can be preserved if we continue refinements with a properly trained DNN expert. Thus, in this paper, we show that from the self-adaptive $hp$-FEM it is possible to train the DNN expert the location of the singularities, and continue with the selection of the quasi-optimal $hp$ refinements, preserving the exponential convergence of the method.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
Deep neural networks for smooth approximation of physics with higher order and continuity B-spline base functions
Authors:
Kamil Doległo,
Anna Paszyńska,
Maciej Paszyński,
Leszek Demkowicz
Abstract:
This paper deals with the following important research question. Traditionally, the neural network employs non-linear activation functions concatenated with linear operators to approximate a given physical phenomenon. They "fill the space" with the concatenations of the activation functions and linear operators and adjust their coefficients to approximate the physical phenomena. We claim that it i…
▽ More
This paper deals with the following important research question. Traditionally, the neural network employs non-linear activation functions concatenated with linear operators to approximate a given physical phenomenon. They "fill the space" with the concatenations of the activation functions and linear operators and adjust their coefficients to approximate the physical phenomena. We claim that it is better to "fill the space" with linear combinations of smooth higher-order B-splines base functions as employed by isogeometric analysis and utilize the neural networks to adjust the coefficients of linear combinations. In other words, the possibilities of using neural networks for approximating the B-spline base functions' coefficients and by approximating the solution directly are evaluated. Solving differential equations with neural networks has been proposed by Maziar Raissi et al. in 2017 by introducing Physics-informed Neural Networks (PINN), which naturally encode underlying physical laws as prior information. Approximation of coefficients using a function as an input leverages the well-known capability of neural networks being universal function approximators. In essence, in the PINN approach the network approximates the value of the given field at a given point. We present an alternative approach, where the physcial quantity is approximated as a linear combination of smooth B-spline basis functions, and the neural network approximates the coefficients of B-splines. This research compares results from the DNN approximating the coefficients of the linear combination of B-spline basis functions, with the DNN approximating the solution directly. We show that our approach is cheaper and more accurate when approximating smooth physical fields.
△ Less
Submitted 3 January, 2022;
originally announced January 2022.
-
Supermodeling of tumor dynamics with parallel isogeometric analysis solver
Authors:
Maciej Paszynski,
Leszek Siwik,
Witold Dzwinel,
Keshav Pingali
Abstract:
Supermodeling is a modern, model-ensembling paradigm that integrates several self-synchronized imperfect sub-models by controlling a few meta-parameters to generate more accurate predictions of complex systems' dynamics. Continual synchronization between sub-models allows for trajectory predictions with superior accuracy compared to a single model or a classical ensemble of independent models whos…
▽ More
Supermodeling is a modern, model-ensembling paradigm that integrates several self-synchronized imperfect sub-models by controlling a few meta-parameters to generate more accurate predictions of complex systems' dynamics. Continual synchronization between sub-models allows for trajectory predictions with superior accuracy compared to a single model or a classical ensemble of independent models whose decision fusion is based on the majority voting or averaging the outcomes. However, it comes out from numerous observations that the supermodeling procedure's convergence depends on a few principal factors such as (1) the number of sub-models, (2) their proper selection, and (3) the choice of the convergent optimization procedure, which assimilates the supermodel meta-parameters to data. Herein, we focus on modeling the evolution of the system described by a set of PDEs. We prove that supermodeling is conditionally convergent to a fixed-point attractor regarding only the supermodel meta-parameters. We investigate the formal conditions of the convergence of the supermodeling scheme theoretically. We employ the Banach fixed point theorem for the supermodeling correction operator, updating the synchronization constants' values iteratively. The "nudging" of the supermodel to the ground truth should be well balanced because both too small and too large attraction to data cause the supermodel desynchronization. The time-step size can control the convergence of the training procedure, by balancing the Lipshitz continuity constant of the PDE operator. All the sub-models have to be close to the ground-truth along the training trajectory but still sufficiently diverse to explore the phase space better. As an example, we discuss the three-dimensional supermodel of tumor evolution to demonstrate the supermodel's perfect fit to artificial data generated based on real medical images.
△ Less
Submitted 26 February, 2021; v1 submitted 30 December, 2019;
originally announced December 2019.
