-
Physics Informed Neural Network Code for 2D Transient Problems (PINN-2DT) Compatible with Google Colab
Authors:
Paweł Maczuga,
Maciej Sikora,
Maciej Skoczeń,
Przemysław Rożnawski,
Filip Tłuszcz,
Marcin Szubert,
Marcin Łoś,
Witold Dzwinel,
Keshav Pingali,
Maciej Paszyński
Abstract:
We present an open-source Physics Informed Neural Network environment for simulations of transient phenomena on two-dimensional rectangular domains, with the following features: (1) it is compatible with Google Colab which allows automatic execution on cloud environment; (2) it supports two dimensional time-dependent PDEs; (3) it provides simple interface for definition of the residual loss, bound…
▽ More
We present an open-source Physics Informed Neural Network environment for simulations of transient phenomena on two-dimensional rectangular domains, with the following features: (1) it is compatible with Google Colab which allows automatic execution on cloud environment; (2) it supports two dimensional time-dependent PDEs; (3) it provides simple interface for definition of the residual loss, boundary condition and initial loss, together with their weights; (4) it support Neumann and Dirichlet boundary conditions; (5) it allows for customizing the number of layers and neurons per layer, as well as for arbitrary activation function; (6) the learning rate and number of epochs are available as parameters; (7) it automatically differentiates PINN with respect to spatial and temporal variables; (8) it provides routines for plotting the convergence (with running average), initial conditions learnt, 2D and 3D snapshots from the simulation and movies (9) it includes a library of problems: (a) non-stationary heat transfer; (b) wave equation modeling a tsunami; (c) atmospheric simulations including thermal inversion; (d) tumor growth simulations.
△ Less
Submitted 19 February, 2024; v1 submitted 24 September, 2023;
originally announced October 2023.
-
Quasi-optimal $hp$-finite element refinements towards singularities via deep neural network prediction
Authors:
Tomasz Sluzalec,
Rafal Grzeszczuk,
Sergio Rojas,
Witold Dzwinel,
Maciej Paszynski
Abstract:
We show how to construct the deep neural network (DNN) expert to predict quasi-optimal $hp$-refinements for a given computational problem. The main idea is to train the DNN expert during executing the self-adaptive $hp$-finite element method ($hp$-FEM) algorithm and use it later to predict further $hp$ refinements. For the training, we use a two-grid paradigm self-adaptive $hp$-FEM algorithm. It e…
▽ More
We show how to construct the deep neural network (DNN) expert to predict quasi-optimal $hp$-refinements for a given computational problem. The main idea is to train the DNN expert during executing the self-adaptive $hp$-finite element method ($hp$-FEM) algorithm and use it later to predict further $hp$ refinements. For the training, we use a two-grid paradigm self-adaptive $hp$-FEM algorithm. It employs the fine mesh to provide the optimal $hp$ refinements for coarse mesh elements. We aim to construct the DNN expert to identify quasi-optimal $hp$ refinements of the coarse mesh elements. During the training phase, we use the direct solver to obtain the solution for the fine mesh to guide the optimal refinements over the coarse mesh element. After training, we turn off the self-adaptive $hp$-FEM algorithm and continue with quasi-optimal refinements as proposed by the DNN expert trained. We test our method on three-dimensional Fichera and two-dimensional L-shaped domain problems. We verify the convergence of the numerical accuracy with respect to the mesh size. We show that the exponential convergence delivered by the self-adaptive $hp$-FEM can be preserved if we continue refinements with a properly trained DNN expert. Thus, in this paper, we show that from the self-adaptive $hp$-FEM it is possible to train the DNN expert the location of the singularities, and continue with the selection of the quasi-optimal $hp$ refinements, preserving the exponential convergence of the method.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
SuperNet -- An efficient method of neural networks ensembling
Authors:
Ludwik Bukowski,
Witold Dzwinel
Abstract:
The main flaw of neural network ensembling is that it is exceptionally demanding computationally, especially, if the individual sub-models are large neural networks, which must be trained separately. Having in mind that modern DNNs can be very accurate, they are already the huge ensembles of simple classifiers, and that one can construct more thrifty compressed neural net of a similar performance…
▽ More
The main flaw of neural network ensembling is that it is exceptionally demanding computationally, especially, if the individual sub-models are large neural networks, which must be trained separately. Having in mind that modern DNNs can be very accurate, they are already the huge ensembles of simple classifiers, and that one can construct more thrifty compressed neural net of a similar performance for any ensemble, the idea of designing the expensive SuperNets can be questionable. The widespread belief that ensembling increases the prediction time, makes it not attractive and can be the reason that the main stream of ML research is directed towards developing better loss functions and learning strategies for more advanced and efficient neural networks. On the other hand, all these factors make the architectures more complex what may lead to overfitting and high computational complexity, that is, to the same flaws for which the highly parametrized SuperNets ensembles are blamed. The goal of the master thesis is to speed up the execution time required for ensemble generation. Instead of training K inaccurate sub-models, each of them can represent various phases of training (representing various local minima of the loss function) of a single DNN [Huang et al., 2017; Gripov et al., 2018]. Thus, the computational performance of the SuperNet can be comparable to the maximum CPU time spent on training its single sub-model, plus usually much shorter CPU time required for training the SuperNet coupling factors.
△ Less
Submitted 29 March, 2020;
originally announced March 2020.
-
Supermodeling of tumor dynamics with parallel isogeometric analysis solver
Authors:
Maciej Paszynski,
Leszek Siwik,
Witold Dzwinel,
Keshav Pingali
Abstract:
Supermodeling is a modern, model-ensembling paradigm that integrates several self-synchronized imperfect sub-models by controlling a few meta-parameters to generate more accurate predictions of complex systems' dynamics. Continual synchronization between sub-models allows for trajectory predictions with superior accuracy compared to a single model or a classical ensemble of independent models whos…
▽ More
Supermodeling is a modern, model-ensembling paradigm that integrates several self-synchronized imperfect sub-models by controlling a few meta-parameters to generate more accurate predictions of complex systems' dynamics. Continual synchronization between sub-models allows for trajectory predictions with superior accuracy compared to a single model or a classical ensemble of independent models whose decision fusion is based on the majority voting or averaging the outcomes. However, it comes out from numerous observations that the supermodeling procedure's convergence depends on a few principal factors such as (1) the number of sub-models, (2) their proper selection, and (3) the choice of the convergent optimization procedure, which assimilates the supermodel meta-parameters to data. Herein, we focus on modeling the evolution of the system described by a set of PDEs. We prove that supermodeling is conditionally convergent to a fixed-point attractor regarding only the supermodel meta-parameters. We investigate the formal conditions of the convergence of the supermodeling scheme theoretically. We employ the Banach fixed point theorem for the supermodeling correction operator, updating the synchronization constants' values iteratively. The "nudging" of the supermodel to the ground truth should be well balanced because both too small and too large attraction to data cause the supermodel desynchronization. The time-step size can control the convergence of the training procedure, by balancing the Lipshitz continuity constant of the PDE operator. All the sub-models have to be close to the ground-truth along the training trajectory but still sufficiently diverse to explore the phase space better. As an example, we discuss the three-dimensional supermodel of tumor evolution to demonstrate the supermodel's perfect fit to artificial data generated based on real medical images.
△ Less
Submitted 26 February, 2021; v1 submitted 30 December, 2019;
originally announced December 2019.
-
2-D Embedding of Large and High-dimensional Data with Minimal Memory and Computational Time Requirements
Authors:
Witold Dzwinel,
Rafal Wcislo,
Stan Matwin
Abstract:
In the advent of big data era, interactive visualization of large data sets consisting of M*10^5+ high-dimensional feature vectors of length N (N ~ 10^3+), is an indispensable tool for data exploratory analysis. The state-of-the-art data embedding (DE) methods of N-D data into 2-D (3-D) visually perceptible space (e.g., based on t-SNE concept) are too demanding computationally to be efficiently em…
▽ More
In the advent of big data era, interactive visualization of large data sets consisting of M*10^5+ high-dimensional feature vectors of length N (N ~ 10^3+), is an indispensable tool for data exploratory analysis. The state-of-the-art data embedding (DE) methods of N-D data into 2-D (3-D) visually perceptible space (e.g., based on t-SNE concept) are too demanding computationally to be efficiently employed for interactive data analytics of large and high-dimensional datasets. Herein we present a simple method, ivhd (interactive visualization of high-dimensional data tool), which radically outperforms the modern data-embedding algorithms in both computational and memory loads, while retaining high quality of N-D data embedding in 2-D (3-D). We show that DE problem is equivalent to the nearest neighbor nn-graph visualization, where only indices of a few nearest neighbors of each data sample has to be known, and binary distance between data samples -- 0 to the nearest and 1 to the other samples -- is defined. These improvements reduce the time-complexity and memory load from O(M log M) to O(M), and ensure minimal O(M) proportionality coefficient as well. We demonstrate high efficiency, quality and robustness of ivhd on popular benchmark datasets such as MNIST, 20NG, NORB and RCV1.
△ Less
Submitted 4 February, 2019;
originally announced February 2019.
