-
Multivariate Newton Interpolation in Downward Closed Spaces Reaches the Optimal Geometric Approximation Rates for Bos--Levenberg--Trefethen Functions
Authors:
Michael Hecht,
Phil-Alexander Hofmann,
Damar Wicaksono,
Uwe Hernandez Acosta,
Krzysztof Gonciarz,
Jannik Kissinger,
Vladimir Sivkin,
Ivo F. Sbalzarini
Abstract:
We extend the univariate Newton interpolation algorithm to arbitrary spatial dimensions and for any choice of downward-closed polynomial space, while preserving its quadratic runtime and linear storage cost. The generalisation supports any choice of the provided notion of non-tensorial unisolvent interpolation nodes, whose number coincides with the dimension of the chosen-downward closed space. Sp…
▽ More
We extend the univariate Newton interpolation algorithm to arbitrary spatial dimensions and for any choice of downward-closed polynomial space, while preserving its quadratic runtime and linear storage cost. The generalisation supports any choice of the provided notion of non-tensorial unisolvent interpolation nodes, whose number coincides with the dimension of the chosen-downward closed space. Specifically, we prove that by selecting Leja-ordered Chebyshev-Lobatto or Leja nodes, the optimal geometric approximation rates for a class of analytic functions -- termed Bos--Levenberg--Trefethen functions -- are achieved and extend to the derivatives of the interpolants. In particular, choosing Euclidean degree results in downward-closed spaces whose dimension only grows sub-exponentially with spatial dimension, while delivering approximation rates close to, or even matching those of the tensorial maximum-degree case, mitigating the curse of dimensionality. Several numerical experiments demonstrate the performance of the resulting multivariate Newton interpolation compared to state-of-the-art alternatives and validate our theoretical results.
△ Less
Submitted 24 April, 2025;
originally announced April 2025.
-
A high-order fully Lagrangian particle level-set method for dynamic surfaces
Authors:
Lennart J. Schulze,
Sachin K. T. Veettill,
Ivo F. Sbalzarini
Abstract:
We present a fully Lagrangian particle level-set method based on high-order polynomial regression. This enables closest-point redistancing without requiring a regular Cartesian mesh, relaxing the need for particle-mesh interpolation. Instead, we perform level-set redistancing directly on irregularly distributed particles by polynomial regression in a Newton-Lagrange basis on a set of unisolvent no…
▽ More
We present a fully Lagrangian particle level-set method based on high-order polynomial regression. This enables closest-point redistancing without requiring a regular Cartesian mesh, relaxing the need for particle-mesh interpolation. Instead, we perform level-set redistancing directly on irregularly distributed particles by polynomial regression in a Newton-Lagrange basis on a set of unisolvent nodes. We demonstrate that the resulting particle closest-point (PCP) redistancing achieves high-order accuracy for 2D and 3D geometries discretized on highly irregular particle distributions and has better robustness against particle distortion than regression in a monomial basis. Further, we show convergence in a classic level-set benchmark case involving ill-conditioned particle distributions, and we present an application to an oscillating droplet simulation in multi-phase flow.
△ Less
Submitted 9 June, 2023;
originally announced June 2023.
-
On the Computational Power of Particle Methods
Authors:
Johannes Pahlke,
Ivo F. Sbalzarini
Abstract:
We investigate the computational power of particle methods, a well-established class of algorithms with applications in scientific computing and computer simulation. The computational power of a compute model determines the class of problems it can solve. Automata theory allows describing the computational power of abstract machines (automata) and the problems they can solve. At the top of the Cho…
▽ More
We investigate the computational power of particle methods, a well-established class of algorithms with applications in scientific computing and computer simulation. The computational power of a compute model determines the class of problems it can solve. Automata theory allows describing the computational power of abstract machines (automata) and the problems they can solve. At the top of the Chomsky hierarchy of formal languages and grammars are Turing machines, which resemble the concept on which most modern computers are built. Although particle methods can be interpreted as automata based on their formal definition, their computational power has so far not been studied. We address this by analyzing Turing completeness of particle methods. In particular, we prove two sets of restrictions under which a particle method is still Turing powerful, and we show when it loses Turing powerfulness. This contributes to understanding the theoretical foundations of particle methods and provides insight into the powerfulness of computer simulations.
△ Less
Submitted 27 March, 2025; v1 submitted 20 April, 2023;
originally announced April 2023.
-
Global Polynomial Level Sets for Numerical Differential Geometry of Smooth Closed Surfaces
Authors:
Sachin K. Thekke Veettil,
Gentian Zavalani,
Uwe Hernandez Acosta,
Ivo F. Sbalzarini,
Michael Hecht
Abstract:
We present a computational scheme that derives a global polynomial level set parametrisation for smooth closed surfaces from a regular surface-point set and prove its uniqueness. This enables us to approximate a broad class of smooth surfaces by affine algebraic varieties. From such a global polynomial level set parametrisation, differential-geometric quantities like mean and Gauss curvature can b…
▽ More
We present a computational scheme that derives a global polynomial level set parametrisation for smooth closed surfaces from a regular surface-point set and prove its uniqueness. This enables us to approximate a broad class of smooth surfaces by affine algebraic varieties. From such a global polynomial level set parametrisation, differential-geometric quantities like mean and Gauss curvature can be efficiently and accurately computed. Even 4$^{\text{th}}$-order terms such as the Laplacian of mean curvature are approximates with high precision. The accuracy performance results in a gain of computational efficiency, significantly reducing the number of surface points required compared to classic alternatives that rely on surface meshes or embedding grids. We mathematically derive and empirically demonstrate the strengths and the limitations of the present approach, suggesting it to be applicable to a large number of computational tasks in numerical differential geometry.
△ Less
Submitted 22 December, 2022;
originally announced December 2022.
-
A meshfree collocation scheme for surface differential operators on point clouds
Authors:
Abhinav Singh,
Alejandra Foggia,
Pietro Incardona,
Ivo F. Sbalzarini
Abstract:
We present a meshfree collocation scheme to discretize intrinsic surface differential operators over scalar fields on smooth curved surfaces with given normal vectors and a non-intersecting tubular neighborhood. The method is based on Discretization-Corrected Particle Strength Exchange (DC-PSE), which generalizes finite difference methods to meshfree point clouds. The proposed Surface DC-PSE metho…
▽ More
We present a meshfree collocation scheme to discretize intrinsic surface differential operators over scalar fields on smooth curved surfaces with given normal vectors and a non-intersecting tubular neighborhood. The method is based on Discretization-Corrected Particle Strength Exchange (DC-PSE), which generalizes finite difference methods to meshfree point clouds. The proposed Surface DC-PSE method is derived from an embedding theorem, but we analytically reduce the operator kernels along surface normals to obtain a purely intrinsic computational scheme over surface point clouds. We benchmark Surface DC-PSE by discretizing the Laplace-Beltrami operator on a circle and a sphere, and we present convergence results for both explicit and implicit solvers. We then showcase the algorithm on the problem of computing Gauss and mean curvature of an ellipsoid and of the Stanford Bunny by approximating the intrinsic divergence of the normal vector field. Finally, we compare Surface DC-PSE with Surface Finite Elements (SFEM) and Diffuse-Interface Finite Elements (DI FEM) in a validation case.
△ Less
Submitted 16 October, 2023; v1 submitted 22 May, 2022;
originally announced May 2022.
-
Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks
Authors:
Suryanarayana Maddu,
Dominik Sturm,
Christian L. Müller,
Ivo F. Sbalzarini
Abstract:
We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of dat…
▽ More
We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse-Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically $\boldsymbolε$-optimal training. We demonstrate the effectiveness of inverse-Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse-Dirichlet weighting protects a PINN against catastrophic forgetting.
△ Less
Submitted 2 July, 2021;
originally announced July 2021.
-
STENCIL-NET: Data-driven solution-adaptive discretization of partial differential equations
Authors:
Suryanarayana Maddu,
Dominik Sturm,
Bevan L. Cheeseman,
Christian L. Müller,
Ivo F. Sbalzarini
Abstract:
Numerical methods for approximately solving partial differential equations (PDE) are at the core of scientific computing. Often, this requires high-resolution or adaptive discretization grids to capture relevant spatio-temporal features in the PDE solution, e.g., in applications like turbulence, combustion, and shock propagation. Numerical approximation also requires knowing the PDE in order to co…
▽ More
Numerical methods for approximately solving partial differential equations (PDE) are at the core of scientific computing. Often, this requires high-resolution or adaptive discretization grids to capture relevant spatio-temporal features in the PDE solution, e.g., in applications like turbulence, combustion, and shock propagation. Numerical approximation also requires knowing the PDE in order to construct problem-specific discretizations. Systematically deriving such solution-adaptive discrete operators, however, is a current challenge. Here we present STENCIL-NET, an artificial neural network architecture for data-driven learning of problem- and resolution-specific local discretizations of nonlinear PDEs. STENCIL-NET achieves numerically stable discretization of the operators in an unknown nonlinear PDE by spatially and temporally adaptive parametric pooling on regular Cartesian grids, and by incorporating knowledge about discrete time integration. Knowing the actual PDE is not necessary, as solution data is sufficient to train the network to learn the discrete operators. A once-trained STENCIL-NET model can be used to predict solutions of the PDE on larger spatial domains and for longer times than it was trained for, hence addressing the problem of PDE-constrained extrapolation from data. To support this claim, we present numerical experiments on long-term forecasting of chaotic PDE solutions on coarse spatio-temporal grids. We also quantify the speed-up achieved by substituting base-line numerical methods with equation-free STENCIL-NET predictions on coarser grids with little compromise on accuracy.
△ Less
Submitted 18 January, 2021; v1 submitted 15 January, 2021;
originally announced January 2021.
-
Multivariate Interpolation in Unisolvent Nodes -- Lifting the Curse of Dimensionality
Authors:
Michael Hecht,
Krzysztof Gonciarz,
Jannik Michelfeit,
Vladimir Sivkin,
Ivo F. Sbalzarini
Abstract:
We extend Newton and Lagrange interpolation to arbitrary dimensions. The core contribution that enables this is a generalized notion of non-tensorial unisolvent nodes, i.e., nodes on which the multivariate polynomial interpolant of a function is unique. By validation, we reach the optimal exponential Trefethen rates for a class of analytic functions, we term Trefethen functions. The number of inte…
▽ More
We extend Newton and Lagrange interpolation to arbitrary dimensions. The core contribution that enables this is a generalized notion of non-tensorial unisolvent nodes, i.e., nodes on which the multivariate polynomial interpolant of a function is unique. By validation, we reach the optimal exponential Trefethen rates for a class of analytic functions, we term Trefethen functions. The number of interpolation nodes required for computing the optimal interpolant depends sub-exponentially on the dimension, hence resisting the curse of dimensionality. Based on these results, we propose an algorithm to efficiently and numerically stably solve arbitrary-dimensional interpolation problems, with at most quadratic runtime and linear memory requirement.
△ Less
Submitted 16 April, 2024; v1 submitted 21 October, 2020;
originally announced October 2020.
-
A robustness measure for singular point and index estimation in discretized orientation and vector fields
Authors:
Karl B. Hoffmann,
Ivo F. Sbalzarini
Abstract:
The identification of singular points or topological defects in discretized vector fields occurs in diverse areas ranging from the polarization of the cosmic microwave background to liquid crystals to fingerprint recognition and bio-medical imaging. Due to their discrete nature, defects and their topological charge cannot depend continuously on each single vector, but they discontinuously change a…
▽ More
The identification of singular points or topological defects in discretized vector fields occurs in diverse areas ranging from the polarization of the cosmic microwave background to liquid crystals to fingerprint recognition and bio-medical imaging. Due to their discrete nature, defects and their topological charge cannot depend continuously on each single vector, but they discontinuously change as soon as a vector changes by more than a threshold. Considering this threshold of admissible change at the level of vectors, we develop a robustness measure for discrete defect estimators. Here, we compare different template paths for defect estimation in discretized vector or orientation fields. Sampling prototypical vector field patterns around defects shows that the robustness increases with the length of template path, but less so in the presence of noise on the vectors. We therefore find an optimal trade-off between resolution and robustness against noise for relatively small templates, except for the "single pixel" defect analysis, which cannot exclude zero robustness. The presented robustness measure paves the way for uncertainty quantification of defects in discretized vector fields.
△ Less
Submitted 30 September, 2020;
originally announced September 2020.
-
Stability selection enables robust learning of partial differential equations from limited noisy data
Authors:
Suryanarayana Maddu,
Bevan L. Cheeseman,
Ivo F. Sbalzarini,
Christian L. Müller
Abstract:
We present a statistical learning framework for robust identification of partial differential equations from noisy spatiotemporal data. Extending previous sparse regression approaches for inferring PDE models from simulated data, we address key issues that have thus far limited the application of these methods to noisy experimental data, namely their robustness against noise and the need for manua…
▽ More
We present a statistical learning framework for robust identification of partial differential equations from noisy spatiotemporal data. Extending previous sparse regression approaches for inferring PDE models from simulated data, we address key issues that have thus far limited the application of these methods to noisy experimental data, namely their robustness against noise and the need for manual parameter tuning. We address both points by proposing a stability-based model selection scheme to determine the level of regularization required for reproducible recovery of the underlying PDE. This avoids manual parameter tuning and provides a principled way to improve the method's robustness against noise in the data. Our stability selection approach, termed PDE-STRIDE, can be combined with any sparsity-promoting penalized regression model and provides an interpretable criterion for model component importance. We show that in particular the combination of stability selection with the iterative hard-thresholding algorithm from compressed sensing provides a fast, parameter-free, and robust computational framework for PDE inference that outperforms previous algorithmic approaches with respect to recovery accuracy, amount of data required, and robustness to noise. We illustrate the performance of our approach on a wide range of noise-corrupted simulated benchmark problems, including 1D Burgers, 2D vorticity-transport, and 3D reaction-diffusion problems. We demonstrate the practical applicability of our method on real-world data by considering a purely data-driven re-evaluation of the advective triggering hypothesis for an embryonic polarization system in C.~elegans. Using fluorescence microscopy images of C.~elegans zygotes as input data, our framework is able to recover the PDE model for the regulatory reaction-diffusion-flow network of the associated proteins.
△ Less
Submitted 17 July, 2019;
originally announced July 2019.
-
Multivariate Newton Interpolation
Authors:
Michael Hecht,
Karl B. Hoffmann,
Bevan L. Cheeseman,
Ivo F. Sbalzarini
Abstract:
For $m,n \in \mathbb{N}$, $m\geq 1$ and a given function $f : \mathbb{R}^m\longrightarrow \mathbb{R}$, the polynomial interpolation problem (PIP) is to determine a unisolvent node set $P_{m,n} \subseteq \mathbb{R}^m$ of $N(m,n):=|P_{m,n}|=\binom{m+n}{n}$ points and the uniquely defined polynomial $Q_{m,n,f}\in Π_{m,n}$ in $m$ variables of degree $\mathrm{deg}(Q_{m,n,f})\leq n \in \mathbb{N}$ that…
▽ More
For $m,n \in \mathbb{N}$, $m\geq 1$ and a given function $f : \mathbb{R}^m\longrightarrow \mathbb{R}$, the polynomial interpolation problem (PIP) is to determine a unisolvent node set $P_{m,n} \subseteq \mathbb{R}^m$ of $N(m,n):=|P_{m,n}|=\binom{m+n}{n}$ points and the uniquely defined polynomial $Q_{m,n,f}\in Π_{m,n}$ in $m$ variables of degree $\mathrm{deg}(Q_{m,n,f})\leq n \in \mathbb{N}$ that fits $f$ on $P_{m,n}$, i.e., $Q_{m,n,f}(p) = f(p)$, $\forall\, p \in P_{m,n}$. For $m=1$ the solution to the PIP is well known. In higher dimensions, however, no closed framework was available. We here present a generalization of the classic Newton interpolation from one-dimensional to arbitrary-dimensional spaces. Further we formulate an algorithm, termed PIP-SOLVER, based on a multivariate divided difference scheme that computes the solution $Q_{m,n,f}$ in $\mathcal{O}\big(N(m,n)^2\big)$ time using $\mathcal{O}\big(mN(m,n)\big)$ memory. Further, we introduce unisolvent Newton-Chebyshev nodes and show that these nodes avoid Runge's phenomenon in the sense that arbitrary periodic Sobolev functions $f \in H^k(Ω,\mathbb{R}) \subsetneq C^0(Ω,\mathbb{R})$, $Ω=[-1,1]^m$ of regularity $k >m/2$ can be uniformly approximated, i.e., $ \lim_{n\rightarrow \infty}||\,f -Q_{m,n,f} \,||_{C^0(Ω)}= 0$. Numerical experiments demonstrate the computational performance and approximation accuracy of the PIP-SOLVER in practice. We expect the presented results to be relevant for many applications, including numerical solvers, quadrature, non-linear optimization, polynomial regression, adaptive sampling, Bayesian inference, and spectral analysis.
△ Less
Submitted 19 March, 2020; v1 submitted 11 December, 2018;
originally announced December 2018.
-
A Quadratic-Time Algorithm for General Multivariate Polynomial Interpolation
Authors:
M. Hecht,
B. L. Cheeseman,
K. B. Hoffmann,
I. F. Sbalzarini
Abstract:
For $m,n \in \mathbb{N}$, $m\geq 1$ and a given function $f : \mathbb{R}^m\longrightarrow \mathbb{R}$ the polynomial interpolation problem (PIP) is to determine a \emph{generic node set} $P \subseteq \mathbb{R}^m$ and the coefficients of the uniquely defined polynomial $Q\in\mathbb{R}[x_1,\dots,x_m]$ in $m$ variables of degree $\mathrm{deg}(Q)\leq n \in \mathbb{N}$ that fits $f$ on $P$, i.e.,…
▽ More
For $m,n \in \mathbb{N}$, $m\geq 1$ and a given function $f : \mathbb{R}^m\longrightarrow \mathbb{R}$ the polynomial interpolation problem (PIP) is to determine a \emph{generic node set} $P \subseteq \mathbb{R}^m$ and the coefficients of the uniquely defined polynomial $Q\in\mathbb{R}[x_1,\dots,x_m]$ in $m$ variables of degree $\mathrm{deg}(Q)\leq n \in \mathbb{N}$ that fits $f$ on $P$, i.e., $Q(p) = f(p)$, $\forall\, p \in P$. We here show that in general, i.e., for arbitrary $m,n \in \mathbb{N}$, $m \geq 1$, there exists an algorithm that determines $P$ and computes the $N(\mbox{m,n})=\#P$ coefficients of $Q$ in $\mathcal{O}\big(N(\mbox{m,n})^2\big)$ time using $\mathcal{O}\big(\mbox{m}N(\mbox{m,n})\big)$ storage, without inverting the occurring Vandermonde matrix. We provide such an algorithm, termed PIP-SOLVER, based on a recursive decomposition of the problem and prove its correctness. Since the present approach solves the PIP without matrix inversion, it is computationally more efficient and numerically more robust than previous approaches. We demonstrate this in numerical experiments and compare with previous approaches based on matrix inversion and linear systems solving.
△ Less
Submitted 30 October, 2017;
originally announced October 2017.
-
Particle methods enable fast and simple approximation of Sobolev gradients in image segmentation
Authors:
Ivo F. Sbalzarini,
Sophie Schneider,
Janick Cardinale
Abstract:
Bio-image analysis is challenging due to inhomogeneous intensity distributions and high levels of noise in the images. Bayesian inference provides a principled way for regularizing the problem using prior knowledge. A fundamental choice is how one measures "distances" between shapes in an image. It has been shown that the straightforward geometric L2 distance is degenerate and leads to pathologica…
▽ More
Bio-image analysis is challenging due to inhomogeneous intensity distributions and high levels of noise in the images. Bayesian inference provides a principled way for regularizing the problem using prior knowledge. A fundamental choice is how one measures "distances" between shapes in an image. It has been shown that the straightforward geometric L2 distance is degenerate and leads to pathological situations. This is avoided when using Sobolev gradients, rendering the segmentation problem less ill-posed. The high computational cost and implementation overhead of Sobolev gradients, however, have hampered practical applications. We show how particle methods as applied to image segmentation allow for a simple and computationally efficient implementation of Sobolev gradients. We show that the evaluation of Sobolev gradients amounts to particle-particle interactions along the contour in an image. We extend an existing particle-based segmentation algorithm to using Sobolev gradients. Using synthetic and real-world images, we benchmark the results for both 2D and 3D images using piecewise smooth and piecewise constant region models. The present particle approximation of Sobolev gradients is 2.8 to 10 times faster than the previous reference implementation, but retains the known favorable properties of Sobolev gradients. This speedup is achieved by using local particle-particle interactions instead of solving a global Poisson equation at each iteration. The computational time per iteration is higher for Sobolev gradients than for L2 gradients. Since Sobolev gradients precondition the optimization problem, however, a smaller number of overall iterations may be necessary for the algorithm to converge, which can in some cases amortize the higher per-iteration cost.
△ Less
Submitted 2 March, 2014;
originally announced March 2014.
