Showing 1–6 of 6 results for author: Dekel, S

Numerical Methods For PDEs Over Manifolds Using Spectral Physics Informed Neural Networks

Authors: Yuval Zelig, Shai Dekel

Abstract: We introduce an approach for solving PDEs over manifolds using physics informed neural networks whose architecture aligns with spectral methods. The networks are trained to take in as input samples of an initial condition, a time stamp and point(s) on the manifold and then output the solution's value at the given time and point(s). We provide proofs of our method for the heat equation on the inter… ▽ More We introduce an approach for solving PDEs over manifolds using physics informed neural networks whose architecture aligns with spectral methods. The networks are trained to take in as input samples of an initial condition, a time stamp and point(s) on the manifold and then output the solution's value at the given time and point(s). We provide proofs of our method for the heat equation on the interval and examples of unique network architectures that are adapted to nonlinear equations on the sphere and the torus. We also show that our spectral-inspired neural network architectures outperform the standard physics informed architectures. Our extensive experimental results include generalization studies where the testing dataset of initial conditions is randomly sampled from a significantly larger space than the training set. △ Less

Submitted 3 September, 2023; v1 submitted 10 February, 2023; originally announced February 2023.

Comments: 25 pages

Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functions

Authors: Ido Ben-Shaul, Tomer Galanti, Shai Dekel

Abstract: Multiplication layers are a key component in various influential neural network modules, including self-attention and hypernetwork layers. In this paper, we investigate the approximation capabilities of deep neural networks with intermediate neurons connected by simple multiplication operations. We consider two classes of target functions: generalized bandlimited functions, which are frequently us… ▽ More Multiplication layers are a key component in various influential neural network modules, including self-attention and hypernetwork layers. In this paper, we investigate the approximation capabilities of deep neural networks with intermediate neurons connected by simple multiplication operations. We consider two classes of target functions: generalized bandlimited functions, which are frequently used to model real-world signals with finite bandwidth, and Sobolev-Type balls, which are embedded in the Sobolev Space $\mathcal{W}^{r,2}$. Our results demonstrate that multiplicative neural networks can approximate these functions with significantly fewer layers and neurons compared to standard ReLU neural networks, with respect to both input dimension and approximation error. These findings suggest that multiplicative gates can outperform standard feed-forward layers and have potential for improving neural network design. △ Less

Submitted 11 January, 2023; originally announced January 2023.

MSC Class: 41A25; 68Q32; 68T07

Sparsity-Probe: Analysis tool for Deep Learning Models

Authors: Ido Ben-Shaul, Shai Dekel

Abstract: We propose a probe for the analysis of deep learning architectures that is based on machine learning and approximation theoretical principles. Given a deep learning architecture and a training set, during or after training, the Sparsity Probe allows to analyze the performance of intermediate layers by quantifying the geometrical features of representations of the training set. We show how the Spar… ▽ More We propose a probe for the analysis of deep learning architectures that is based on machine learning and approximation theoretical principles. Given a deep learning architecture and a training set, during or after training, the Sparsity Probe allows to analyze the performance of intermediate layers by quantifying the geometrical features of representations of the training set. We show how the Sparsity Probe enables measuring the contribution of adding depth to a given architecture, to detect under-performing layers, etc., all this without any auxiliary test data set. △ Less

Submitted 14 May, 2021; originally announced May 2021.

Exact recovery of non-uniform splines from the projection onto spaces of algebraic polynomials

Authors: Tamir Bendory, Shai Dekel, Arie Feuer

Abstract: In this work we consider the problem of recovering non-uniform splines from their projection onto spaces of algebraic polynomials. We show that under a certain Chebyshev-type separation condition on its knots, a spline whose inner-products with a polynomial basis and boundary conditions are known, can be recovered using Total Variation norm minimization. The proof of the uniqueness of the solution… ▽ More In this work we consider the problem of recovering non-uniform splines from their projection onto spaces of algebraic polynomials. We show that under a certain Chebyshev-type separation condition on its knots, a spline whose inner-products with a polynomial basis and boundary conditions are known, can be recovered using Total Variation norm minimization. The proof of the uniqueness of the solution uses the method of `dual' interpolating polynomials and is based on \cite{SR}, where the theory was developed for trigonometric polynomials. We also show results for the multivariate case. △ Less

Submitted 19 December, 2014; originally announced December 2014.

Hardy spaces associated with non-negative self-adjoint operators

Authors: S. Dekel, G. Kerkyacharian, G. Kyriazis, P. Petrushev

Abstract: Maximal and atomic Hardy spaces Hp and HAp , are considered in the setting of a doubling metric measure space in the presence of a non-negative self-adjoint operator whose heat kernel has Gaussian localization and the Markov property. It is shown that Hp = HAp with equivalent norms. Maximal and atomic Hardy spaces Hp and HAp , are considered in the setting of a doubling metric measure space in the presence of a non-negative self-adjoint operator whose heat kernel has Gaussian localization and the Markov property. It is shown that Hp = HAp with equivalent norms. △ Less

Submitted 1 September, 2014; originally announced September 2014.

MSC Class: Primary 42B30; Secondary 47G10

A new proof of the atomic decomposition of Hardy spaces

Authors: S. Dekel, G. Kerkyacharian, G. Kyriazis, P. Petrushev

Abstract: A new proof is given of the atomic decomposition of Hardy spaces Hp, in the classical setting of Rn. The new method can be used to establish atomic decomposition of maximal Hardy spaces in general setting and non classical settings. A new proof is given of the atomic decomposition of Hardy spaces Hp, in the classical setting of Rn. The new method can be used to establish atomic decomposition of maximal Hardy spaces in general setting and non classical settings. △ Less

Submitted 1 September, 2014; originally announced September 2014.

MSC Class: 42B30