-
Faber polynomials in a deltoid region and power iteration momentum methods
Authors:
Peter Cowal,
Nicholas F. Marshall,
Sara Pollock
Abstract:
We consider a region in the complex plane enclosed by a deltoid curve inscribed in the unit circle, and define a family of polynomials $P_n$ that satisfy the same recurrence relation as the Faber polynomials for this region. We use this family of polynomials to give a constructive proof that $z^n$ is approximately a polynomial of degree $\sim\sqrt{n}$ within the deltoid region. Moreover, we show t…
▽ More
We consider a region in the complex plane enclosed by a deltoid curve inscribed in the unit circle, and define a family of polynomials $P_n$ that satisfy the same recurrence relation as the Faber polynomials for this region. We use this family of polynomials to give a constructive proof that $z^n$ is approximately a polynomial of degree $\sim\sqrt{n}$ within the deltoid region. Moreover, we show that $|P_n| \le 1$ in this deltoid region, and that, if $|z| = 1+\varepsilon$, then the magnitude $|P_n(z)|$ is at least $\frac{1}{3}(1+\sqrt{\varepsilon})^n$, for all $\varepsilon > 0$. We illustrate our polynomial approximation theory with an application to iterative linear algebra. In particular, we construct a higher-order momentum-based method that accelerates the power iteration for certain matrices with complex eigenvalues. We show how the method can be run dynamically when the two dominant eigenvalues are real and positive.
△ Less
Submitted 2 July, 2025;
originally announced July 2025.
-
Fast expansion into harmonics on the ball
Authors:
Joe Kileel,
Nicholas F. Marshall,
Oscar Mickelin,
Amit Singer
Abstract:
We devise fast and provably accurate algorithms to transform between an $N\times N \times N$ Cartesian voxel representation of a three-dimensional function and its expansion into the {ball harmonics}, that is, the eigenbasis of the Dirichlet Laplacian on the unit ball in $\mathbb{R}^3$. Given $\varepsilon > 0$, our algorithms achieve relative $\ell^1$ - $\ell^\infty$ accuracy $\varepsilon$ in time…
▽ More
We devise fast and provably accurate algorithms to transform between an $N\times N \times N$ Cartesian voxel representation of a three-dimensional function and its expansion into the {ball harmonics}, that is, the eigenbasis of the Dirichlet Laplacian on the unit ball in $\mathbb{R}^3$. Given $\varepsilon > 0$, our algorithms achieve relative $\ell^1$ - $\ell^\infty$ accuracy $\varepsilon$ in time $O(N^3 (\log N)^2 + N^3 |\log \varepsilon|^2)$, while the naïve direct application of the expansion operators has time complexity $O(N^6)$. We illustrate our methods on numerical examples.
△ Less
Submitted 1 May, 2025; v1 submitted 9 June, 2024;
originally announced June 2024.
-
Laplace-HDC: Understanding the geometry of binary hyperdimensional computing
Authors:
Saeid Pourmand,
Wyatt D. Whiting,
Alireza Aghasi,
Nicholas F. Marshall
Abstract:
This paper studies the geometry of binary hyperdimensional computing (HDC), a computational scheme in which data are encoded using high-dimensional binary vectors. We establish a result about the similarity structure induced by the HDC binding operator and show that the Laplace kernel naturally arises in this setting, motivating our new encoding method Laplace-HDC, which improves upon previous met…
▽ More
This paper studies the geometry of binary hyperdimensional computing (HDC), a computational scheme in which data are encoded using high-dimensional binary vectors. We establish a result about the similarity structure induced by the HDC binding operator and show that the Laplace kernel naturally arises in this setting, motivating our new encoding method Laplace-HDC, which improves upon previous methods. We describe how our results indicate limitations of binary HDC in encoding spatial information from images and discuss potential solutions, including using Haar convolutional features and the definition of a translation-equivariant HDC encoding. Several numerical experiments highlighting the improved accuracy of Laplace-HDC in contrast to alternative methods are presented. We also numerically study other aspects of the proposed framework such as robustness and the underlying translation-equivariant encoding.
△ Less
Submitted 26 April, 2024; v1 submitted 16 April, 2024;
originally announced April 2024.
-
Randomized Kaczmarz with geometrically smoothed momentum
Authors:
Seth J. Alderman,
Roan W. Luikart,
Nicholas F. Marshall
Abstract:
This paper studies the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our…
▽ More
This paper studies the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our result and pose several questions.
△ Less
Submitted 24 August, 2024; v1 submitted 17 January, 2024;
originally announced January 2024.
-
From the binomial reshuffling model to Poisson distribution of money
Authors:
Fei Cao,
Nicholas F. Marshall
Abstract:
We present a novel reshuffling exchange model and investigate its long time behavior. In this model, two individuals are picked randomly, and their wealth $X_i$ and $X_j$ are redistributed by flipping a sequence of fair coins leading to a binomial distribution denoted $B \circ (X_i+X_j)$. This dynamics can be considered as a natural variant of the so-called uniform reshuffling model in econophysic…
▽ More
We present a novel reshuffling exchange model and investigate its long time behavior. In this model, two individuals are picked randomly, and their wealth $X_i$ and $X_j$ are redistributed by flipping a sequence of fair coins leading to a binomial distribution denoted $B \circ (X_i+X_j)$. This dynamics can be considered as a natural variant of the so-called uniform reshuffling model in econophysics [2,14]. As the number of individuals goes to infinity, we derive its mean-field limit, which links the stochastic dynamics to a deterministic infinite system of ordinary differential equations. The main result of this work is then to prove (using a coupling argument) that the distribution of wealth converges to the Poisson distribution in the $2$-Wasserstein metric. Numerical simulations illustrate the main result and suggest that the polynomial convergence decay might be further improved.
△ Less
Submitted 29 December, 2022;
originally announced December 2022.
-
Fast Principal Component Analysis for Cryo-EM Images
Authors:
Nicholas F. Marshall,
Oscar Mickelin,
Yunpeng Shi,
Amit Singer
Abstract:
Principal component analysis (PCA) plays an important role in the analysis of cryo-EM images for various tasks such as classification, denoising, compression, and ab-initio modeling. We introduce a fast method for estimating a compressed representation of the 2-D covariance matrix of noisy cryo-electron microscopy projection images that enables fast PCA computation. Our method is based on a new al…
▽ More
Principal component analysis (PCA) plays an important role in the analysis of cryo-EM images for various tasks such as classification, denoising, compression, and ab-initio modeling. We introduce a fast method for estimating a compressed representation of the 2-D covariance matrix of noisy cryo-electron microscopy projection images that enables fast PCA computation. Our method is based on a new algorithm for expanding images in the Fourier-Bessel basis (the harmonics on the disk), which provides a convenient way to handle the effect of the contrast transfer functions. For $N$ images of size $L\times L$, our method has time complexity $O(N L^3 + L^4)$ and space complexity $O(NL^2 + L^3)$. In contrast to previous work, these complexities are independent of the number of different contrast transfer functions of the images. We demonstrate our approach on synthetic and experimental data and show acceleration by factors of up to two orders of magnitude.
△ Less
Submitted 31 October, 2022;
originally announced October 2022.
-
Fast expansion into harmonics on the disk: a steerable basis with fast radial convolutions
Authors:
Nicholas F. Marshall,
Oscar Mickelin,
Amit Singer
Abstract:
We present a fast and numerically accurate method for expanding digitized $L \times L$ images representing functions on $[-1,1]^2$ supported on the disk $\{x \in \mathbb{R}^2 : |x|<1\}$ in the harmonics (Dirichlet Laplacian eigenfunctions) on the disk. Our method, which we refer to as the Fast Disk Harmonics Transform (FDHT), runs in $O(L^2 \log L)$ operations. This basis is also known as the Four…
▽ More
We present a fast and numerically accurate method for expanding digitized $L \times L$ images representing functions on $[-1,1]^2$ supported on the disk $\{x \in \mathbb{R}^2 : |x|<1\}$ in the harmonics (Dirichlet Laplacian eigenfunctions) on the disk. Our method, which we refer to as the Fast Disk Harmonics Transform (FDHT), runs in $O(L^2 \log L)$ operations. This basis is also known as the Fourier-Bessel basis, and it has several computational advantages: it is orthogonal, ordered by frequency, and steerable in the sense that images expanded in the basis can be rotated by applying a diagonal transform to the coefficients. Moreover, we show that convolution with radial functions can also be efficiently computed by applying a diagonal transform to the coefficients.
△ Less
Submitted 21 December, 2022; v1 submitted 27 July, 2022;
originally announced July 2022.
-
An optimal scheduled learning rate for a randomized Kaczmarz algorithm
Authors:
Nicholas F. Marshall,
Oscar Mickelin
Abstract:
We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving $A x \approx b + \varepsilon$, where $A x =b$ is a consistent linear system and $\varepsilon$ has independent mean zero random entries. We derive a learning rate schedule which optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergenc…
▽ More
We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving $A x \approx b + \varepsilon$, where $A x =b$ is a consistent linear system and $\varepsilon$ has independent mean zero random entries. We derive a learning rate schedule which optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert-$W$ function of an exponential.
△ Less
Submitted 9 August, 2022; v1 submitted 24 February, 2022;
originally announced February 2022.
-
On a linearization of quadratic Wasserstein distance
Authors:
Philip Greengard,
Jeremy G. Hoskins,
Nicholas F. Marshall,
Amit Singer
Abstract:
This paper studies the problem of computing a linear approximation of quadratic Wasserstein distance $W_2$. In particular, we compute an approximation of the negative homogeneous weighted Sobolev norm whose connection to Wasserstein distance follows from a classic linearization of a general Monge-Ampére equation. Our contribution is threefold. First, we provide expository material on this classic…
▽ More
This paper studies the problem of computing a linear approximation of quadratic Wasserstein distance $W_2$. In particular, we compute an approximation of the negative homogeneous weighted Sobolev norm whose connection to Wasserstein distance follows from a classic linearization of a general Monge-Ampére equation. Our contribution is threefold. First, we provide expository material on this classic linearization of Wasserstein distance including a quantitative error estimate. Second, we reduce the computational problem to solving an elliptic boundary value problem involving the Witten Laplacian, which is a Schrödinger operator of the form $H = -Δ+ V$, and describe an associated embedding. Third, for the case of probability distributions on the unit square $[0,1]^2$ represented by $n \times n$ arrays we present a fast code demonstrating our approach. Several numerical examples are presented.
△ Less
Submitted 1 March, 2022; v1 submitted 31 January, 2022;
originally announced January 2022.
-
A common variable minimax theorem for graphs
Authors:
Ronald R. Coifman,
Nicholas F. Marshall,
Stefan Steinerberger
Abstract:
Let $\mathcal{G} = \{G_1 = (V, E_1), \dots, G_m = (V, E_m)\}$ be a collection of $m$ graphs defined on a common set of vertices $V$ but with different edge sets $E_1, \dots, E_m$. Informally, a function $f :V \rightarrow \mathbb{R}$ is smooth with respect to $G_k = (V,E_k)$ if $f(u) \sim f(v)$ whenever $(u, v) \in E_k$. We study the problem of understanding whether there exists a nonconstant funct…
▽ More
Let $\mathcal{G} = \{G_1 = (V, E_1), \dots, G_m = (V, E_m)\}$ be a collection of $m$ graphs defined on a common set of vertices $V$ but with different edge sets $E_1, \dots, E_m$. Informally, a function $f :V \rightarrow \mathbb{R}$ is smooth with respect to $G_k = (V,E_k)$ if $f(u) \sim f(v)$ whenever $(u, v) \in E_k$. We study the problem of understanding whether there exists a nonconstant function that is smooth with respect to all graphs in $\mathcal{G}$, simultaneously, and how to find it if it exists.
△ Less
Submitted 30 July, 2021;
originally announced July 2021.
-
Randomized mixed Hölder function approximation in higher-dimensions
Authors:
Nicholas F. Marshall
Abstract:
The purpose of this paper is to extend the result of arXiv:1810.00823 to mixed Hölder functions on $[0,1]^d$ for all $d \ge 1$. In particular, we prove that by sampling an $α$-mixed Hölder function $f : [0,1]^d \rightarrow \mathbb{R}$ at $\sim \frac{1}{\varepsilon} \left(\log \frac{1}{\varepsilon} \right)^d$ independent uniformly random points from $[0,1]^d$, we can construct an approximation…
▽ More
The purpose of this paper is to extend the result of arXiv:1810.00823 to mixed Hölder functions on $[0,1]^d$ for all $d \ge 1$. In particular, we prove that by sampling an $α$-mixed Hölder function $f : [0,1]^d \rightarrow \mathbb{R}$ at $\sim \frac{1}{\varepsilon} \left(\log \frac{1}{\varepsilon} \right)^d$ independent uniformly random points from $[0,1]^d$, we can construct an approximation $\tilde{f}$ such that $$ \|f - \tilde{f}\|_{L^2} \lesssim \varepsilon^α\left(\log \textstyle{\frac{1}{\varepsilon}} \right)^{d-1/2}, $$ with high probability.
△ Less
Submitted 9 October, 2019;
originally announced October 2019.
-
A fast simple algorithm for computing the potential of charges on a line
Authors:
Zydrunas Gimbutas,
Nicholas F. Marshall,
Vladimir Rokhlin
Abstract:
We present a fast method for evaluating expressions of the form $$ u_j = \sum_{i = 1,i \not = j}^n \frac{α_i}{x_i - x_j}, \quad \text{for} \quad j = 1,\ldots,n, $$ where $α_i$ are real numbers, and $x_i$ are points in a compact interval of $\mathbb{R}$. This expression can be viewed as representing the electrostatic potential generated by charges on a line in $\mathbb{R}^3$. While fast algorithms…
▽ More
We present a fast method for evaluating expressions of the form $$ u_j = \sum_{i = 1,i \not = j}^n \frac{α_i}{x_i - x_j}, \quad \text{for} \quad j = 1,\ldots,n, $$ where $α_i$ are real numbers, and $x_i$ are points in a compact interval of $\mathbb{R}$. This expression can be viewed as representing the electrostatic potential generated by charges on a line in $\mathbb{R}^3$. While fast algorithms for computing the electrostatic potential of general distributions of charges in $\mathbb{R}^3$ exist, in a number of situations in computational physics it is useful to have a simple and extremely fast method for evaluating the potential of charges on a line; we present such a method in this paper, and report numerical results for several examples.
△ Less
Submitted 25 November, 2019; v1 submitted 8 July, 2019;
originally announced July 2019.
-
A Cheeger inequality for graphs based on a reflection principle
Authors:
Edward Gelernt,
Diana Halikias,
Charles Kenney,
Nicholas F. Marshall
Abstract:
Given a graph with a designated set of boundary vertices, we define a new notion of a Neumann Laplace operator on a graph using a reflection principle. We show that the first eigenvalue of this Neumann graph Laplacian satisfies a Cheeger inequality.
Given a graph with a designated set of boundary vertices, we define a new notion of a Neumann Laplace operator on a graph using a reflection principle. We show that the first eigenvalue of this Neumann graph Laplacian satisfies a Cheeger inequality.
△ Less
Submitted 26 May, 2020; v1 submitted 18 February, 2019;
originally announced February 2019.
-
Approximating mixed Hölder functions using random samples
Authors:
Nicholas F. Marshall
Abstract:
Suppose $f : [0,1]^2 \rightarrow \mathbb{R}$ is a $(c,α)$-mixed Hölder function that we sample at $l$ points $X_1,\ldots,X_l$ chosen uniformly at random from the unit square. Let the location of these points and the function values $f(X_1),\ldots,f(X_l)$ be given. If $l \ge c_1 n \log^2 n$, then we can compute an approximation $\tilde{f}$ such that…
▽ More
Suppose $f : [0,1]^2 \rightarrow \mathbb{R}$ is a $(c,α)$-mixed Hölder function that we sample at $l$ points $X_1,\ldots,X_l$ chosen uniformly at random from the unit square. Let the location of these points and the function values $f(X_1),\ldots,f(X_l)$ be given. If $l \ge c_1 n \log^2 n$, then we can compute an approximation $\tilde{f}$ such that $$ \|f - \tilde{f} \|_{L^2} = \mathcal{O}(n^{-α} \log^{3/2} n), $$ with probability at least $1 - n^{2 -c_1}$, where the implicit constant only depends on the constants $c > 0$ and $c_1 > 0$.
△ Less
Submitted 12 August, 2019; v1 submitted 1 October, 2018;
originally announced October 2018.
-
Manifold learning with bi-stochastic kernels
Authors:
Nicholas F. Marshall,
Ronald R. Coifman
Abstract:
In this paper we answer the following question: what is the infinitesimal generator of the diffusion process defined by a kernel that is normalized such that it is bi-stochastic with respect to a specified measure? More precisely, under the assumption that data is sampled from a Riemannian manifold we determine how the resulting infinitesimal generator depends on the potentially nonuniform distrib…
▽ More
In this paper we answer the following question: what is the infinitesimal generator of the diffusion process defined by a kernel that is normalized such that it is bi-stochastic with respect to a specified measure? More precisely, under the assumption that data is sampled from a Riemannian manifold we determine how the resulting infinitesimal generator depends on the potentially nonuniform distribution of the sample points, and the specified measure for the bi-stochastic normalization. In a special case, we demonstrate a connection to the heat kernel. We consider both the case where only a single data set is given, and the case where a data set and a reference set are given. The spectral theory of the constructed operators is studied, and Nyström extension formulas for the gradients of the eigenfunctions are computed. Applications to discrete point sets and manifold learning are discussed.
△ Less
Submitted 26 February, 2018; v1 submitted 17 November, 2017;
originally announced November 2017.
-
Stretching convex domains to capture many lattice points
Authors:
Nicholas F. Marshall
Abstract:
We consider an optimal stretching problem for strictly convex domains in $\mathbb{R}^d$ that are symmetric with respect to each coordinate hyperplane, where stretching refers to transformation by a diagonal matrix of determinant $1$. Specifically, we prove that the stretched convex domain which captures the most positive lattice points in the large volume limit is balanced: the $(d-1)$-dimensional…
▽ More
We consider an optimal stretching problem for strictly convex domains in $\mathbb{R}^d$ that are symmetric with respect to each coordinate hyperplane, where stretching refers to transformation by a diagonal matrix of determinant $1$. Specifically, we prove that the stretched convex domain which captures the most positive lattice points in the large volume limit is balanced: the $(d-1)$-dimensional measures of the intersections of the domain with each coordinate hyperplane are equal. Our results extend those of Antunes & Freitas, van den Berg, Bucur & Gittins, Ariturk & Laugesen, van den Berg & Gittins, and Gittins & Larson. The approach is motivated by the Fourier analysis techniques used to prove the classical $\#\{(i,j) \in \mathbb{Z}^2 : i^2 +j^2 \le r^2 \} =πr^2 + \mathcal{O}(r^{2/3})$ result for the Gauss circle problem.
△ Less
Submitted 28 March, 2018; v1 submitted 1 July, 2017;
originally announced July 2017.
-
Triangles capturing many lattice points
Authors:
Nicholas F. Marshall,
Stefan Steinerberger
Abstract:
We study a combinatorial problem that recently arose in the context of shape optimization: among all triangles with vertices $(0,0)$, $(x,0)$, and $(0,y)$ and fixed area, which one encloses the most lattice points from $\mathbb{Z}_{>0}^2$? Moreover, does its shape necessarily converge to the isosceles triangle $(x=y)$ as the area becomes large? Laugesen and Liu suggested that, in contrast to simil…
▽ More
We study a combinatorial problem that recently arose in the context of shape optimization: among all triangles with vertices $(0,0)$, $(x,0)$, and $(0,y)$ and fixed area, which one encloses the most lattice points from $\mathbb{Z}_{>0}^2$? Moreover, does its shape necessarily converge to the isosceles triangle $(x=y)$ as the area becomes large? Laugesen and Liu suggested that, in contrast to similar problems, there might not be a limiting shape. We prove that the limiting set is indeed nontrivial and contains infinitely many elements. We also show that there exist `bad' areas where no triangle is particularly good at capturing lattice points and show that there exists an infinite set of slopes $y/x$ such that any associated triangle captures more lattice points than any other fixed triangle for infinitely many (and arbitrarily large) areas; this set of slopes is a fractal subset of $[1/3, 3]$ and has Minkowski dimension at most $3/4$.
△ Less
Submitted 9 February, 2018; v1 submitted 13 June, 2017;
originally announced June 2017.
-
The Stability of the First Neumann Laplacian Eigenfunction Under Domain Deformations and Applications
Authors:
Nicholas F. Marshall
Abstract:
The robustness of manifold learning methods is often predicated on the stability of the Neumann Laplacian eigenfunctions under deformations of the assumed underlying domain. Indeed, many manifold learning methods are based on approximating the Neumann Laplacian eigenfunctions on a manifold that is assumed to underlie data, which is viewed through a source of distortion. In this paper, we study the…
▽ More
The robustness of manifold learning methods is often predicated on the stability of the Neumann Laplacian eigenfunctions under deformations of the assumed underlying domain. Indeed, many manifold learning methods are based on approximating the Neumann Laplacian eigenfunctions on a manifold that is assumed to underlie data, which is viewed through a source of distortion. In this paper, we study the stability of the first Neumann Laplacian eigenfunction with respect to deformations of a domain by a diffeomorphism. In particular, we are interested in the stability of the first eigenfunction on tall thin domains where, intuitively, the first Neumann Laplacian eigenfunction should only depend on the length along the domain. We prove a rigorous version of this statement and apply it to a machine learning problem in geophysical interpretation.
△ Less
Submitted 16 December, 2018; v1 submitted 10 April, 2017;
originally announced April 2017.
-
Time Coupled Diffusion Maps
Authors:
Nicholas F. Marshall,
Matthew J. Hirn
Abstract:
We consider a collection of $n$ points in $\mathbb{R}^d$ measured at $m$ times, which are encoded in an $n \times d \times m$ data tensor. Our objective is to define a single embedding of the $n$ points into Euclidean space which summarizes the geometry as described by the data tensor. In the case of a fixed data set, diffusion maps (and related graph Laplacian methods) define such an embedding vi…
▽ More
We consider a collection of $n$ points in $\mathbb{R}^d$ measured at $m$ times, which are encoded in an $n \times d \times m$ data tensor. Our objective is to define a single embedding of the $n$ points into Euclidean space which summarizes the geometry as described by the data tensor. In the case of a fixed data set, diffusion maps (and related graph Laplacian methods) define such an embedding via the eigenfunctions of a diffusion operator constructed on the data. Given a sequence of $m$ measurements of $n$ points, we construct a corresponding sequence of diffusion operators and study their product. Via this product, we introduce the notion of time coupled diffusion distance and time coupled diffusion maps which have natural geometric and probabilistic interpretations. To frame our method in the context of manifold learning, we model evolving data as samples from an underlying manifold with a time dependent metric, and we describe a connection of our method to the heat equation over a manifold with time dependent metric.
△ Less
Submitted 13 November, 2017; v1 submitted 11 August, 2016;
originally announced August 2016.
