-
Uniform Approximation by Polynomials with Integer Coefficients via the Bernstein Lattice
Authors:
C. Sinan Güntürk,
Weilin Li
Abstract:
Let $\mathscr{C}_\mathbb{Z}([0,1])$ be the metric space of real-valued continuous functions on $[0,1]$ with integer values at $0$ and $1$, equipped with the uniform (supremum) metric $d_\infty$. It is a classical theorem in approximation theory that the ring $\mathbb{Z}[X]$ of polynomials with integer coefficients, when considered as a set of functions on $[0,1]$, is dense in…
▽ More
Let $\mathscr{C}_\mathbb{Z}([0,1])$ be the metric space of real-valued continuous functions on $[0,1]$ with integer values at $0$ and $1$, equipped with the uniform (supremum) metric $d_\infty$. It is a classical theorem in approximation theory that the ring $\mathbb{Z}[X]$ of polynomials with integer coefficients, when considered as a set of functions on $[0,1]$, is dense in $\mathscr{C}_\mathbb{Z}([0,1])$. In this paper, we offer a strengthening of this result by identifying a substantially small subset $\bigcup_n \mathscr{B}_n$ of $\mathbb{Z}[X]$ which is still dense in $\mathscr{C}_\mathbb{Z}([0,1])$. Here $\mathscr{B}_n$, which we call the ``Bernstein lattice,'' is the lattice generated by the polynomials $$p_{n,k}(x) := \binom{n}{k} x^k(1-x)^{n-k}, ~~k=0,\dots,n.$$ Quantitatively, we show that for any $f \in \mathscr{C}_\mathbb{Z}([0,1])$, $$d_\infty(f, \mathscr{B}_n) \leq \frac{9}{4} ω_f(n^{-1/3}) + 2 n^{-1/3}, ~~n \geq 1,$$ where $ω_f$ stands for the modulus of continuity of $f$. We also offer a more general bound which can be optimized to yield better decay of approximation error for specific classes of continuous functions.
△ Less
Submitted 17 November, 2023;
originally announced November 2023.
-
Approximation with one-bit polynomials in Bernstein form
Authors:
C. Sinan Güntürk,
Weilin Li
Abstract:
We prove various theorems on approximation using polynomials with integer coefficients in the Bernstein basis of any given order. In the extreme, we draw the coefficients from $\{ \pm 1\}$ only. A basic case of our results states that for any Lipschitz function $f:[0,1] \to [-1,1]$ and for any positive integer $n$, there are signs $σ_0,\dots,σ_n \in \{\pm 1\}$ such that…
▽ More
We prove various theorems on approximation using polynomials with integer coefficients in the Bernstein basis of any given order. In the extreme, we draw the coefficients from $\{ \pm 1\}$ only. A basic case of our results states that for any Lipschitz function $f:[0,1] \to [-1,1]$ and for any positive integer $n$, there are signs $σ_0,\dots,σ_n \in \{\pm 1\}$ such that $$\left |f(x) - \sum_{k=0}^n σ_k \, \binom{n}{k} x^k (1-x)^{n-k} \right | \leq \frac{C (1+|f|_{\mathrm{Lip}})}{1+\sqrt{nx(1-x)}} ~\mbox{ for all } x \in [0,1].$$ More generally, we show that higher accuracy is achievable for smoother functions: For any integer $s\geq 1$, if $f$ has a Lipschitz $(s{-}1)$st derivative, then approximation accuracy of order $O(n^{-s/2})$ is achievable with coefficients in $\{\pm 1\}$ provided $\|f \|_\infty < 1$, and of order $O(n^{-s})$ with unrestricted integer coefficients, both uniformly on closed subintervals of $(0,1)$ as above. Hence these polynomial approximations are not constrained by the saturation of classical Bernstein polynomials. Our approximations are constructive and can be implemented using feedforward neural networks whose weights are chosen from $\{\pm 1\}$ only.
△ Less
Submitted 6 December, 2022; v1 submitted 16 December, 2021;
originally announced December 2021.
-
Approximation of functions with one-bit neural networks
Authors:
C. Sinan Güntürk,
Weilin Li
Abstract:
The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the approximation capabilities of one-bit neural networks -- those whose nonzero parameters are $\pm a$ for some fixed $a\not=0$. One of our main theorems shows th…
▽ More
The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the approximation capabilities of one-bit neural networks -- those whose nonzero parameters are $\pm a$ for some fixed $a\not=0$. One of our main theorems shows that for any $f\in C^s([0,1]^d)$ with $\|f\|_\infty<1$ and error $\varepsilon$, there is a $f_{NN}$ such that $|f(\boldsymbol{x})-f_{NN}(\boldsymbol{x})|\leq \varepsilon$ for all $\boldsymbol{x}$ away from the boundary of $[0,1]^d$, and $f_{NN}$ is either implementable by a $\{\pm 1\}$ quadratic network with $O(\varepsilon^{-2d/s})$ parameters or a $\{\pm \frac 1 2 \}$ ReLU network with $O(\varepsilon^{-2d/s}\log (1/\varepsilon))$ parameters, as $\varepsilon\to0$. We establish new approximation results for iterated multivariate Bernstein operators, error estimates for noise-shaping quantization on the Bernstein basis, and novel implementation of the Bernstein polynomials by one-bit quadratic and ReLU neural networks.
△ Less
Submitted 16 March, 2023; v1 submitted 16 December, 2021;
originally announced December 2021.
-
Unrestricted iterations of relaxed projections in Hilbert space: Regularity, absolute convergence, and statistics of displacements
Authors:
C. Sinan Güntürk,
Nguyen T. Thao
Abstract:
Given a finite collection $\mathbf{V}:=(V_1,\dots,V_N)$ of closed linear subspaces of a real Hilbert space $H$, let $P_i$ denote the orthogonal projection operator onto $V_i$ and $P_{i,λ}:= (1-λ)I + λP_i$ denote its relaxation with parameter $λ\in [0,2]$, $i=1,\dots,N$. Under a mild regularity assumption on $\mathbf{V}$ known as `innate regularity' (which, for example, is always satisfied if each…
▽ More
Given a finite collection $\mathbf{V}:=(V_1,\dots,V_N)$ of closed linear subspaces of a real Hilbert space $H$, let $P_i$ denote the orthogonal projection operator onto $V_i$ and $P_{i,λ}:= (1-λ)I + λP_i$ denote its relaxation with parameter $λ\in [0,2]$, $i=1,\dots,N$. Under a mild regularity assumption on $\mathbf{V}$ known as `innate regularity' (which, for example, is always satisfied if each $V_i$ has finite dimension or codimension), we show that all trajectories $(x_n)_{0}^\infty$ resulting from the iteration $x_{n+1} := P_{i_n,λ_n}(x_n)$, where the $i_n$ and the $λ_n$ are unrestricted other than the assumption that $\{λ_n : n \in \mathbb{N}\} \subset [η,2{-}η]$ for some $η\in (0,1]$, possess uniformly bounded displacement moments of arbitrarily small orders. In particular, we show that $$ \sum_{n=0}^\infty \|x_{n+1} - x_n \|^γ\leq C \|x_0\|^γ~\mbox{ for all }~ γ> 0,$$ where $C:=C(\mathbf{V},η,γ)<\infty$. This result strengthens prior results on norm convergence of these trajectories, known to hold under the same regularity assumption. For example, with $γ=1$, it follows that the displacements series $\sum (x_{n+1}-x_n)$ converges absolutely in $H$.
Quantifying the constant $C(\mathbf{V},η,γ)$, we also derive an effective bound on the distribution function of the norms of the displacements (normalized by the norm of the initial condition) which yields a root-exponential type decay bound on their decreasing rearrangement, again uniformly for all trajectories.
△ Less
Submitted 22 January, 2019;
originally announced January 2019.
-
Convergence of the randomized Kaczmarz method for phase retrieval
Authors:
Halyun Jeong,
C. Sinan Güntürk
Abstract:
The classical Kaczmarz iteration and its randomized variants are popular tools for fast inversion of linear overdetermined systems. This method extends naturally to the setting of the phase retrieval problem via substituting at each iteration the phase of any measurement of the available approximate solution for the unknown phase of the measurement of the true solution. Despite the simplicity of t…
▽ More
The classical Kaczmarz iteration and its randomized variants are popular tools for fast inversion of linear overdetermined systems. This method extends naturally to the setting of the phase retrieval problem via substituting at each iteration the phase of any measurement of the available approximate solution for the unknown phase of the measurement of the true solution. Despite the simplicity of the method, rigorous convergence guarantees that are available for the classical linear setting have not been established so far for the phase retrieval setting. In this short note, we provide a convergence result for the randomized Kaczmarz method for phase retrieval in $\mathbb{R}^d$. We show that with high probability a random measurement system of size $m \asymp d$ will be admissible for this method in the sense that convergence in the mean square sense is guaranteed with any prescribed probability. The convergence is exponential and comparable to the linear setting.
△ Less
Submitted 24 July, 2017; v1 submitted 30 June, 2017;
originally announced June 2017.
-
Noise-shaping Quantization Methods for Frame-based and Compressive Sampling Systems
Authors:
Evan Chou,
C. Sinan Güntürk,
Felix Krahmer,
Rayan Saab,
Özgür Yılmaz
Abstract:
Noise shaping refers to an analog-to-digital conversion methodology in which quantization error is arranged to lie mostly outside the signal spectrum by means of oversampling and feedback. Recently it has been successfully applied to more general redundant linear sampling and reconstruction systems associated with frames as well as non-linear systems associated with compressive sampling. This chap…
▽ More
Noise shaping refers to an analog-to-digital conversion methodology in which quantization error is arranged to lie mostly outside the signal spectrum by means of oversampling and feedback. Recently it has been successfully applied to more general redundant linear sampling and reconstruction systems associated with frames as well as non-linear systems associated with compressive sampling. This chapter reviews some of the recent progress in this subject.
△ Less
Submitted 20 February, 2015;
originally announced February 2015.
-
An Optimal Family of Exponentially Accurate One-Bit Sigma-Delta Quantization Schemes
Authors:
Percy Deift,
C. Sinan Güntürk,
Felix Krahmer
Abstract:
Sigma-Delta modulation is a popular method for analog-to-digital conversion of bandlimited signals that employs coarse quantization coupled with oversampling. The standard mathematical model for the error analysis of the method measures the performance of a given scheme by the rate at which the associated reconstruction error decays as a function of the oversampling ratio $λ$. It was recently sh…
▽ More
Sigma-Delta modulation is a popular method for analog-to-digital conversion of bandlimited signals that employs coarse quantization coupled with oversampling. The standard mathematical model for the error analysis of the method measures the performance of a given scheme by the rate at which the associated reconstruction error decays as a function of the oversampling ratio $λ$. It was recently shown that exponential accuracy of the form $O(2^{-rλ})$ can be achieved by appropriate one-bit Sigma-Delta modulation schemes. By general information-entropy arguments $r$ must be less than 1. The current best known value for $r$ is approximately 0.088. The schemes that were designed to achieve this accuracy employ the "greedy" quantization rule coupled with feedback filters that fall into a class we call "minimally supported". In this paper, we study the minimization problem that corresponds to optimizing the error decay rate for this class of feedback filters. We solve a relaxed version of this problem exactly and provide explicit asymptotics of the solutions. From these relaxed solutions, we find asymptotically optimal solutions of the original problem, which improve the best known exponential error decay rate to $r \approx 0.102$. Our method draws from the theory of orthogonal polynomials; in particular, it relates the optimal filters to the zero sets of Chebyshev polynomials of the second kind.
△ Less
Submitted 27 January, 2010;
originally announced January 2010.
-
Iteratively re-weighted least squares minimization for sparse recovery
Authors:
Ingrid Daubechies,
Ronald DeVore,
Massimo Fornasier,
C. Sinan Gunturk
Abstract:
We analyze an Iteratively Re-weighted Least Squares (IRLS) algorithm for promoting l1-minimization in sparse and compressible vector recovery. We prove its convergence and we estimate its local rate. We show how the algorithm can be modified in order to promote lt-minimization for t<1, and how this modification produces superlinear rates of convergence.
We analyze an Iteratively Re-weighted Least Squares (IRLS) algorithm for promoting l1-minimization in sparse and compressible vector recovery. We prove its convergence and we estimate its local rate. We show how the algorithm can be modified in order to promote lt-minimization for t<1, and how this modification produces superlinear rates of convergence.
△ Less
Submitted 3 July, 2008;
originally announced July 2008.
-
Approximation by power series with $\pm 1$ coefficients
Authors:
C. Sinan Gunturk
Abstract:
In this paper we construct certain type of near-optimal approximations of a class of analytic functions in the unit disc by power series with two distinct coefficients. More precisely, we show that if all the coefficients of the power series f(z) are real and lie in [-a,a] where a < 1, then there exists a power series Q(z) with coefficients in {-1,+1} such that |f(z)-Q(z)| approaches 0 at the ra…
▽ More
In this paper we construct certain type of near-optimal approximations of a class of analytic functions in the unit disc by power series with two distinct coefficients. More precisely, we show that if all the coefficients of the power series f(z) are real and lie in [-a,a] where a < 1, then there exists a power series Q(z) with coefficients in {-1,+1} such that |f(z)-Q(z)| approaches 0 at the rate exp(-C/|1-z|) as z approaches 1 non-tangentially inside the unit disc. A result by Borwein-Erdelyi-Kos shows that this type of decay rate is best possible. The special case f=0 yields a near-optimal solution to the "fair duel" problem of Konyagin.
△ Less
Submitted 18 March, 2005;
originally announced March 2005.
-
A new upper bound for finite additive bases
Authors:
Sinan Gunturk,
Melvyn B. Nathanson
Abstract:
Let n(2,k) denote the largest integer n for which there exists a set A of k nonnegative integers such that the sumset 2A contains {0,1,2,...,n-1}. A classical problem in additive number theory is to find an upper bound for n(2,k). In this paper it is proved that limsup_{k\to\infty} n(2,k)/k^2 \leq 0.4789.
Let n(2,k) denote the largest integer n for which there exists a set A of k nonnegative integers such that the sumset 2A contains {0,1,2,...,n-1}. A classical problem in additive number theory is to find an upper bound for n(2,k). In this paper it is proved that limsup_{k\to\infty} n(2,k)/k^2 \leq 0.4789.
△ Less
Submitted 13 March, 2005;
originally announced March 2005.
-
Ergodic Dynamics in Sigma-Delta Quantization: Tiling Invariant Sets and Spectral Analysis of Error
Authors:
Nguyen T. Thao,
C. Sinan Gunturk
Abstract:
This paper has two themes that are intertwined: The first is the dynamics of certain piecewise affine maps on the Euclidean space that arise from a class of analog-to-digital conversion methods called Sigma-Delta quantization. The second is the analysis of reconstruction error associated to each such method. Sigma-Delta quantization generates approximate representations of functions by sequences…
▽ More
This paper has two themes that are intertwined: The first is the dynamics of certain piecewise affine maps on the Euclidean space that arise from a class of analog-to-digital conversion methods called Sigma-Delta quantization. The second is the analysis of reconstruction error associated to each such method. Sigma-Delta quantization generates approximate representations of functions by sequences that lie in a restricted set of discrete values. These are special sequences in that their local averages track the function values closely, thus enabling simple convolutional reconstruction. In this paper, we are concerned with the approximation of constant functions only, a basic case that presents surprisingly complex behavior. An m'th order Sigma-Delta scheme with input x can be translated into a dynamical system that produces a discrete-valued sequence (in particular, a 0-1 sequence) q as its output. When the schemes are stable, we show that the underlying piecewise affine maps possess invariant sets that tile the Euclidean space up to a finite multiplicity. When this multiplicity is one (the single-tile case), the dynamics within the tile is isomorphic to that of a generalized skew translation on the m dimensional torus. The value of x can be approximated using any consecutive M elements in q with increasing accuracy in M. We show that the asymptotical behavior of reconstruction error depends on the regularity of the invariant sets, the order m, and some arithmetic properties of x. We determine the behavior in a number of cases of practical interest and provide good upper bounds in some other cases when exact analysis is not yet available.
△ Less
Submitted 24 August, 2003;
originally announced August 2003.
