-
How many continuous measurements are needed to learn a vector?
Authors:
David Krieg,
Erich Novak,
Mario Ullrich
Abstract:
One can recover vectors from $\mathbb{R}^m$ with arbitrary precision, using only $\lceil \log_2(m+1)\rceil +1$ continuous measurements that are chosen adaptively. This surprising result is explained and discussed, and we present applications to infinite-dimensional approximation problems.
One can recover vectors from $\mathbb{R}^m$ with arbitrary precision, using only $\lceil \log_2(m+1)\rceil +1$ continuous measurements that are chosen adaptively. This surprising result is explained and discussed, and we present applications to infinite-dimensional approximation problems.
△ Less
Submitted 9 December, 2024;
originally announced December 2024.
-
On the power of adaption and randomization
Authors:
David Krieg,
Erich Novak,
Mario Ullrich
Abstract:
We present bounds between different widths of convex subsets of Banach spaces, including Gelfand and Bernstein widths. Using this, and some relations between widths and minimal errors, we obtain bounds on the maximal gain of adaptive and randomized algorithms over non-adaptive, deterministic ones for approximating linear operators on convex sets. Our results also apply to the approximation of embe…
▽ More
We present bounds between different widths of convex subsets of Banach spaces, including Gelfand and Bernstein widths. Using this, and some relations between widths and minimal errors, we obtain bounds on the maximal gain of adaptive and randomized algorithms over non-adaptive, deterministic ones for approximating linear operators on convex sets. Our results also apply to the approximation of embeddings into the space of bounded functions based on function evaluations, i.e., to sampling recovery in the uniform norm. We conclude with a list of open problems.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Conditions for tractability of the weighted $L_p$-discrepancy and integration in non-homogeneous tensor product spaces
Authors:
Erich Novak,
Friedrich Pillichshammer
Abstract:
We study tractability properties of the weighted $L_p$-discrepancy. The concept of {\it weighted} discrepancy was introduced by Sloan and Woź\-nia\-kowski in 1998 in order to prove a weighted version of the Koksma-Hlawka inequality for the error of quasi-Monte Carlo integration rules. The weights have the aim to model the influence of different coordinates of integrands on the error. A discrepancy…
▽ More
We study tractability properties of the weighted $L_p$-discrepancy. The concept of {\it weighted} discrepancy was introduced by Sloan and Woź\-nia\-kowski in 1998 in order to prove a weighted version of the Koksma-Hlawka inequality for the error of quasi-Monte Carlo integration rules. The weights have the aim to model the influence of different coordinates of integrands on the error. A discrepancy is said to be tractable if the information complexity, i.e., the minimal number $N$ of points such that the discrepancy is less than the initial discrepancy times an error threshold $\varepsilon$, does not grow exponentially fast with the dimension. In this case there are various notions of tractabilities used in order to classify the exact rate.
For even integer parameters $p$ there are sufficient conditions on the weights available in literature, which guarantee the one or other notion of tractability. In the present paper we prove matching sufficient conditions (upper bounds) and neccessary conditions (lower bounds) for polynomial and weak tractability for all $p \in (1, \infty)$.
The proofs of the lower bounds are based on a general result for the information complexity of integration with positive quadrature formulas for tensor product spaces. In order to demonstrate this lower bound we consider as a second application the integration of tensor products of polynomials of degree at most 2.
△ Less
Submitted 21 May, 2024;
originally announced May 2024.
-
Intractability results for integration in tensor product spaces
Authors:
Erich Novak,
Friedrich Pillichshammer
Abstract:
We study lower bounds on the worst-case error of numerical integration in tensor product spaces. As reference we use the $N$-th minimal error of linear rules that use $N$ function values. The information complexity is the minimal number $N$ of function evaluations that is necessary such that the $N$-th minimal error is less than a factor $\varepsilon$ times the initial error. We are interested to…
▽ More
We study lower bounds on the worst-case error of numerical integration in tensor product spaces. As reference we use the $N$-th minimal error of linear rules that use $N$ function values. The information complexity is the minimal number $N$ of function evaluations that is necessary such that the $N$-th minimal error is less than a factor $\varepsilon$ times the initial error. We are interested to which extent the information complexity depends on the number $d$ of variables of the integrands. If the information complexity grows exponentially fast in $d$, then the integration problem is said to suffer from the curse of dimensionality.
Under the assumption of the existence of a worst-case function for the uni-variate problem we present two methods for providing good lower bounds on the information complexity. The first method is based on a suitable decomposition of the worst-case function. This method can be seen as a generalization of the method of decomposable reproducing kernels, that is often successfully applied when integration in Hilbert spaces with a reproducing kernel is studied. The second method, although only applicable for positive quadrature rules, has the advantage, that it does not require a suitable decomposition of the worst-case function. Rather, it is based on a spline approximation of the worst-case function and can be used for analytic functions.
The methods presented can be applied to problems beyond the Hilbert space setting. For demonstration purposes we apply them to several examples, notably to uniform integration over the unit-cube, weighted integration over the whole space, and integration of infinitely smooth functions over the cube. Some of these results have interesting consequences in discrepancy theory.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
The $L_p$-discrepancy for finite $p>1$ suffers from the curse of dimensionality
Authors:
Erich Novak,
Friedrich Pillichshammer
Abstract:
The $L_p$-discrepancy is a classical quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the number of points in $[0,1)^d$ that is required such that the minimal normalized $L_p$-discrepancy is less or equal $\varepsilon$. It is well known, that the inver…
▽ More
The $L_p$-discrepancy is a classical quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the number of points in $[0,1)^d$ that is required such that the minimal normalized $L_p$-discrepancy is less or equal $\varepsilon$. It is well known, that the inverse of $L_2$-discrepancy grows exponentially fast with the dimension $d$, i.e., we have the curse of dimensionality, whereas the inverse of $L_{\infty}$-discrepancy depends exactly linearly on $d$. The behavior of inverse of $L_p$-discrepancy for general $p \not\in \{2,\infty\}$ was an open problem since many years. Recently, the curse of dimensionality for the $L_p$-discrepancy was shown for an infinite sequence of values $p$ in $(1,2]$, but the general result seemed to be out of reach. In the present paper we show that the $L_p$-discrepancy suffers from the curse of dimensionality for all $p$ in $(1,\infty)$ and only the case $p=1$ is still open. This result follows from a more general result that we show for the worst-case error of positive quadrature formulas for an anchored Sobolev space of once differentiable functions in each variable whose first mixed derivative has finite $L_q$-norm, where $q$ is the Hölder conjugate of $p$.
△ Less
Submitted 26 March, 2024; v1 submitted 12 March, 2024;
originally announced March 2024.
-
Randomized approximation of summable sequences -- adaptive and non-adaptive
Authors:
Robert Kunsch,
Erich Novak,
Marcin Wnuk
Abstract:
We prove lower bounds for the randomized approximation of the embedding $\ell_1^m \rightarrow \ell_\infty^m$ based on algorithms that use arbitrary linear (hence non-adaptive) information provided by a (randomized) measurement matrix $N \in \mathbb{R}^{n \times m}$. These lower bounds reflect the increasing difficulty of the problem for $m \to \infty$, namely, a term $\sqrt{\log m}$ in the complex…
▽ More
We prove lower bounds for the randomized approximation of the embedding $\ell_1^m \rightarrow \ell_\infty^m$ based on algorithms that use arbitrary linear (hence non-adaptive) information provided by a (randomized) measurement matrix $N \in \mathbb{R}^{n \times m}$. These lower bounds reflect the increasing difficulty of the problem for $m \to \infty$, namely, a term $\sqrt{\log m}$ in the complexity $n$. This result implies that non-compact operators between arbitrary Banach spaces are not approximable using non-adaptive Monte Carlo methods. We also compare these lower bounds for non-adaptive methods with upper bounds based on adaptive, randomized methods for recovery for which the complexity $n$ only exhibits a $(\log\log m)$-dependence. In doing so we give an example of linear problems where the error for adaptive vs. non-adaptive Monte Carlo methods shows a gap of order $n^{1/2} ( \log n)^{-1/2}$.
△ Less
Submitted 23 May, 2024; v1 submitted 3 August, 2023;
originally announced August 2023.
-
Optimal Algorithms for Numerical Integration: Recent Results and Open Problems
Authors:
Erich Novak
Abstract:
We present recent results on optimal algorithms for numerical integration and several open problems. The paper has six parts:
1. Introduction
2. Lower Bounds
3. Universality
4. General Domains
5. iid Information
6. Concluding Remarks
We present recent results on optimal algorithms for numerical integration and several open problems. The paper has six parts:
1. Introduction
2. Lower Bounds
3. Universality
4. General Domains
5. iid Information
6. Concluding Remarks
△ Less
Submitted 13 July, 2023;
originally announced July 2023.
-
The curse of dimensionality for the $L_p$-discrepancy with finite $p$
Authors:
Erich Novak,
Friedrich Pillichshammer
Abstract:
The $L_p$-discrepancy is a quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube, which is closely related to the worst-case error of quasi-Monte Carlo algorithms for numerical integration. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the minimal number of points in $[0,1)^d$ such that the minimal norm…
▽ More
The $L_p$-discrepancy is a quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube, which is closely related to the worst-case error of quasi-Monte Carlo algorithms for numerical integration. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the minimal number of points in $[0,1)^d$ such that the minimal normalized $L_p$-discrepancy is less or equal $\varepsilon$. It is well known, that the inverse of $L_2$-discrepancy grows exponentially fast with the dimension $d$, i.e., we have the curse of dimensionality, whereas the inverse of $L_{\infty}$-discrepancy depends exactly linearly on $d$. The behavior of inverse of $L_p$-discrepancy for general $p \not\in \{2,\infty\}$ has been an open problem for many years. In this paper we show that the $L_p$-discrepancy suffers from the curse of dimensionality for all $p$ in $(1,2]$ which are of the form $p=2 \ell/(2 \ell -1)$ with $\ell \in \mathbb{N}$.
This result follows from a more general result that we show for the worst-case error of numerical integration in an anchored Sobolev space with anchor 0 of once differentiable functions in each variable whose first derivative has finite $L_q$-norm, where $q$ is an even positive integer satisfying $1/p+1/q=1$.
△ Less
Submitted 12 June, 2023; v1 submitted 3 March, 2023;
originally announced March 2023.
-
Lower bounds for integration and recovery in $L_2$
Authors:
Aicke Hinrichs,
David Krieg,
Erich Novak,
Jan Vybiral
Abstract:
Function values are, in some sense, "almost as good" as general linear information for $L_2$-approximation (optimal recovery, data assimilation) of functions from a reproducing kernel Hilbert space. This was recently proved by new upper bounds on the sampling numbers under the assumption that the singular values of the embedding of this Hilbert space into $L_2$ are square-summable. Here we mainly…
▽ More
Function values are, in some sense, "almost as good" as general linear information for $L_2$-approximation (optimal recovery, data assimilation) of functions from a reproducing kernel Hilbert space. This was recently proved by new upper bounds on the sampling numbers under the assumption that the singular values of the embedding of this Hilbert space into $L_2$ are square-summable. Here we mainly prove new lower bounds. In particular we prove that the sampling numbers behave worse than the approximation numbers for Sobolev spaces with small smoothness. Hence there can be a logarithmic gap also in the case where the singular numbers of the embedding are square-summable. We first prove new lower bounds for the integration problem, again for rather classical Sobolev spaces of periodic univariate functions.
△ Less
Submitted 22 March, 2022; v1 submitted 26 August, 2021;
originally announced August 2021.
-
Recovery of Sobolev functions restricted to iid sampling
Authors:
David Krieg,
Erich Novak,
Mathias Sonnleitner
Abstract:
We study $L_q$-approximation and integration for functions from the Sobolev space $W^s_p(Ω)$ and compare optimal randomized (Monte Carlo) algorithms with algorithms that can only use iid sample points, uniformly distributed on the domain. The main result is that we obtain the same optimal rate of convergence if we restrict to iid sampling, a common assumption in learning and uncertainty quantifica…
▽ More
We study $L_q$-approximation and integration for functions from the Sobolev space $W^s_p(Ω)$ and compare optimal randomized (Monte Carlo) algorithms with algorithms that can only use iid sample points, uniformly distributed on the domain. The main result is that we obtain the same optimal rate of convergence if we restrict to iid sampling, a common assumption in learning and uncertainty quantification. The only exception is when $p=q=\infty$, where a logarithmic loss cannot be avoided.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
-
Lower Bounds for the Error of Quadrature Formulas for Hilbert Spaces
Authors:
Aicke Hinrichs,
David Krieg,
Erich Novak,
Jan Vybíral
Abstract:
We prove lower bounds for the worst case error of quadrature formulas that use given sample points $\X_n = \{ x_1, \dots , x_n \}$. We are mainly interested in optimal point sets $\X_n$, but also prove lower bounds that hold with high probability for sets of independently and uniformly distributed points. As a tool, we use a recent result (and extensions thereof) of Vybíral on the positive semi-de…
▽ More
We prove lower bounds for the worst case error of quadrature formulas that use given sample points $\X_n = \{ x_1, \dots , x_n \}$. We are mainly interested in optimal point sets $\X_n$, but also prove lower bounds that hold with high probability for sets of independently and uniformly distributed points. As a tool, we use a recent result (and extensions thereof) of Vybíral on the positive semi-definiteness of certain matrices related to the product theorem of Schur. The new technique also works for spaces of analytic functions where known methods based on decomposable kernels cannot be applied.
△ Less
Submitted 7 December, 2020; v1 submitted 1 April, 2020;
originally announced April 2020.
-
Algorithms and Complexity for Functions on General Domains
Authors:
Erich Novak
Abstract:
Error bounds and complexity bounds in numerical analysis and information-based complexity are often proved for functions that are defined on very simple domains, such as a cube, a torus, or a sphere. We study optimal error bounds for the approximation or integration of functions defined on $D_d \subset R^d$ and only assume that $D_d$ is a bounded Lipschitz domain. Some results are even more genera…
▽ More
Error bounds and complexity bounds in numerical analysis and information-based complexity are often proved for functions that are defined on very simple domains, such as a cube, a torus, or a sphere. We study optimal error bounds for the approximation or integration of functions defined on $D_d \subset R^d$ and only assume that $D_d$ is a bounded Lipschitz domain. Some results are even more general. We study three different concepts to measure the complexity: order of convergence, asymptotic constant, and explicit uniform bounds, i.e., bounds that hold for all $n$ (number of pieces of information) and all (normalized) domains. It is known for many problems that the order of convergence of optimal algorithms does not depend on the domain $D_d \subset R^d$. We present examples for which the following statements are true:
1) Also the asymptotic constant does not depend on the shape of $D_d$ or the imposed boundary values, it only depends on the volume of the domain.
2) There are explicit and uniform lower (or upper, respectively) bounds for the error that are only slightly smaller (or larger, respectively) than the asymptotic error bound.
△ Less
Submitted 13 January, 2020; v1 submitted 16 August, 2019;
originally announced August 2019.
-
On the power of random information
Authors:
Aicke Hinrichs,
David Krieg,
Erich Novak,
Joscha Prochno,
Mario Ullrich
Abstract:
We study approximation and integration problems and compare the quality of optimal information with the quality of random information. For some problems random information is almost optimal and for some other problems random information is much worse than optimal information. We prove new results and give a short survey of known results.
We study approximation and integration problems and compare the quality of optimal information with the quality of random information. For some problems random information is almost optimal and for some other problems random information is much worse than optimal information. We prove new results and give a short survey of known results.
△ Less
Submitted 2 March, 2019;
originally announced March 2019.
-
Random sections of ellipsoids and the power of random information
Authors:
Aicke Hinrichs,
David Krieg,
Erich Novak,
Joscha Prochno,
Mario Ullrich
Abstract:
We study the circumradius of the intersection of an $m$-dimensional ellipsoid $\mathcal E$ with semi-axes $σ_1\geq\dots\geq σ_m$ with random subspaces of codimension $n$. We find that, under certain assumptions on $σ$, this random radius $\mathcal{R}_n=\mathcal{R}_n(σ)$ is of the same order as the minimal such radius $σ_{n+1}$ with high probability. In other situations $\mathcal{R}_n$ is close to…
▽ More
We study the circumradius of the intersection of an $m$-dimensional ellipsoid $\mathcal E$ with semi-axes $σ_1\geq\dots\geq σ_m$ with random subspaces of codimension $n$. We find that, under certain assumptions on $σ$, this random radius $\mathcal{R}_n=\mathcal{R}_n(σ)$ is of the same order as the minimal such radius $σ_{n+1}$ with high probability. In other situations $\mathcal{R}_n$ is close to the maximum $σ_1$. The random variable $\mathcal{R}_n$ naturally corresponds to the worst-case error of the best algorithm based on random information for $L_2$-approximation of functions from a compactly embedded Hilbert space $H$ with unit ball $\mathcal E$. In particular, $σ_k$ is the $k$th largest singular value of the embedding $H\hookrightarrow L_2$. In this formulation, one can also consider the case $m=\infty$, and we prove that random information behaves very differently depending on whether $σ\in \ell_2$ or not. For $σ\notin \ell_2$ random information is completely useless, i.e., $\mathbb E[\mathcal{R}_n] = σ_1$. For $σ\in \ell_2$ the expected radius of random information tends to zero at least at rate $o(1/\sqrt{n})$ as $n\to\infty$. In the important case $σ_k \asymp k^{-α} \ln^{-β}(k+1)$, where $α> 0$ and $β\in\mathbb R$, we obtain that $$ \mathbb E [\mathcal{R}_n(σ)] \asymp \begin{cases} σ_1 & : α<1/2 \,\text{ or }\, β\leqα=1/2 \\ σ_n \, \sqrt{\ln(n+1)} & : β>α=1/2 \\ σ_{n+1} & : α>1/2. \end{cases} $$ In the proofs we use a comparison result for Gaussian processes à la Gordon, exponential estimates for sums of chi-squared random variables, and estimates for the extreme singular values of (structured) Gaussian random matrices. The upper bound is constructive. It is proven for the worst case error of a least squares estimator.
△ Less
Submitted 9 April, 2020; v1 submitted 20 January, 2019;
originally announced January 2019.
-
Solvable Integration Problems and Optimal Sample Size Selection
Authors:
Robert J. Kunsch,
Erich Novak,
Daniel Rudolf
Abstract:
We compute the integral of a function or the expectation of a random variable with minimal cost and use, for our new algorithm and for upper bounds of the complexity, i.i.d. samples. Under certain assumptions it is possible to select a sample size based on a variance estimation, or -- more generally -- based on an estimation of a (central absolute) $p$-moment. That way one can guarantee a small ab…
▽ More
We compute the integral of a function or the expectation of a random variable with minimal cost and use, for our new algorithm and for upper bounds of the complexity, i.i.d. samples. Under certain assumptions it is possible to select a sample size based on a variance estimation, or -- more generally -- based on an estimation of a (central absolute) $p$-moment. That way one can guarantee a small absolute error with high probability, the problem is thus called solvable. The expected cost of the method depends on the $p$-moment of the random variable, which can be arbitrarily large.
In order to prove the optimality of our algorithm we also provide lower bounds. These bounds apply not only to methods based on i.i.d. samples but also to general randomized algorithms. They show that -- up to constants -- the cost of the algorithm is optimal in terms of accuracy, confidence level, and norm of the particular input random variable. Since the considered classes of random variables or integrands are very large, the worst case cost would be infinite. Nevertheless one can define adaptive stopping rules such that for each input the expected cost is finite.
We contrast these positive results with examples of integration problems that are not solvable.
△ Less
Submitted 23 October, 2018; v1 submitted 22 May, 2018;
originally announced May 2018.
-
Reproducing Kernels of Sobolev Spaces on $\mathbb{R}^d$ and Applications to Embedding Constants and Tractability
Authors:
Erich Novak,
Mario Ullrich,
Henryk Woźniakowski,
Shun Zhang
Abstract:
The standard Sobolev space $W^s_2(\mathbb{R}^d)$, with arbitrary positive integers $s$ and $d$ for which $s>d/2$, has the reproducing kernel $$ K_{d,s}(x,t)=\int_{\mathbb{R}^d}\frac{\prod_{j=1}^d\cos\left(2π\,(x_j-t_j)u_j\right)} {1+\sum_{0<|α|_1\le s}\prod_{j=1}^d(2π\,u_j)^{2α_j}}\,{\rm
d}u $$ for all $x,t\in\mathbb{R}^d$, where $x_j,t_j,u_j,α_j$ are components of $d$-variate $x,t,u,α$, and…
▽ More
The standard Sobolev space $W^s_2(\mathbb{R}^d)$, with arbitrary positive integers $s$ and $d$ for which $s>d/2$, has the reproducing kernel $$ K_{d,s}(x,t)=\int_{\mathbb{R}^d}\frac{\prod_{j=1}^d\cos\left(2π\,(x_j-t_j)u_j\right)} {1+\sum_{0<|α|_1\le s}\prod_{j=1}^d(2π\,u_j)^{2α_j}}\,{\rm
d}u $$ for all $x,t\in\mathbb{R}^d$, where $x_j,t_j,u_j,α_j$ are components of $d$-variate $x,t,u,α$, and $|α|_1=\sum_{j=1}^dα_j$ with non-negative integers $α_j$. We obtain a more explicit form for the reproducing kernel $K_{1,s}$ and find a closed form for the kernel $K_{d, \infty}$.
Knowing the form of $K_{d,s}$, we present applications on the best embedding constants between the Sobolev space $W^s_2(\mathbb{R}^d)$ and $L_\infty(\mathbb{R}^d)$, and on strong polynomial tractability of integration with an arbitrary probability density. We prove that the best embedding constants are exponentially small in $d$, whereas worst case integration errors of algorithms using $n$ function values are also exponentially small in $d$ and decay at least like $n^{-1/2}$. This yields strong polynomial tractability in the worst case setting for the absolute error criterion.
△ Less
Submitted 8 September, 2017;
originally announced September 2017.
-
Tractability of multivariate problems for standard and linear information in the worst case setting: part II
Authors:
Henryk Woźniakowski,
Erich Novak
Abstract:
We study QPT (quasi-polynomial tractability) in the worst case setting for linear tensor product problems defined over Hilbert spaces. We assume that the domain space is a reproducing kernel Hilbert space so that function values are well defined. We prove QPT for algorithms that use only function values under the three assumptions:
1) the minimal errors for the univariate case decay polynomially…
▽ More
We study QPT (quasi-polynomial tractability) in the worst case setting for linear tensor product problems defined over Hilbert spaces. We assume that the domain space is a reproducing kernel Hilbert space so that function values are well defined. We prove QPT for algorithms that use only function values under the three assumptions:
1) the minimal errors for the univariate case decay polynomially fast to zero,
2) the largest singular value for the univariate case is simple and
3) the eigenfunction corresponding to the largest singular value is a multiple of the function value at some point.
The first two assumptions are necessary for QPT. The third assumption is necessary for QPT for some Hilbert spaces.
△ Less
Submitted 14 August, 2017;
originally announced August 2017.
-
Optimal Quadrature Formulas for the Sobolev Space $H^1$
Authors:
Erich Novak,
Shun Zhang
Abstract:
We study optimal quadrature formulas for arbitrary weighted integrals and integrands from the Sobolev space $H^1([0,1])$. We obtain general formulas for the worst case error depending on the nodes $x_j$. A particular case is the computation of Fourier coefficients, where the oscillatory weight is given by $ρ_k(x) = \exp(- 2 πi k x)$. Here we study the question whether equidistant nodes are optimal…
▽ More
We study optimal quadrature formulas for arbitrary weighted integrals and integrands from the Sobolev space $H^1([0,1])$. We obtain general formulas for the worst case error depending on the nodes $x_j$. A particular case is the computation of Fourier coefficients, where the oscillatory weight is given by $ρ_k(x) = \exp(- 2 πi k x)$. Here we study the question whether equidistant nodes are optimal or not. We prove that this depends on $n$ and $k$: equidistant nodes are optimal if $n \ge 2.7 |k| +1 $ but might be suboptimal for small $n$. In particular, the equidistant nodes $x_j = j/ |k|$ for $j=0, 1, \dots , |k| = n+1$ are the worst possible nodes and do not give any useful information. To characterize the worst case function we use certain results from the theory of weak solutions of boundary value problems and related quadratic extremal problems.
△ Less
Submitted 5 September, 2016;
originally announced September 2016.
-
Product rules are optimal for numerical integration in classical smoothness spaces
Authors:
Aicke Hinrichs,
Erich Novak,
Mario Ullrich,
Henryk Wozniakowski
Abstract:
We mainly study numerical integration of real valued functions defined on the $d$-dimensional unit cube with all partial derivatives up to some finite order $r\ge1$ bounded by one. It is well known that optimal algorithms that use $n$ function values achieve the error rate $n^{-r/d}$, where the hidden constant depends on $r$ and $d$. Here we prove explicit error bounds without hidden constants and…
▽ More
We mainly study numerical integration of real valued functions defined on the $d$-dimensional unit cube with all partial derivatives up to some finite order $r\ge1$ bounded by one. It is well known that optimal algorithms that use $n$ function values achieve the error rate $n^{-r/d}$, where the hidden constant depends on $r$ and $d$. Here we prove explicit error bounds without hidden constants and, in particular, show that the optimal order of the error is $\min \bigl\{1, d \, n^{-r/d}\bigr\}$, where now the hidden constant only depends on $r$, not on $d$. For $n=m^d$, this optimal order can be achieved by (tensor) product rules.
We also provide lower bounds for integration defined over an arbitrary open domain of volume one. We briefly discuss how lower bounds for integration may be applied for other problems such as multivariate approximation and optimization.
△ Less
Submitted 1 April, 2016;
originally announced April 2016.
-
The weight distribution of the self-dual $[128,64]$ polarity design code
Authors:
Masaaki Harada,
Ethan Novak,
Vladimir D. Tonchev
Abstract:
The weight distribution of the binary self-dual $[128,64]$ code being the extended code $C^{*}$ of the code $C$ spanned by the incidence vectors of the blocks of the polarity design in $PG(6,2)$ [11] is computed. It is shown also that $R(3,7)$ and $C^{*}$ have no self-dual $[128,64,d]$ neighbor with $d \in \{ 20, 24 \}$.
The weight distribution of the binary self-dual $[128,64]$ code being the extended code $C^{*}$ of the code $C$ spanned by the incidence vectors of the blocks of the polarity design in $PG(6,2)$ [11] is computed. It is shown also that $R(3,7)$ and $C^{*}$ have no self-dual $[128,64,d]$ neighbor with $d \in \{ 20, 24 \}$.
△ Less
Submitted 15 February, 2016;
originally announced February 2016.
-
Tractability of Multivariate Problems for Standard and Linear Information in the Worst Case Setting: Part I
Authors:
Erich Novak,
Henryk Wozniakowski
Abstract:
We present a lower error bound for approximating linear multivariate operators defined over Hilbert spaces in terms of the error bounds for appropriately constructed linear functionals as long as algorithms use function values. Furthermore, some of these linear functionals have the same norm as the linear operators. We then apply this error bound for linear (unweighted) tensor products. In this wa…
▽ More
We present a lower error bound for approximating linear multivariate operators defined over Hilbert spaces in terms of the error bounds for appropriately constructed linear functionals as long as algorithms use function values. Furthermore, some of these linear functionals have the same norm as the linear operators. We then apply this error bound for linear (unweighted) tensor products. In this way we use negative tractability results known for linear functionals to conclude the same negative results for linear operators. In particular, we prove that $L_2$-multivariate approximation defined for standard Sobolev space suffers the curse of dimensionality if function values are used although the curse is not present if linear functionals are allowed.
△ Less
Submitted 18 November, 2015;
originally announced November 2015.
-
Complexity of Oscillatory Integrals on the Real Line
Authors:
Erich Novak,
Mario Ullrich,
Henryk Woźniakowski,
Shun Zhang
Abstract:
We analyze univariate oscillatory integrals defined on the real line for functions from the standard Sobolev space $H^s({\mathbb{R}})$ and from the space $C^s({\mathbb{R}})$ with an arbitrary integer $s\ge1$. We find tight upper and lower bounds for the worst case error of optimal algorithms that use $n$ function values. More specifically, we study integrals of the form \[ I_k^ρ(f) = \int_{ {\math…
▽ More
We analyze univariate oscillatory integrals defined on the real line for functions from the standard Sobolev space $H^s({\mathbb{R}})$ and from the space $C^s({\mathbb{R}})$ with an arbitrary integer $s\ge1$. We find tight upper and lower bounds for the worst case error of optimal algorithms that use $n$ function values. More specifically, we study integrals of the form \[ I_k^ρ(f) = \int_{ {\mathbb{R}}} f(x) \,e^{-i\,kx} ρ(x) \, {\rm d} x\ \ \ \mbox{for}\ \ f\in H^s({\mathbb{R}})\ \ \mbox{or}\ \ f\in C^s({\mathbb{R}}) \] with $k\in {\mathbb{R}}$ and a smooth density function $ρ$ such as $ ρ(x) = \frac{1}{\sqrt{2 π}} \exp( -x^2/2) $. The optimal error bounds are $Θ((n+\max(1,|k|))^{-s})$ with the factors in the $Θ$ notation dependent only on $s$ and $ρ$.
△ Less
Submitted 18 November, 2015; v1 submitted 17 November, 2015;
originally announced November 2015.
-
A Universal Algorithm for Multivariate Integration
Authors:
David Krieg,
Erich Novak
Abstract:
We present an algorithm for multivariate integration over cubes that is unbiased and has optimal order of convergence (in the randomized sense as well as in the worst case setting) for all Sobolev spaces $H^{r, mix}([0,1]^d)$ and $H^s([0,1]^d)$ for $s>d/2$.
We present an algorithm for multivariate integration over cubes that is unbiased and has optimal order of convergence (in the randomized sense as well as in the worst case setting) for all Sobolev spaces $H^{r, mix}([0,1]^d)$ and $H^s([0,1]^d)$ for $s>d/2$.
△ Less
Submitted 30 January, 2016; v1 submitted 24 July, 2015;
originally announced July 2015.
-
Some Results on the Complexity of Numerical Integration
Authors:
Erich Novak
Abstract:
This is a survey (21 pages, 124 references) written for the MCQMC 2014 conference in Leuven, April 2014. We start with the seminal paper of Bakhvalov (1959) and end with new results on the curse of dimension and on the complexity of oscillatory integrals. Some small errors of earlier versions are corrected.
This is a survey (21 pages, 124 references) written for the MCQMC 2014 conference in Leuven, April 2014. We start with the seminal paper of Bakhvalov (1959) and end with new results on the curse of dimension and on the complexity of oscillatory integrals. Some small errors of earlier versions are corrected.
△ Less
Submitted 18 November, 2015; v1 submitted 23 September, 2014;
originally announced September 2014.
-
Tractability of the approximation of high-dimensional rank one tensors
Authors:
Erich Novak,
Daniel Rudolf
Abstract:
We study the approximation of high-dimensional rank one tensors using point evaluations and consider deterministic as well as randomized algorithms. We prove that for certain parameters (smoothness and norm of the $r$th derivative) this problem is intractable while for other parameters the problem is tractable and the complexity is only polynomial in the dimension for every fixed $\varepsilon>0$.…
▽ More
We study the approximation of high-dimensional rank one tensors using point evaluations and consider deterministic as well as randomized algorithms. We prove that for certain parameters (smoothness and norm of the $r$th derivative) this problem is intractable while for other parameters the problem is tractable and the complexity is only polynomial in the dimension for every fixed $\varepsilon>0$. For randomized algorithms we completely characterize the set of parameters that lead to easy or difficult problems, respectively. In the "difficult" case we modify the class to obtain a tractable problem: The problem gets tractable with a polynomial (in the dimension) complexity if the support of the function is not too small.
△ Less
Submitted 2 December, 2014; v1 submitted 20 February, 2014;
originally announced February 2014.
-
Computation of expectations by Markov chain Monte Carlo methods
Authors:
Erich Novak,
Daniel Rudolf
Abstract:
Markov chain Monte Carlo (MCMC) methods are a very versatile and widely used tool to compute integrals and expectations. In this short survey we focus on error bounds, rules for choosing the burn in, high dimensional problems and tractability versus curse of dimension.
Markov chain Monte Carlo (MCMC) methods are a very versatile and widely used tool to compute integrals and expectations. In this short survey we focus on error bounds, rules for choosing the burn in, high dimensional problems and tractability versus curse of dimension.
△ Less
Submitted 8 September, 2014; v1 submitted 8 November, 2013;
originally announced November 2013.
-
Complexity of Oscillatory Integration for Univariate Sobolev Spaces
Authors:
Erich Novak,
Mario Ullrich,
Henryk Woźniakowski
Abstract:
We analyze univariate oscillatory integrals for the standard Sobolev spaces $H^s$ of periodic and non-periodic functions with an arbitrary integer $s\ge1$. We find matching lower and upper bounds on the minimal worst case error of algorithms that use $n$ function or derivative values. We also find sharp bounds on the information complexity which is the minimal $n$ for which the absolute or normali…
▽ More
We analyze univariate oscillatory integrals for the standard Sobolev spaces $H^s$ of periodic and non-periodic functions with an arbitrary integer $s\ge1$. We find matching lower and upper bounds on the minimal worst case error of algorithms that use $n$ function or derivative values. We also find sharp bounds on the information complexity which is the minimal $n$ for which the absolute or normalized error is at most $\varepsilon$. We show surprising relations between the information complexity and the oscillatory weight. We also briefly consider the case of $s=\infty$.
△ Less
Submitted 24 October, 2014; v1 submitted 6 November, 2013;
originally announced November 2013.
-
On Weak Tractability of the Clenshaw-Curtis Smolyak Algorithm
Authors:
Aicke Hinrichs,
Erich Novak,
Mario Ullrich
Abstract:
We consider the problem of integration of d-variate analytic functions defined on the unit cube with directional derivatives of all orders bounded by 1. We prove that the Clenshaw Curtis Smolyak algorithm leads to weak tractability of the problem. This seems to be the first positive tractability result for the Smolyak algorithm for a normalized and unweighted problem. The space of integrands is no…
▽ More
We consider the problem of integration of d-variate analytic functions defined on the unit cube with directional derivatives of all orders bounded by 1. We prove that the Clenshaw Curtis Smolyak algorithm leads to weak tractability of the problem. This seems to be the first positive tractability result for the Smolyak algorithm for a normalized and unweighted problem. The space of integrands is not a tensor product space and therefore we have to develop a different proof technique. We use the polynomial exactness of the algorithm as well as an explicit bound on the operator norm of the algorithm.
△ Less
Submitted 22 April, 2014; v1 submitted 2 September, 2013;
originally announced September 2013.
-
The Curse of Dimensionality for Numerical Integration of Smooth Functions II
Authors:
Aicke Hinrichs,
Erich Novak,
Mario Ullrich,
Henryk Wozniakowski
Abstract:
We prove the curse of dimensionality in the worst case setting for numerical integration for a number of classes of smooth $d$-variate functions. Roughly speaking, we consider different bounds for the derivatives of $f \in C^k(D_d)$ and ask whether the curse of dimensionality holds for the respective classes of functions. We always assume that $D_d \subset \mathbb{R}^d$ has volume one and consider…
▽ More
We prove the curse of dimensionality in the worst case setting for numerical integration for a number of classes of smooth $d$-variate functions. Roughly speaking, we consider different bounds for the derivatives of $f \in C^k(D_d)$ and ask whether the curse of dimensionality holds for the respective classes of functions. We always assume that $D_d \subset \mathbb{R}^d$ has volume one and consider various values of $k$ including the case $k=\infty$ which corresponds to infinitely many differentiable functions. We obtain necessary and sufficient conditions, and in some cases a full characterization for the curse of dimensionality. For infinitely many differentiable functions we prove the curse if the bounds on the successive derivatives are appropriately large. The proof technique is based on a volume estimate of a neighborhood of the convex hull of $n$ points which decays exponentially fast if $n$ is small relative to $d$. For $k=\infty$, we also also study conditions for quasi-polynomial, weak and uniform weak tractability.
△ Less
Submitted 12 November, 2013; v1 submitted 11 April, 2013;
originally announced April 2013.
-
The Curse of Dimensionality for Numerical Integration of Smooth Functions
Authors:
Aicke Hinrichs,
Erich Novak,
Mario Ullrich,
Henryk Wozniakowski
Abstract:
We prove the curse of dimensionality for multivariate integration of C^r functions: The number of needed function values to achieve an error ε is larger than c_r (1+γ)^d for ε\le ε_0, where c_r,γ>0 and d is the dimension. The proofs are based on volume estimates for r=1 together with smoothing by convolution. This allows us to obtain smooth fooling functions for r>1.
We prove the curse of dimensionality for multivariate integration of C^r functions: The number of needed function values to achieve an error ε is larger than c_r (1+γ)^d for ε\le ε_0, where c_r,γ>0 and d is the dimension. The proofs are based on volume estimates for r=1 together with smoothing by convolution. This allows us to obtain smooth fooling functions for r>1.
△ Less
Submitted 16 April, 2013; v1 submitted 5 November, 2012;
originally announced November 2012.
-
Discontinuous information in the worst case and randomized settings
Authors:
Aicke Hinrichs,
Erich Novak,
Henryk Wozniakowski
Abstract:
We believe that discontinuous linear information is never more powerful than continuous linear information for approximating continuous operators. We prove such a result in the worst case setting. In the randomized setting we consider compact linear operators defined between Hilbert spaces. In this case, the use of discontinuous linear information in the randomized setting cannot be much more powe…
▽ More
We believe that discontinuous linear information is never more powerful than continuous linear information for approximating continuous operators. We prove such a result in the worst case setting. In the randomized setting we consider compact linear operators defined between Hilbert spaces. In this case, the use of discontinuous linear information in the randomized setting cannot be much more powerful than continuous linear information in the worst case setting. These results can be applied when function evaluations are used even if function values are defined only almost everywhere.
△ Less
Submitted 15 June, 2011;
originally announced June 2011.
-
On the power of function values for the approximation problem in various settings
Authors:
Erich Novak,
Henryk Woźniakowski
Abstract:
This is an expository paper on approximating functions from general Hilbert or Banach spaces in the worst case, average case and randomized settings with error measured in the $L_p$ sense. We define the power function as the ratio between the best rate of convergence of algorithms that use function values over the best rate of convergence of algorithms that use arbitrary linear functionals for a w…
▽ More
This is an expository paper on approximating functions from general Hilbert or Banach spaces in the worst case, average case and randomized settings with error measured in the $L_p$ sense. We define the power function as the ratio between the best rate of convergence of algorithms that use function values over the best rate of convergence of algorithms that use arbitrary linear functionals for a worst possible Hilbert or Banach space for which the problem of approximating functions is well defined. Obviously, the power function takes values at most one. If these values are one or close to one than the power of function values is the same or almost the same as the power of arbitrary linear functionals. We summarize and supply a few new estimates on the power function. We also indicate eight open problems related to the power function since this function has not yet been studied for many cases. We believe that the open problems will be of interest to a general audience of mathematicians.
△ Less
Submitted 6 June, 2011; v1 submitted 16 November, 2010;
originally announced November 2010.
-
The Curse of Dimensionality for Monotone and Convex Functions of Many Variables
Authors:
Aicke Hinrichs,
Erich Novak,
Henryk Woźniakowski
Abstract:
We study the integration and approximation problems for monotone and convex bounded functions that depend on $d$ variables, where $d$ can be arbitrarily large. We consider the worst case error for algorithms that use finitely many function values. We prove that these problems suffer from the curse of dimensionality. That is, one needs exponentially many (in $d$) function values to achieve an error…
▽ More
We study the integration and approximation problems for monotone and convex bounded functions that depend on $d$ variables, where $d$ can be arbitrarily large. We consider the worst case error for algorithms that use finitely many function values. We prove that these problems suffer from the curse of dimensionality. That is, one needs exponentially many (in $d$) function values to achieve an error $ε$.
△ Less
Submitted 16 November, 2010;
originally announced November 2010.
-
Open letter on "Adaptivity and computational complexity in the numerical solution of ODEs" by Silvana Ilie, Gustaf Soederlind and Robert M. Corless
Authors:
Erich Novak,
Henryk Wozniakowski
Abstract:
This is an open letter that we sent to S. Ilie, G. Soederlind and R.M. Corless in August 2008.
This is an open letter that we sent to S. Ilie, G. Soederlind and R.M. Corless in August 2008.
△ Less
Submitted 28 April, 2009;
originally announced April 2009.
-
Optimal Approximation of Elliptic Problems by Linear and Nonlinear Mappings III: Frames
Authors:
Stephan Dahlke,
Erich Novak,
Winfried Sickel
Abstract:
We study the optimal approximation of the solution of an operator equation by certain n-term approximations with respect to specific classes of frames. We study worst case errors and the optimal order of convergence and define suitable nonlinear frame widths.
The main advantage of frames compared to Riesz basis, which were studied in our earlier papers, is the fact that we can now handle arbit…
▽ More
We study the optimal approximation of the solution of an operator equation by certain n-term approximations with respect to specific classes of frames. We study worst case errors and the optimal order of convergence and define suitable nonlinear frame widths.
The main advantage of frames compared to Riesz basis, which were studied in our earlier papers, is the fact that we can now handle arbitrary bounded Lipschitz domains--also for the upper bounds.
Key words: elliptic operator equation, worst case error, frames, nonlinear approximation, best n-term approximation, manifold width, Besov spaces on Lipschitz domains
△ Less
Submitted 9 March, 2007; v1 submitted 19 November, 2006;
originally announced November 2006.
-
Simple Monte Carlo and the Metropolis Algorithm
Authors:
Peter Mathe,
Erich Novak
Abstract:
We study the integration of functions with respect to an unknown density. We compare the simple Monte Carlo method (which is almost optimal for a certain large class of inputs) and compare it with the Metropolis algorithm (based on a suitable ball walk).
Using MCMC we prove (for certain classes of inputs) that adaptive methods are much better than nonadaptive ones. Actually, the curse of dimen…
▽ More
We study the integration of functions with respect to an unknown density. We compare the simple Monte Carlo method (which is almost optimal for a certain large class of inputs) and compare it with the Metropolis algorithm (based on a suitable ball walk).
Using MCMC we prove (for certain classes of inputs) that adaptive methods are much better than nonadaptive ones. Actually, the curse of dimension (for nonadaptive methods) can be broken by adaption.
△ Less
Submitted 1 June, 2007; v1 submitted 9 November, 2006;
originally announced November 2006.
-
Optimal Approximation of Elliptic Problems by Linear and Nonlinear Mappings II
Authors:
Stephan Dahlke,
Erich Novak,
Winfried Sickel
Abstract:
We study the optimal approximation of the solution of an operator equation Au=f by linear and nonlinear mappings.
We identify those cases where optimal nonlinear approximation is better than optimal linear approximation.
We study the optimal approximation of the solution of an operator equation Au=f by linear and nonlinear mappings.
We identify those cases where optimal nonlinear approximation is better than optimal linear approximation.
△ Less
Submitted 20 June, 2006;
originally announced June 2006.
-
Optimal Approximation of Elliptic Problems by Linear and Nonlinear Mappings I
Authors:
Stephan Dahlke,
Erich Novak,
Winfried Sickel
Abstract:
We study the optimal approximation of the solution of an operator equation Au=f by linear and nonlinear mappings.
We study the optimal approximation of the solution of an operator equation Au=f by linear and nonlinear mappings.
△ Less
Submitted 20 June, 2006;
originally announced June 2006.
-
Cubature Formulas for Symmetric Measures in Higher Dimensions with Few Points
Authors:
Aicke Hinrichs,
Erich Novak
Abstract:
We study cubature formulas for d-dimensional integrals with an arbitrary symmetric weight function of tensor product form. We present a construction that yields a high polynomial exactness: for fixed degree l=5 or l=7 and large dimension, the number of knots is only slightly larger than the lower bound of Möller and much smaller compared to the known constructions.
We also show, for any odd de…
▽ More
We study cubature formulas for d-dimensional integrals with an arbitrary symmetric weight function of tensor product form. We present a construction that yields a high polynomial exactness: for fixed degree l=5 or l=7 and large dimension, the number of knots is only slightly larger than the lower bound of Möller and much smaller compared to the known constructions.
We also show, for any odd degree l=2k+1, that the minimal number of points is almost independent of the weight function. This is also true for the integration over the (Euclidean) sphere.
△ Less
Submitted 20 June, 2006; v1 submitted 5 September, 2005;
originally announced September 2005.
-
On a Problem in Quantum Summation
Authors:
Stefan Heinrich,
Erich Novak
Abstract:
We consider the computation of the mean of sequences in the quantum model of computation. We determine the query complexity in the case of sequences which satisfy a $p$-summability condition for $1\le p<2$. This settles a problem left open in Heinrich (2001).
We consider the computation of the mean of sequences in the quantum model of computation. We determine the query complexity in the case of sequences which satisfy a $p$-summability condition for $1\le p<2$. This settles a problem left open in Heinrich (2001).
△ Less
Submitted 7 September, 2001;
originally announced September 2001.
-
Optimal Summation and Integration by Deterministic, Randomized, and Quantum Algorithms
Authors:
S. Heinrich,
E. Novak
Abstract:
We survey old and new results about optimal algorithms for summation of finite sequences and for integration of functions from Hoelder or Sobolev spaces. First we discuss optimal deterministic and randomized algorithms. Then we add a new aspect, which has not been covered before on conferences about (quasi-) Monte Carlo methods: quantum computation. We give a short introduction into this setting…
▽ More
We survey old and new results about optimal algorithms for summation of finite sequences and for integration of functions from Hoelder or Sobolev spaces. First we discuss optimal deterministic and randomized algorithms. Then we add a new aspect, which has not been covered before on conferences about (quasi-) Monte Carlo methods: quantum computation. We give a short introduction into this setting and present recent results of the authors on optimal quantum algorithms for summation and integration. We discuss comparisons between the three settings. The most interesting case for Monte Carlo and quantum integration is that of moderate smoothness k and large dimension d which, in fact, occurs in a number of important applied problems. In that case the deterministic exponent is negligible, so the n^{-1/2} Monte Carlo and the n^{-1} quantum speedup essentially constitute the entire convergence rate. We observe that -- there is an exponential speed-up of quantum algorithms over deterministic (classical) algorithms, if k/d tends to zero; -- there is a (roughly) quadratic speed-up of quantum algorithms over randomized classical algorithms, if k/d is small.
△ Less
Submitted 23 May, 2001;
originally announced May 2001.
-
Quantum Complexity of Integration
Authors:
Erich Novak
Abstract:
It is known that quantum computers yield a speed-up for certain discrete problems. Here we want to know whether quantum computers are useful for continuous problems. We study the computation of the integral of functions from the classical Hoelder classes with d variables. The optimal orders for the complexity of deterministic and (general) randomized methods are known. We obtain the respective o…
▽ More
It is known that quantum computers yield a speed-up for certain discrete problems. Here we want to know whether quantum computers are useful for continuous problems. We study the computation of the integral of functions from the classical Hoelder classes with d variables. The optimal orders for the complexity of deterministic and (general) randomized methods are known. We obtain the respective optimal orders for quantum algorithms and also for restricted Monte Carlo (only coin tossing instead of general random numbers). To summarize the results one can say that (1) there is an exponential speed-up of quantum algorithms over deterministic (classical) algorithms, if the smoothness is small; (2) there is a (roughly) quadratic speed-up of quantum algorithms over randomized classical methods, if the smoothness is small.
△ Less
Submitted 24 June, 2001; v1 submitted 29 August, 2000;
originally announced August 2000.
