-
Removing numerical dispersion from linear evolution equations
Authors:
Jens Wittsten,
Erik F. M. Koene,
Fredrik Andersson,
Johan O. A. Robertsson
Abstract:
We describe a method for removing the numerical errors in the modeling of linear evolution equations that are caused by approximating the time derivative by a finite difference operator. The method is based on integral transforms realized as certain Fourier integral operators, called time dispersion transforms, and we prove that, under an assumption about the frequency content, it yields a solutio…
▽ More
We describe a method for removing the numerical errors in the modeling of linear evolution equations that are caused by approximating the time derivative by a finite difference operator. The method is based on integral transforms realized as certain Fourier integral operators, called time dispersion transforms, and we prove that, under an assumption about the frequency content, it yields a solution with correct evolution throughout the entire lifespan. We demonstrate the method on a model equation as well as on the simulation of elastic and viscoelastic wave propagation.
△ Less
Submitted 20 January, 2021; v1 submitted 22 June, 2019;
originally announced June 2019.
-
ESPRIT for multidimensional general grids
Authors:
Fredrik Andersson,
Marcus Carlsson
Abstract:
We present a new method for complex frequency estimation in several variables, extending the classical (1d) ESPRIT-algorithm. We also consider how to work with data sampled on non-standard domains (i.e going beyond multi-rectangles).
We present a new method for complex frequency estimation in several variables, extending the classical (1d) ESPRIT-algorithm. We also consider how to work with data sampled on non-standard domains (i.e going beyond multi-rectangles).
△ Less
Submitted 22 August, 2018; v1 submitted 21 May, 2017;
originally announced May 2017.
-
Non-Convex Rank/Sparsity Regularization and Local Minima
Authors:
Carl Olsson,
Marcus Carlsson,
Fredrik Andersson,
Viktor Larsson
Abstract:
This paper considers the problem of recovering either a low rank matrix or a sparse vector from observations of linear combinations of the vector or matrix elements. Recent methods replace the non-convex regularization with $\ell_1$ or nuclear norm relaxations. It is well known that this approach can be guaranteed to recover a near optimal solutions if a so called restricted isometry property (RIP…
▽ More
This paper considers the problem of recovering either a low rank matrix or a sparse vector from observations of linear combinations of the vector or matrix elements. Recent methods replace the non-convex regularization with $\ell_1$ or nuclear norm relaxations. It is well known that this approach can be guaranteed to recover a near optimal solutions if a so called restricted isometry property (RIP) holds. On the other hand it is also known to perform soft thresholding which results in a shrinking bias which can degrade the solution.
In this paper we study an alternative non-convex regularization term. This formulation does not penalize elements that are larger than a certain threshold making it much less prone to small solutions. Our main theoretical results show that if a RIP holds then the stationary points are often well separated, in the sense that their differences must be of high cardinality/rank. Thus, with a suitable initial solution the approach is unlikely to fall into a bad local minima. Our numerical tests show that the approach is likely to converge to a better solution than standard $\ell_1$/nuclear-norm relaxation even when starting from trivial initializations. In many cases our results can also be used to verify global optimality of our method.
△ Less
Submitted 21 March, 2017;
originally announced March 2017.
-
Convergence of dual ascent in non-convex/non-differentiable optimization
Authors:
Fredrik Andersson,
Marcus Carlsson,
Carl Olsson
Abstract:
We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the convex envelope. We also introduce a new augmented version, which is shown to have superior convergence properties, and provide new results even for convex but…
▽ More
We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the convex envelope. We also introduce a new augmented version, which is shown to have superior convergence properties, and provide new results even for convex but non-differentiable objective functionals (as well as non-convex).
The results are applied to low rank approximation of a given matrix, subject to linear constraints. In particular, letting the linear constraints enforce Hankel structure of the respective matrices, the algorithms can be applied to complex frequency estimation. We provide numerical tests in this setting.
△ Less
Submitted 21 September, 2016;
originally announced September 2016.
-
Convex envelopes for fixed rank approximation
Authors:
Fredrik Andersson,
Marcus Carlsson,
Carl Olsson
Abstract:
A convex envelope for the problem of finding the best approximation to a given matrix with a prescribed rank is constructed. This convex envelope allows the usage of traditional optimization techniques when additional constraints are added to the finite rank approximation problem. Expression for the dependence of the convex envelope on the singular values of the given matrix is derived and global…
▽ More
A convex envelope for the problem of finding the best approximation to a given matrix with a prescribed rank is constructed. This convex envelope allows the usage of traditional optimization techniques when additional constraints are added to the finite rank approximation problem. Expression for the dependence of the convex envelope on the singular values of the given matrix is derived and global minimization properties are derived. The corresponding proximity operator is also studied.
△ Less
Submitted 27 August, 2016;
originally announced August 2016.
-
Fast hyperbolic Radon transform represented as convolutions in log-polar coordinates
Authors:
Viktor V. Nikitin,
Fredrik Andersson,
Marcus Carlsson,
Anton A. Duchkov
Abstract:
The hyperbolic Radon transform is a commonly used tool in seismic processing, for instance in seismic velocity analysis, data interpolation and for multiple removal. A direct implementation by summation of traces with different moveouts is computationally expensive for large data sets. In this paper we present a new method for fast computation of the hyperbolic Radon transforms. It is based on usi…
▽ More
The hyperbolic Radon transform is a commonly used tool in seismic processing, for instance in seismic velocity analysis, data interpolation and for multiple removal. A direct implementation by summation of traces with different moveouts is computationally expensive for large data sets. In this paper we present a new method for fast computation of the hyperbolic Radon transforms. It is based on using a log-polar sampling with which the main computational parts reduce to computing convolutions. This allows for fast implementations by means of FFT. In addition to the FFT operations, interpolation procedures are required for switching between coordinates in the time-offset; Radon; and log-polar domains. Graphical Processor Units (GPUs) are suitable to use as a computational platform for this purpose, due to the hardware supported interpolation routines as well as optimized routines for FFT. Performance tests show large speed-ups of the proposed algorithm. Hence, it is suitable to use in iterative methods, and we provide examples for data interpolation and multiple removal using this approach.
△ Less
Submitted 22 April, 2016;
originally announced April 2016.
-
Fixed-point algorithms for frequency estimation and structured low rank approximation
Authors:
Fredrik Andersson,
Marcus Carlsson
Abstract:
We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, or frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the minimum of the convex envelope of the original objective function along with its struc…
▽ More
We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, or frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the minimum of the convex envelope of the original objective function along with its structured matrix constraint. It often happens that this solution agrees with the solution to the original minimization problem, and we provide a simple criterium for when this is true. We also provide more general fixed-point algorithms that can be used to treat the problems of making weighted approximations by sums of exponentials given equally or unequally spaced sampling. We apply the method to the case of missing data, although optimal convergence of the fixed-point algorithm is not guaranteed in this case. However, it turns out that the method often gives perfect reconstruction (up to machine precision) in such cases. We also discuss multidimensional extensions, and illustrate how the proposed algorithms can be used to recover sums of exponentials in several variables, but when samples are available only along a curve.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
On the Kronecker and Caratheodory-Fejer theorems in several variables
Authors:
Fredrik Andersson,
Marcus Carlsson
Abstract:
Multivariate versions of the Kronecker theorem in the continuous multivariate setting has recently been published. These theorems characterize the symbols that give rise to finite rank multidimensional Hankel and Toeplitz type operators defined on general domains. In this paper we study how the additional assumption of positive semidefinite affects the characterization of the corresponding symbols…
▽ More
Multivariate versions of the Kronecker theorem in the continuous multivariate setting has recently been published. These theorems characterize the symbols that give rise to finite rank multidimensional Hankel and Toeplitz type operators defined on general domains. In this paper we study how the additional assumption of positive semidefinite affects the characterization of the corresponding symbols, which we refer to as Carathéodory-Fejer type theorems. We show that these theorems become particularly transparent in the continuous setting, by providing elegant if and only if statements connecting the rank with sums of exponential functions. We also discuss how these objects can be discretized, giving rise to an interesting class of structured matrices that inherit these desirable properties from their continuous analogs. We describe how the continuous Kronecker theorem also applies to these structured matrices, given sufficient sampling. We also provide a new proof for the Carathéodory-Fejer theorem for block Toeplitz matrices, based on tools from tensor algebra.
△ Less
Submitted 13 August, 2015; v1 submitted 18 June, 2015;
originally announced June 2015.
-
Fast algorithms and efficient GPU implementations for the Radon transform and the back-projection operator represented as convolution operators
Authors:
Fredrik Andersson,
Marcus Carlsson,
Viktor V. Nikitin
Abstract:
The Radon transform and its adjoint, the back-projection operator, can both be expressed as convolutions in log-polar coordinates. Hence, fast algorithms for the application of the operators can be constructed by using FFT, if data is resampled at log-polar coordinates. Radon data is typically measured on an equally spaced grid in polar coordinates, and reconstructions are represented (as images)…
▽ More
The Radon transform and its adjoint, the back-projection operator, can both be expressed as convolutions in log-polar coordinates. Hence, fast algorithms for the application of the operators can be constructed by using FFT, if data is resampled at log-polar coordinates. Radon data is typically measured on an equally spaced grid in polar coordinates, and reconstructions are represented (as images) in Cartesian coordinates. Therefore, in addition to FFT, several steps of interpolation have to be conducted in order to apply the Radon transform and the back-projection operator by means of convolutions.
Both the interpolation and the FFT operations can be efficiently implemented on Graphical Processor Units (GPUs). For the interpolation, it is possible to make use of the fact that linear interpolation is hard-wired on GPUs, meaning that it has the same computational cost as direct memory access. Cubic order interpolation schemes can be constructed by combining linear interpolation steps which provides important computation speedup.
We provide details about how the Radon transform and the back-projection can be implemented efficiently as convolution operators on GPUs. For large data sizes, speedups of about 10 times are obtained in relation to the computational times of other software packages based on GPU implementations of the Radon transform and the back-projection operator. Moreover, speedups of more than a 1000 times are obtained against the CPU-implementations provided in the MATLAB image processing toolbox.
△ Less
Submitted 29 May, 2015;
originally announced June 2015.
-
Operator-Lipschitz estimates for the singular value functional calculus
Authors:
Fredrik Andersson,
Marcus Carlsson,
Karl-Mikael Perfekt
Abstract:
We consider a functional calculus for compact operators, acting on the singular values rather than the spectrum, which appears frequently in applied mathematics. Necessary and sufficient conditions for this singular value functional calculus to be Lipschitz-continuous with respect to the Hilbert-Schmidt norm are given. We also provide sharp constants.
We consider a functional calculus for compact operators, acting on the singular values rather than the spectrum, which appears frequently in applied mathematics. Necessary and sufficient conditions for this singular value functional calculus to be Lipschitz-continuous with respect to the Hilbert-Schmidt norm are given. We also provide sharp constants.
△ Less
Submitted 17 March, 2015;
originally announced March 2015.
-
Multiscale reverse-time-migration-type imaging using the dyadic parabolic decomposition of phase space
Authors:
Fredrik Andersson,
Maarten V. de Hoop,
Herwig Wendt
Abstract:
We develop a representation of reverse-time migration in terms of Fourier integral operators the canonical relations of which are graphs. Through the dyadic parabolic decomposition of phase space, we obtain the solution of the wave equation with a boundary source and homogeneous initial conditions using wave packets. On this basis, we develop a numerical procedure for the reverse time continuation…
▽ More
We develop a representation of reverse-time migration in terms of Fourier integral operators the canonical relations of which are graphs. Through the dyadic parabolic decomposition of phase space, we obtain the solution of the wave equation with a boundary source and homogeneous initial conditions using wave packets. On this basis, we develop a numerical procedure for the reverse time continuation from the boundary of scattering data and for RTM migration. The algorithms are derived from those we recently developed for the discrete approximate evaluation of the action of Fourier integral operators and inherit from their conceptual and numerical properties.
△ Less
Submitted 23 September, 2015; v1 submitted 27 June, 2013;
originally announced June 2013.
-
Frequency estimation based on Hankel matrices and the alternating direction method of multipliers
Authors:
Fredrik Andersson,
Marcus Carlsson,
Jean-Yves Tourneret,
Herwig Wendt
Abstract:
We develop a parametric high-resolution method for the estimation of the frequency nodes of linear combinations of complex exponentials with exponential damping. We use Kronecker's theorem to formulate the associated nonlinear least squares problem as an optimization problem in the space of vectors generating Hankel matrices of fixed rank. Approximate solutions to this problem are obtained by usin…
▽ More
We develop a parametric high-resolution method for the estimation of the frequency nodes of linear combinations of complex exponentials with exponential damping. We use Kronecker's theorem to formulate the associated nonlinear least squares problem as an optimization problem in the space of vectors generating Hankel matrices of fixed rank. Approximate solutions to this problem are obtained by using the alternating direction method of multipliers. Finally, we extract the frequency estimates from the con-eigenvectors of the solution Hankel matrix. The resulting algorithm is simple, easy to implement and can be applied to data with equally spaced samples with approximation weights, which for instance allows cases of missing data samples. By means of numerical simulations, we analyze and illustrate the excellent performance of the method, attaining the Cramér-Rao bound.
△ Less
Submitted 12 June, 2013;
originally announced June 2013.
-
Alternating projections on non-tangential manifolds
Authors:
Fredrik Andersson,
Marcus Carlsson
Abstract:
We consider sequences $(B_k)_{k=0}^\infty$ of points obtained by projecting back and forth between two manifolds $\M_1$ and $\M_2$, and give conditions guaranteeing that the sequence converge to a limit $B_\infty\in\M_1\cap\M_2$. Our motivation is the study of algorithms based on finding the limit of such sequences, which have proven useful in a number of areas. The intersection is typically a set…
▽ More
We consider sequences $(B_k)_{k=0}^\infty$ of points obtained by projecting back and forth between two manifolds $\M_1$ and $\M_2$, and give conditions guaranteeing that the sequence converge to a limit $B_\infty\in\M_1\cap\M_2$. Our motivation is the study of algorithms based on finding the limit of such sequences, which have proven useful in a number of areas. The intersection is typically a set with desirable properties, but for which there is no efficient method of finding the closest point $B_{opt}$ in $\M_1\cap\M_2$. We prove not only that the sequence of alternating projections converges, but that the limit point is fairly close to $B_{opt}$, in a manner relative to the distance $\|B_0-B_{opt}\|$, thereby significantly improving earlier results in the field. A concrete example with applications to frequency estimation of signals is also presented.
△ Less
Submitted 8 July, 2011;
originally announced July 2011.
-
A fast alternating projection method for complex frequency estimation
Authors:
Fredrik Andersson,
Marcus Carlsson,
Per-Anders Ivert
Abstract:
The problem of approximating a sampled function using sums of a fixed number of complex exponentials is considered. We use alternating projections between fixed rank matrices and Hankel matrices to obtain such an approximation. Convergence, convergence rates and error estimates for this technique are proven, and fast algorithms are developed. We compare the numerical results obtain with the MUSIC…
▽ More
The problem of approximating a sampled function using sums of a fixed number of complex exponentials is considered. We use alternating projections between fixed rank matrices and Hankel matrices to obtain such an approximation. Convergence, convergence rates and error estimates for this technique are proven, and fast algorithms are developed. We compare the numerical results obtain with the MUSIC and ESPRIT methods.
△ Less
Submitted 8 July, 2011;
originally announced July 2011.
