-
WHOMP: Optimizing Randomized Controlled Trials via Wasserstein Homogeneity
Authors:
Shizhou Xu,
Thomas Strohmer
Abstract:
We investigate methods for partitioning datasets into subgroups that maximize diversity within each subgroup while minimizing dissimilarity across subgroups. We introduce a novel partitioning method called the $\textit{Wasserstein Homogeneity Partition}$ (WHOMP), which optimally minimizes type I and type II errors that often result from imbalanced group splitting or partitioning, commonly referred…
▽ More
We investigate methods for partitioning datasets into subgroups that maximize diversity within each subgroup while minimizing dissimilarity across subgroups. We introduce a novel partitioning method called the $\textit{Wasserstein Homogeneity Partition}$ (WHOMP), which optimally minimizes type I and type II errors that often result from imbalanced group splitting or partitioning, commonly referred to as accidental bias, in comparative and controlled trials. We conduct an analytical comparison of WHOMP against existing partitioning methods, such as random subsampling, covariate-adaptive randomization, rerandomization, and anti-clustering, demonstrating its advantages. Moreover, we characterize the optimal solutions to the WHOMP problem and reveal an inherent trade-off between the stability of subgroup means and variances among these solutions. Based on our theoretical insights, we design algorithms that not only obtain these optimal solutions but also equip practitioners with tools to select the desired trade-off. Finally, we validate the effectiveness of WHOMP through numerical experiments, highlighting its superiority over traditional methods.
△ Less
Submitted 3 October, 2024; v1 submitted 27 September, 2024;
originally announced September 2024.
-
Differentially Private Synthetic High-dimensional Tabular Stream
Authors:
Girish Kumar,
Thomas Strohmer,
Roman Vershynin
Abstract:
While differentially private synthetic data generation has been explored extensively in the literature, how to update this data in the future if the underlying private data changes is much less understood. We propose an algorithmic framework for streaming data that generates multiple synthetic datasets over time, tracking changes in the underlying private data. Our algorithm satisfies differential…
▽ More
While differentially private synthetic data generation has been explored extensively in the literature, how to update this data in the future if the underlying private data changes is much less understood. We propose an algorithmic framework for streaming data that generates multiple synthetic datasets over time, tracking changes in the underlying private data. Our algorithm satisfies differential privacy for the entire input stream (continual differential privacy) and can be used for high-dimensional tabular data. Furthermore, we show the utility of our method via experiments on real-world datasets. The proposed algorithm builds upon a popular select, measure, fit, and iterate paradigm (used by offline synthetic data generation algorithms) and private counters for streams.
△ Less
Submitted 30 August, 2024;
originally announced September 2024.
-
Quantum paving: When sphere packings meet Gabor frames
Authors:
Markus Faulhuber,
Thomas Strohmer
Abstract:
We introduce the new problems of quantum packing, quantum covering, and quantum paving. These problems arise naturally when considering an algebra of non-commutative operators that is deeply rooted in quantum physics as well as in Gabor analysis. Quantum packing and quantum covering show similarities with energy minimization and the dual problem of polarization. Quantum paving, in turn, aims to si…
▽ More
We introduce the new problems of quantum packing, quantum covering, and quantum paving. These problems arise naturally when considering an algebra of non-commutative operators that is deeply rooted in quantum physics as well as in Gabor analysis. Quantum packing and quantum covering show similarities with energy minimization and the dual problem of polarization. Quantum paving, in turn, aims to simultaneously optimize both quantum packing and quantum covering. Classical sphere packing and covering hint the optimal configurations for our new problems. We present solutions in certain cases, state several conjectures related to quantum paving and discuss some applications.
△ Less
Submitted 16 August, 2024;
originally announced August 2024.
-
Convergence Analysis of the Alternating Anderson-Picard Method for Nonlinear Fixed-point Problems
Authors:
Xue Feng,
M. Paul Laiu,
Thomas Strohmer
Abstract:
Anderson Acceleration (AA) has been widely used to solve nonlinear fixed-point problems due to its rapid convergence. This work focuses on a variant of AA in which multiple Picard iterations are performed between each AA step, referred to as the Alternating Anderson-Picard (AAP) method. Despite introducing more 'slow' Picard iterations, this method has been shown to be efficient and even more robu…
▽ More
Anderson Acceleration (AA) has been widely used to solve nonlinear fixed-point problems due to its rapid convergence. This work focuses on a variant of AA in which multiple Picard iterations are performed between each AA step, referred to as the Alternating Anderson-Picard (AAP) method. Despite introducing more 'slow' Picard iterations, this method has been shown to be efficient and even more robust in both linear and nonlinear cases. However, there is a lack of theoretical analysis for AAP in the nonlinear case, which this paper aims to address. We show the equivalence between AAP and a multisecant-GMRES method that uses GMRES to solve a multisecant linear system at each iteration. More interestingly, the incorporation of Picard iterations and AA establishes a deep connection between AAP and the Newton-GMRES method. This connection is evident in terms of the multisecant matrix, the approximate Jacobian inverse, search direction, and optimization gain -- an essential factor in the convergence analysis of AA. We show that these terms converge to their corresponding terms in the Newton-GMRES method as the residual approaches zero. Consequently, we build the convergence analysis of AAP. To validate our theoretical findings, numerical examples are provided.
△ Less
Submitted 15 July, 2024;
originally announced July 2024.
-
Metric geometry of the privacy-utility tradeoff
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
Synthetic data are an attractive concept to enable privacy in data sharing. A fundamental question is how similar the privacy-preserving synthetic data are compared to the true data. Using metric privacy, an effective generalization of differential privacy beyond the discrete setting, we raise the problem of characterizing the optimal privacy-accuracy tradeoff by the metric geometry of the underly…
▽ More
Synthetic data are an attractive concept to enable privacy in data sharing. A fundamental question is how similar the privacy-preserving synthetic data are compared to the true data. Using metric privacy, an effective generalization of differential privacy beyond the discrete setting, we raise the problem of characterizing the optimal privacy-accuracy tradeoff by the metric geometry of the underlying space. We provide a partial solution to this problem in terms of the "entropic scale", a quantity that captures the multiscale geometry of a metric space via the behavior of its packing numbers. We illustrate the applicability of our privacy-accuracy tradeoff framework via a diverse set of examples of metric spaces.
△ Less
Submitted 1 May, 2024;
originally announced May 2024.
-
Improving Autoencoder Image Interpolation via Dynamic Optimal Transport
Authors:
Xue Feng,
Thomas Strohmer
Abstract:
Autoencoders are important generative models that, among others, have the ability to interpolate image sequences. However, interpolated images are usually not semantically meaningful.In this paper, motivated by dynamic optimal transport, we consider image interpolation as a mass transfer problem and propose a novel regularization term to penalize non-smooth and unrealistic changes in the interpola…
▽ More
Autoencoders are important generative models that, among others, have the ability to interpolate image sequences. However, interpolated images are usually not semantically meaningful.In this paper, motivated by dynamic optimal transport, we consider image interpolation as a mass transfer problem and propose a novel regularization term to penalize non-smooth and unrealistic changes in the interpolation result. Specifically, we define the path energy function for each path connecting the source and target images. The autoencoder is trained to generate the $L^2$ optimal transport geodesic path when decoding a linear interpolation of their latent codes. With a simple extension, this model can handle complicated environments, such as allowing mass transfer between obstacles and unbalanced optimal transport. A key feature of the proposed method is that it is physics-driven and can generate robust and realistic interpretation results even when only very limited training data are available.
△ Less
Submitted 13 April, 2024;
originally announced April 2024.
-
An Algorithm for Streaming Differentially Private Data
Authors:
Girish Kumar,
Thomas Strohmer,
Roman Vershynin
Abstract:
Much of the research in differential privacy has focused on offline applications with the assumption that all data is available at once. When these algorithms are applied in practice to streams where data is collected over time, this either violates the privacy guarantees or results in poor utility. We derive an algorithm for differentially private synthetic streaming data generation, especially c…
▽ More
Much of the research in differential privacy has focused on offline applications with the assumption that all data is available at once. When these algorithms are applied in practice to streams where data is collected over time, this either violates the privacy guarantees or results in poor utility. We derive an algorithm for differentially private synthetic streaming data generation, especially curated towards spatial datasets. Furthermore, we provide a general framework for online selective counting among a collection of queries which forms a basis for many tasks such as query answering and synthetic data generation. The utility of our algorithm is verified on both real-world and simulated datasets.
△ Less
Submitted 30 January, 2024; v1 submitted 25 January, 2024;
originally announced January 2024.
-
Auto-Calibration and Biconvex Compressive Sensing with Applications to Parallel MRI
Authors:
Yuan Ni,
Thomas Strohmer
Abstract:
We study an auto-calibration problem in which a transform-sparse signal is acquired via compressive sensing by multiple sensors in parallel, but with unknown calibration parameters of the sensors. This inverse problem has an important application in pMRI reconstruction, where the calibration parameters of the receiver coils are often difficult and costly to obtain explicitly, but nonetheless are a…
▽ More
We study an auto-calibration problem in which a transform-sparse signal is acquired via compressive sensing by multiple sensors in parallel, but with unknown calibration parameters of the sensors. This inverse problem has an important application in pMRI reconstruction, where the calibration parameters of the receiver coils are often difficult and costly to obtain explicitly, but nonetheless are a fundamental requirement for high-precision reconstructions. Most auto-calibration strategies for this problem involve solving a challenging biconvex optimization problem, which lacks reconstruction guarantees. In this work, we transform the auto-calibrated parallel compressive sensing problem to a convex optimization problem using the idea of `lifting'. By exploiting sparsity structures in the signal and the redundancy introduced by multiple sensors, we solve a mixed-norm minimization problem to recover the underlying signal and the sensing parameters simultaneously. Our method provides robust and stable recovery guarantees that take into account the presence of noise and sparsity deficiencies in the signals. As such, it offers a theoretically guaranteed approach to auto-calibrated parallel imaging in MRI under appropriate assumptions. Applications in compressive sensing pMRI are discussed, and numerical experiments using real and simulated MRI data are presented to support our theoretical results.
△ Less
Submitted 11 November, 2024; v1 submitted 18 January, 2024;
originally announced January 2024.
-
On the (In)Compatibility between Group Fairness and Individual Fairness
Authors:
Shizhou Xu,
Thomas Strohmer
Abstract:
We study the compatibility between the optimal statistical parity solutions and individual fairness. While individual fairness seeks to treat similar individuals similarly, optimal statistical parity aims to provide similar treatment to individuals who share relative similarity within their respective sensitive groups. The two fairness perspectives, while both desirable from a fairness perspective…
▽ More
We study the compatibility between the optimal statistical parity solutions and individual fairness. While individual fairness seeks to treat similar individuals similarly, optimal statistical parity aims to provide similar treatment to individuals who share relative similarity within their respective sensitive groups. The two fairness perspectives, while both desirable from a fairness perspective, often come into conflict in applications. Our goal in this work is to analyze the existence of this conflict and its potential solution. In particular, we establish sufficient (sharp) conditions for the compatibility between the optimal (post-processing) statistical parity $L^2$ learning and the ($K$-Lipschitz or $(ε,δ)$) individual fairness requirements. Furthermore, when there exists a conflict between the two, we first relax the former to the Pareto frontier (or equivalently the optimal trade-off) between $L^2$ error and statistical disparity, and then analyze the compatibility between the frontier and the individual fairness requirements. Our analysis identifies regions along the Pareto frontier that satisfy individual fairness requirements. (Lastly, we provide individual fairness guarantees for the composition of a trained model and the optimal post-processing step so that one can determine the compatibility of the post-processed model.) This provides practitioners with a valuable approach to attain Pareto optimality for statistical parity while adhering to the constraints of individual fairness.
△ Less
Submitted 13 January, 2024;
originally announced January 2024.
-
Differentially Private Low-dimensional Synthetic Data from High-dimensional Datasets
Authors:
Yiyun He,
Thomas Strohmer,
Roman Vershynin,
Yizhe Zhu
Abstract:
Differentially private synthetic data provide a powerful mechanism to enable data analysis while protecting sensitive information about individuals. However, when the data lie in a high-dimensional space, the accuracy of the synthetic data suffers from the curse of dimensionality. In this paper, we propose a differentially private algorithm to generate low-dimensional synthetic data efficiently fr…
▽ More
Differentially private synthetic data provide a powerful mechanism to enable data analysis while protecting sensitive information about individuals. However, when the data lie in a high-dimensional space, the accuracy of the synthetic data suffers from the curse of dimensionality. In this paper, we propose a differentially private algorithm to generate low-dimensional synthetic data efficiently from a high-dimensional dataset with a utility guarantee with respect to the Wasserstein distance. A key step of our algorithm is a private principal component analysis (PCA) procedure with a near-optimal accuracy bound that circumvents the curse of dimensionality. Unlike the standard perturbation analysis, our analysis of private PCA works without assuming the spectral gap for the covariance matrix.
△ Less
Submitted 11 December, 2024; v1 submitted 25 May, 2023;
originally announced May 2023.
-
Covariance loss, Szemeredi regularity, and differential privacy
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
We show how randomized rounding based on Grothendieck's identity can be used to prove a nearly tight bound on the covariance loss--the amount of covariance that is lost by taking conditional expectation. This result yields a new type of weak Szemeredi regularity lemma for positive semidefinite matrices and kernels. Moreover, it can be used to construct differentially private synthetic data.
We show how randomized rounding based on Grothendieck's identity can be used to prove a nearly tight bound on the covariance loss--the amount of covariance that is lost by taking conditional expectation. This result yields a new type of weak Szemeredi regularity lemma for positive semidefinite matrices and kernels. Moreover, it can be used to construct differentially private synthetic data.
△ Less
Submitted 6 January, 2023;
originally announced January 2023.
-
Semi-Supervised Clustering of Sparse Graphs: Crossing the Information-Theoretic Threshold
Authors:
Junda Sheng,
Thomas Strohmer
Abstract:
The stochastic block model is a canonical random graph model for clustering and community detection on network-structured data. Decades of extensive study on the problem have established many profound results, among which the phase transition at the Kesten-Stigum threshold is particularly interesting both from a mathematical and an applied standpoint. It states that no estimator based on the netwo…
▽ More
The stochastic block model is a canonical random graph model for clustering and community detection on network-structured data. Decades of extensive study on the problem have established many profound results, among which the phase transition at the Kesten-Stigum threshold is particularly interesting both from a mathematical and an applied standpoint. It states that no estimator based on the network topology can perform substantially better than chance on sparse graphs if the model parameter is below a certain threshold. Nevertheless, if we slightly extend the horizon to the ubiquitous semi-supervised setting, such a fundamental limitation will disappear completely. We prove that with an arbitrary fraction of the labels revealed, the detection problem is feasible throughout the parameter domain. Moreover, we introduce two efficient algorithms, one combinatorial and one based on optimization, to integrate label information with graph structures. Our work brings a new perspective to the stochastic model of networks and semidefinite program research.
△ Less
Submitted 27 February, 2024; v1 submitted 23 May, 2022;
originally announced May 2022.
-
Private measures, random walks, and synthetic data
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
Differential privacy is a mathematical concept that provides an information-theoretic security guarantee. While differential privacy has emerged as a de facto standard for guaranteeing privacy in data sharing, the known mechanisms to achieve it come with some serious limitations. Utility guarantees are usually provided only for a fixed, a priori specified set of queries. Moreover, there are no uti…
▽ More
Differential privacy is a mathematical concept that provides an information-theoretic security guarantee. While differential privacy has emerged as a de facto standard for guaranteeing privacy in data sharing, the known mechanisms to achieve it come with some serious limitations. Utility guarantees are usually provided only for a fixed, a priori specified set of queries. Moreover, there are no utility guarantees for more complex - but very common - machine learning tasks such as clustering or classification. In this paper we overcome some of these limitations. Working with metric privacy, a powerful generalization of differential privacy, we develop a polynomial-time algorithm that creates a private measure from a data set. This private measure allows us to efficiently construct private synthetic data that are accurate for a wide range of statistical analysis tools. Moreover, we prove an asymptotically sharp min-max result for private measures and synthetic data for general compact metric spaces. A key ingredient in our construction is a new superregular random walk, whose joint distribution of steps is as regular as that of independent random variables, yet which deviates from the origin logarithmicaly slowly.
△ Less
Submitted 23 March, 2024; v1 submitted 19 April, 2022;
originally announced April 2022.
-
Fair Data Representation for Machine Learning at the Pareto Frontier
Authors:
Shizhou Xu,
Thomas Strohmer
Abstract:
As machine learning powered decision-making becomes increasingly important in our daily lives, it is imperative to strive for fairness in the underlying data processing. We propose a pre-processing algorithm for fair data representation via which supervised learning results in estimations of the Pareto frontier between prediction error and statistical disparity. Particularly, the present work appl…
▽ More
As machine learning powered decision-making becomes increasingly important in our daily lives, it is imperative to strive for fairness in the underlying data processing. We propose a pre-processing algorithm for fair data representation via which supervised learning results in estimations of the Pareto frontier between prediction error and statistical disparity. Particularly, the present work applies the optimal affine transport to approach the post-processing Wasserstein-2 barycenter characterization of the optimal fair $L^2$-objective supervised learning via a pre-processing data deformation. Furthermore, we show that the Wasserstein-2 geodesics from the conditional (on sensitive information) distributions of the learning outcome to their barycenter characterizes the Pareto frontier between $L^2$-loss and the average pairwise Wasserstein-2 distance among sensitive groups on the learning outcome. Numerical simulations underscore the advantages: (1) the pre-processing step is compositive with arbitrary conditional expectation estimation supervised learning methods and unseen data; (2) the fair representation protects the sensitive information by limiting the inference capability of the remaining data with respect to the sensitive data; (3) the optimal affine maps are computationally efficient even for high-dimensional data.
△ Less
Submitted 24 November, 2023; v1 submitted 2 January, 2022;
originally announced January 2022.
-
Private sampling: a noiseless approach for generating differentially private synthetic data
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
In a world where artificial intelligence and data science become omnipresent, data sharing is increasingly locking horns with data-privacy concerns. Differential privacy has emerged as a rigorous framework for protecting individual privacy in a statistical database, while releasing useful statistical information about the database. The standard way to implement differential privacy is to inject a…
▽ More
In a world where artificial intelligence and data science become omnipresent, data sharing is increasingly locking horns with data-privacy concerns. Differential privacy has emerged as a rigorous framework for protecting individual privacy in a statistical database, while releasing useful statistical information about the database. The standard way to implement differential privacy is to inject a sufficient amount of noise into the data. However, in addition to other limitations of differential privacy, this process of adding noise will affect data accuracy and utility. Another approach to enable privacy in data sharing is based on the concept of synthetic data. The goal of synthetic data is to create an as-realistic-as-possible dataset, one that not only maintains the nuances of the original data, but does so without risk of exposing sensitive information. The combination of differential privacy with synthetic data has been suggested as a best-of-both-worlds solutions. In this work, we propose the first noisefree method to construct differentially private synthetic data; we do this through a mechanism called "private sampling". Using the Boolean cube as benchmark data model, we derive explicit bounds on accuracy and privacy of the constructed synthetic data. The key mathematical tools are hypercontractivity, duality, and empirical processes. A core ingredient of our private sampling mechanism is a rigorous "marginal correction" method, which has the remarkable property that importance reweighting can be utilized to exactly match the marginals of the sample to the marginals of the population.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Privacy of synthetic data: a statistical framework
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
Privacy-preserving data analysis is emerging as a challenging problem with far-reaching impact. In particular, synthetic data are a promising concept toward solving the aporetic conflict between data privacy and data sharing. Yet, it is known that accurately generating private, synthetic data of certain kinds is NP-hard. We develop a statistical framework for differentially private synthetic data,…
▽ More
Privacy-preserving data analysis is emerging as a challenging problem with far-reaching impact. In particular, synthetic data are a promising concept toward solving the aporetic conflict between data privacy and data sharing. Yet, it is known that accurately generating private, synthetic data of certain kinds is NP-hard. We develop a statistical framework for differentially private synthetic data, which enables us to circumvent the computational hardness of the problem. We consider the true data as a random sample drawn from a population Omega according to some unknown density. We then replace Omega by a much smaller random subset Omega^*, which we sample according to some known density. We generate synthetic data on the reduced space Omega^* by fitting the specified linear statistics obtained from the true data. To ensure privacy we use the common Laplacian mechanism. Employing the concept of Renyi condition number, which measures how well the sampling distribution is correlated with the population distribution, we derive explicit bounds on the privacy and accuracy provided by the proposed method.
△ Less
Submitted 3 September, 2021;
originally announced September 2021.
-
Covariance's Loss is Privacy's Gain: Computationally Efficient, Private and Accurate Synthetic Data
Authors:
March Boedihardjo,
Thomas Strohmer,
Roman Vershynin
Abstract:
The protection of private information is of vital importance in data-driven research, business, and government. The conflict between privacy and utility has triggered intensive research in the computer science and statistics communities, who have developed a variety of methods for privacy-preserving data release. Among the main concepts that have emerged are anonymity and differential privacy. Tod…
▽ More
The protection of private information is of vital importance in data-driven research, business, and government. The conflict between privacy and utility has triggered intensive research in the computer science and statistics communities, who have developed a variety of methods for privacy-preserving data release. Among the main concepts that have emerged are anonymity and differential privacy. Today, another solution is gaining traction, synthetic data. However, the road to privacy is paved with NP-hard problems. In this paper we focus on the NP-hard challenge to develop a synthetic data generation method that is computationally efficient, comes with provable privacy guarantees, and rigorously quantifies data utility. We solve a relaxed version of this problem by studying a fundamental, but a first glance completely unrelated, problem in probability concerning the concept of covariance loss. Namely, we find a nearly optimal and constructive answer to the question how much information is lost when we take conditional expectation. Surprisingly, this excursion into theoretical probability produces mathematical techniques that allow us to derive constructive, approximately optimal solutions to difficult applied problems concerning microaggregation, privacy, and synthetic data.
△ Less
Submitted 10 August, 2022; v1 submitted 12 July, 2021;
originally announced July 2021.
-
The Numerics of Phase Retrieval
Authors:
Albert Fannjiang,
Thomas Strohmer
Abstract:
Phase retrieval, i.e., the problem of recovering a function from the squared magnitude of its Fourier transform, arises in many applications such as X-ray crystallography, diffraction imaging, optics, quantum mechanics, and astronomy. This problem has confounded engineers, physicists, and mathematicians for many decades. Recently, phase retrieval has seen a resurgence in research activity, ignited…
▽ More
Phase retrieval, i.e., the problem of recovering a function from the squared magnitude of its Fourier transform, arises in many applications such as X-ray crystallography, diffraction imaging, optics, quantum mechanics, and astronomy. This problem has confounded engineers, physicists, and mathematicians for many decades. Recently, phase retrieval has seen a resurgence in research activity, ignited by new imaging modalities and novel mathematical concepts. As our scientific experiments produce larger and larger datasets and we aim for faster and faster throughput, it becomes increasingly important to study the involved numerical algorithms in a systematic and principled manner. Indeed, the last decade has witnessed a surge in the systematic study of computational algorithms for phase retrieval. In this paper we will review these recent advances from a numerical viewpoint.
△ Less
Submitted 13 April, 2020;
originally announced April 2020.
-
Certifying Global Optimality of Graph Cuts via Semidefinite Relaxation: A Performance Guarantee for Spectral Clustering
Authors:
Shuyang Ling,
Thomas Strohmer
Abstract:
Spectral clustering has become one of the most widely used clustering techniques when the structure of the individual clusters is non-convex or highly anisotropic. Yet, despite its immense popularity, there exists fairly little theory about performance guarantees for spectral clustering. This issue is partly due to the fact that spectral clustering typically involves two steps which complicated it…
▽ More
Spectral clustering has become one of the most widely used clustering techniques when the structure of the individual clusters is non-convex or highly anisotropic. Yet, despite its immense popularity, there exists fairly little theory about performance guarantees for spectral clustering. This issue is partly due to the fact that spectral clustering typically involves two steps which complicated its theoretical analysis: first, the eigenvectors of the associated graph Laplacian are used to embed the dataset, and second, k-means clustering algorithm is applied to the embedded dataset to get the labels. This paper is devoted to the theoretical foundations of spectral clustering and graph cuts. We consider a convex relaxation of graph cuts, namely ratio cuts and normalized cuts, that makes the usual two-step approach of spectral clustering obsolete and at the same time gives rise to a rigorous theoretical analysis of graph cuts and spectral clustering. We derive deterministic bounds for successful spectral clustering via a spectral proximity condition that naturally depends on the algebraic connectivity of each cluster and the inter-cluster connectivity. Moreover, we demonstrate by means of some popular examples that our bounds can achieve near-optimality. Our findings are also fundamental for the theoretical understanding of kernel k-means. Numerical simulations confirm and complement our analysis.
△ Less
Submitted 14 April, 2019; v1 submitted 29 June, 2018;
originally announced June 2018.
-
When Do Birds of a Feather Flock Together? k-Means, Proximity, and Conic Programming
Authors:
Xiaodong Li,
Yang Li,
Shuyang Ling,
Thomas Strohmer,
Ke Wei
Abstract:
Given a set of data, one central goal is to group them into clusters based on some notion of similarity between the individual objects. One of the most popular and widely-used approaches is k-means despite the computational hardness to find its global minimum. We study and compare the properties of different convex relaxations by relating them to corresponding proximity conditions, an idea origina…
▽ More
Given a set of data, one central goal is to group them into clusters based on some notion of similarity between the individual objects. One of the most popular and widely-used approaches is k-means despite the computational hardness to find its global minimum. We study and compare the properties of different convex relaxations by relating them to corresponding proximity conditions, an idea originally introduced by Kumar and Kannan. Using conic duality theory, we present an improved proximity condition under which the Peng-Wei relaxation of k-means recovers the underlying clusters exactly. Our proximity condition improves upon Kumar and Kannan, and is comparable to that of Awashti and Sheffet where proximity conditions are established for projective k-means. In addition, we provide a necessary proximity condition for the exactness of the Peng-Wei relaxation. For the special case of equal cluster sizes, we establish a different and completely localized proximity condition under which the Amini-Levina relaxation yields exact clustering, thereby having addressed an open problem by Awasthi and Sheffet in the balanced case. Our framework is not only deterministic and model-free but also comes with a clear geometric meaning which allows for further analysis and generalization. Moreover, it can be conveniently applied to analyzing various data generative models such as the stochastic ball models and Gaussian mixture models. With this method, we improve the current minimum separation bound for the stochastic ball models and achieve the state-of-the-art results of learning Gaussian mixture models.
△ Less
Submitted 22 July, 2018; v1 submitted 16 October, 2017;
originally announced October 2017.
-
Almost Eigenvalues and Eigenvectors of Almost Mathieu Operators
Authors:
Thomas Strohmer,
Tim Wertz
Abstract:
The almost Mathieu operator is the discrete Schrödinger operator $H_{α,β,θ}$ on $\ell^2(\mathbb{Z})$ defined via $(H_{α,β,θ}f)(k) = f(k + 1) + f(k - 1) + β\cos(2παk + θ) f(k)$. We derive explicit estimates for the eigenvalues at the edge of the spectrum of the finite-dimensional almost Mathieu operator. We furthermore show that the (properly rescaled) $m$-th Hermite function $φ_m$ is an approximat…
▽ More
The almost Mathieu operator is the discrete Schrödinger operator $H_{α,β,θ}$ on $\ell^2(\mathbb{Z})$ defined via $(H_{α,β,θ}f)(k) = f(k + 1) + f(k - 1) + β\cos(2παk + θ) f(k)$. We derive explicit estimates for the eigenvalues at the edge of the spectrum of the finite-dimensional almost Mathieu operator. We furthermore show that the (properly rescaled) $m$-th Hermite function $φ_m$ is an approximate eigenvector of this operator, and that it satisfies the same properties that characterize the true eigenvector associated to the $m$-th largest eigenvalue. Moreover, a properly translated and modulated version of $φ_m$ is also an approximate eigenvector of this operator, and it satisfies the properties that characterize the true eigenvector associated to the $m$-th largest (in modulus) negative eigenvalue. The results hold at the edge of the spectrum, for any choice of $θ$ and under very mild conditions on $α$ and $β$. We also give precise estimates for the size of the "edge", and extend some of our results to the infinite dimensional case. The ingredients for our proofs comprise Taylor expansions, basic time-frequency analysis, Sturm sequences, and perturbation theory for eigenvalues and eigenvectors. Numerical simulations demonstrate the tight fit of the theoretical estimates.
△ Less
Submitted 23 January, 2015;
originally announced January 2015.
-
Localization of Matrix Factorizations
Authors:
Ilya Krishtal,
Thomas Strohmer,
Tim Wertz
Abstract:
Matrices with off-diagonal decay appear in a variety of fields in mathematics and in numerous applications, such as signal processing, statistics, communications engineering, condensed matter physics, and quantum chemistry. Numerical algorithms dealing with such matrices often take advantage (implicitly or explicitly) of the empirical observation that this off-diagonal decay property seems to be p…
▽ More
Matrices with off-diagonal decay appear in a variety of fields in mathematics and in numerous applications, such as signal processing, statistics, communications engineering, condensed matter physics, and quantum chemistry. Numerical algorithms dealing with such matrices often take advantage (implicitly or explicitly) of the empirical observation that this off-diagonal decay property seems to be preserved when computing various useful matrix factorizations, such as the Cholesky factorization or the QR-factorization. There is a fairly extensive theory describing when the inverse of a matrix inherits the localization properties of the original matrix. Yet, except for the special case of band matrices, surprisingly very little theory exists that would establish similar results for matrix factorizations. We will derive a comprehensive framework to rigorously answer the question when and under which conditions the matrix factors inherit the localization of the original matrix for such fundamental matrix factorizations as the LU-, QR-, Cholesky, and Polar factorization.
△ Less
Submitted 7 May, 2013;
originally announced May 2013.
-
Accurate detection of moving targets via random sensor arrays and Kerdock codes
Authors:
Thomas Strohmer,
Haichao Wang
Abstract:
The detection and parameter estimation of moving targets is one of the most important tasks in radar. Arrays of randomly distributed antennas have been popular for this purpose for about half a century. Yet, surprisingly little rigorous mathematical theory exists for random arrays that addresses fundamental question such as how many targets can be recovered, at what resolution, at which noise leve…
▽ More
The detection and parameter estimation of moving targets is one of the most important tasks in radar. Arrays of randomly distributed antennas have been popular for this purpose for about half a century. Yet, surprisingly little rigorous mathematical theory exists for random arrays that addresses fundamental question such as how many targets can be recovered, at what resolution, at which noise level, and with which algorithm. In a different line of research in radar, mathematicians and engineers have invested significant effort into the design of radar transmission waveforms which satisfy various desirable properties. In this paper we bring these two seemingly unrelated areas together. Using tools from compressive sensing we derive a theoretical framework for the recovery of targets in the azimuth-range-Doppler domain via random antennas arrays. In one manifestation of our theory we use Kerdock codes as transmission waveforms and exploit some of their peculiar properties in our analysis. Our paper provides two main contributions: (i) We derive the first rigorous mathematical theory for the detection of moving targets using random sensor arrays. (ii) The transmitted waveforms satisfy a variety of properties that are very desirable and important from a practical viewpoint. Thus our approach does not just lead to useful theoretical insights, but is also of practical importance. Various extensions of our results are derived and numerical simulations confirming our theory are presented.
△ Less
Submitted 14 January, 2013;
originally announced January 2013.
-
Remote sensing via $\ell_1$ minimization
Authors:
Max Hügel,
Holger Rauhut,
Thomas Strohmer
Abstract:
We consider the problem of detecting the locations of targets in the far field by sending probing signals from an antenna array and recording the reflected echoes. Drawing on key concepts from the area of compressive sensing, we use an $\ell_1$-based regularization approach to solve this, in general ill-posed, inverse scattering problem. As common in compressed sensing, we exploit randomness, whic…
▽ More
We consider the problem of detecting the locations of targets in the far field by sending probing signals from an antenna array and recording the reflected echoes. Drawing on key concepts from the area of compressive sensing, we use an $\ell_1$-based regularization approach to solve this, in general ill-posed, inverse scattering problem. As common in compressed sensing, we exploit randomness, which in this context comes from choosing the antenna locations at random. With $n$ antennas we obtain $n^2$ measurements of a vector $x \in \C^{N}$ representing the target locations and reflectivities on a discretized grid. It is common to assume that the scene $x$ is sparse due to a limited number of targets. Under a natural condition on the mesh size of the grid, we show that an $s$-sparse scene can be recovered via $\ell_1$-minimization with high probability if $n^2 \geq C s \log^2(N)$. The reconstruction is stable under noise and under passing from sparse to approximately sparse vectors. Our theoretical findings are confirmed by numerical simulations.
△ Less
Submitted 24 April, 2013; v1 submitted 7 May, 2012;
originally announced May 2012.
-
Analysis of Sparse MIMO Radar
Authors:
Thomas Strohmer,
Benjamin Friedlander
Abstract:
We consider a multiple-input-multiple-output radar system and derive a theoretical framework for the recoverability of targets in the azimuth-range domain and the azimuth-range-Doppler domain via sparse approximation algorithms. Using tools developed in the area of compressive sensing, we prove bounds on the number of detectable targets and the achievable resolution in the presence of additive noi…
▽ More
We consider a multiple-input-multiple-output radar system and derive a theoretical framework for the recoverability of targets in the azimuth-range domain and the azimuth-range-Doppler domain via sparse approximation algorithms. Using tools developed in the area of compressive sensing, we prove bounds on the number of detectable targets and the achievable resolution in the presence of additive noise. Our theoretical findings are validated by numerical simulations.
△ Less
Submitted 12 March, 2012;
originally announced March 2012.
-
PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming
Authors:
Emmanuel J. Candes,
Thomas Strohmer,
Vladislav Voroninski
Abstract:
Suppose we wish to recover a signal x in C^n from m intensity measurements of the form |<x,z_i>|^2, i = 1, 2,..., m; that is, from data in which phase information is missing. We prove that if the vectors z_i are sampled independently and uniformly at random on the unit sphere, then the signal x can be recovered exactly (up to a global phase factor) by solving a convenient semidefinite program---a…
▽ More
Suppose we wish to recover a signal x in C^n from m intensity measurements of the form |<x,z_i>|^2, i = 1, 2,..., m; that is, from data in which phase information is missing. We prove that if the vectors z_i are sampled independently and uniformly at random on the unit sphere, then the signal x can be recovered exactly (up to a global phase factor) by solving a convenient semidefinite program---a trace-norm minimization problem; this holds with large probability provided that m is on the order of n log n, and without any assumption about the signal whatsoever. This novel result demonstrates that in some instances, the combinatorial phase retrieval problem can be solved by convex programming techniques. Finally, we also prove that our methodology is robust vis a vis additive noise.
△ Less
Submitted 21 September, 2011;
originally announced September 2011.
-
Phase Retrieval via Matrix Completion
Authors:
Emmanuel J. Candes,
Yonina Eldar,
Thomas Strohmer,
Vlad Voroninski
Abstract:
This paper develops a novel framework for phase retrieval, a problem which arises in X-ray crystallography, diffraction imaging, astronomical imaging and many other applications. Our approach combines multiple structured illuminations together with ideas from convex programming to recover the phase from intensity measurements, typically from the modulus of the diffracted wave. We demonstrate empir…
▽ More
This paper develops a novel framework for phase retrieval, a problem which arises in X-ray crystallography, diffraction imaging, astronomical imaging and many other applications. Our approach combines multiple structured illuminations together with ideas from convex programming to recover the phase from intensity measurements, typically from the modulus of the diffracted wave. We demonstrate empirically that any complex-valued object can be recovered from the knowledge of the magnitude of just a few diffracted patterns by solving a simple convex optimization problem inspired by the recent literature on matrix completion. More importantly, we also demonstrate that our noise-aware algorithms are stable in the sense that the reconstruction degrades gracefully as the signal-to-noise ratio decreases. Finally, we introduce some theory showing that one can design very simple structured illumination patterns such that three diffracted figures uniquely determine the phase of the object we wish to recover.
△ Less
Submitted 20 September, 2011; v1 submitted 2 September, 2011;
originally announced September 2011.
-
Performance Analysis of Spectral Clustering on Compressed, Incomplete and Inaccurate Measurements
Authors:
Blake Hunter,
Thomas Strohmer
Abstract:
Spectral clustering is one of the most widely used techniques for extracting the underlying global structure of a data set. Compressed sensing and matrix completion have emerged as prevailing methods for efficiently recovering sparse and partially observed signals respectively. We combine the distance preserving measurements of compressed sensing and matrix completion with the power of robust spec…
▽ More
Spectral clustering is one of the most widely used techniques for extracting the underlying global structure of a data set. Compressed sensing and matrix completion have emerged as prevailing methods for efficiently recovering sparse and partially observed signals respectively. We combine the distance preserving measurements of compressed sensing and matrix completion with the power of robust spectral clustering. Our analysis provides rigorous bounds on how small errors in the affinity matrix can affect the spectral coordinates and clusterability. This work generalizes the current perturbation results of two-class spectral clustering to incorporate multi-class clustering with k eigenvectors. We thoroughly track how small perturbation from using compressed sensing and matrix completion affect the affinity matrix and in succession the spectral coordinates. These perturbation results for multi-class clustering require an eigengap between the kth and (k+1)th eigenvalues of the affinity matrix, which naturally occurs in data with k well-defined clusters. Our theoretical guarantees are complemented with numerical results along with a number of examples of the unsupervised organization and clustering of image data.
△ Less
Submitted 3 November, 2010;
originally announced November 2010.
-
High-Resolution Radar via Compressed Sensing
Authors:
Matthew A. Herman,
Thomas Strohmer
Abstract:
A stylized compressed sensing radar is proposed in which the time-frequency plane is discretized into an N by N grid. Assuming the number of targets K is small (i.e., K much less than N^2), then we can transmit a sufficiently "incoherent" pulse and employ the techniques of compressed sensing to reconstruct the target scene. A theoretical upper bound on the sparsity K is presented. Numerical simu…
▽ More
A stylized compressed sensing radar is proposed in which the time-frequency plane is discretized into an N by N grid. Assuming the number of targets K is small (i.e., K much less than N^2), then we can transmit a sufficiently "incoherent" pulse and employ the techniques of compressed sensing to reconstruct the target scene. A theoretical upper bound on the sparsity K is presented. Numerical simulations verify that even better performance can be achieved in practice. This novel compressed sensing approach offers great potential for better resolution over classical radar.
△ Less
Submitted 22 December, 2008; v1 submitted 14 March, 2008;
originally announced March 2008.
-
Constructing packings in Grassmannian manifolds via alternating projection
Authors:
I. S. Dhillon,
R. W. Heath Jr,
T. Strohmer,
J. A. Tropp
Abstract:
This paper describes a numerical method for finding good packings in Grassmannian manifolds equipped with various metrics. This investigation also encompasses packing in projective spaces. In each case, producing a good packing is equivalent to constructing a matrix that has certain structural and spectral properties. By alternately enforcing the structural condition and then the spectral condit…
▽ More
This paper describes a numerical method for finding good packings in Grassmannian manifolds equipped with various metrics. This investigation also encompasses packing in projective spaces. In each case, producing a good packing is equivalent to constructing a matrix that has certain structural and spectral properties. By alternately enforcing the structural condition and then the spectral condition, it is often possible to reach a matrix that satisfies both. One may then extract a packing from this matrix.
This approach is both powerful and versatile. In cases where experiments have been performed, the alternating projection method yields packings that compete with the best packings recorded. It also extends to problems that have not been studied numerically. For example, it can be used to produce packings of subspaces in real and complex Grassmannian spaces equipped with the Fubini--Study distance; these packings are valuable in wireless communications. One can prove that some of the novel configurations constructed by the algorithm have packing diameters that are nearly optimal.
△ Less
Submitted 4 September, 2007;
originally announced September 2007.
-
A randomized Kaczmarz algorithm with exponential convergence
Authors:
Thomas Strohmer,
Roman Vershynin
Abstract:
The Kaczmarz method for solving linear systems of equations is an iterative algorithm that has found many applications ranging from computer tomography to digital signal processing. Despite the popularity of this method, useful theoretical estimates for its rate of convergence are still scarce. We introduce a randomized version of the Kaczmarz method for consistent, overdetermined linear systems…
▽ More
The Kaczmarz method for solving linear systems of equations is an iterative algorithm that has found many applications ranging from computer tomography to digital signal processing. Despite the popularity of this method, useful theoretical estimates for its rate of convergence are still scarce. We introduce a randomized version of the Kaczmarz method for consistent, overdetermined linear systems and we prove that it converges with expected exponential rate. Furthermore, this is the first solver whose rate does not depend on the number of equations in the system. The solver does not even need to know the whole system, but only a small random part of it. It thus outperforms all previously known methods on general extremely overdetermined systems. Even for moderately overdetermined systems, numerical simulations as well as theoretical analysis reveal that our algorithm can converge faster than the celebrated conjugate gradient algorithm. Furthermore, our theory and numerical simulations confirm a prediction of Feichtinger et al. in the context of reconstructing bandlimited functions from nonuniform sampling.
△ Less
Submitted 8 February, 2007;
originally announced February 2007.
-
Inverse-Closedness of a Banach Algebra of Integral Operators on the Heisenberg Group
Authors:
Brendan Farrell,
Thomas Strohmer
Abstract:
Let $\mathbb{H}$ be the general, reduced Heisenberg group. Our main result establishes the inverse-closedness of a class of integral operators acting on $L^{p}(\mathbb{H})$, given by the off-diagonal decay of the kernel. As a consequence of this result, we show that if $α_{1}I+S_{f}$, where $S_{f}$ is the operator given by convolution with $f$, $f\in L^{1}_{v}(\mathbb{H})$, is invertible in…
▽ More
Let $\mathbb{H}$ be the general, reduced Heisenberg group. Our main result establishes the inverse-closedness of a class of integral operators acting on $L^{p}(\mathbb{H})$, given by the off-diagonal decay of the kernel. As a consequence of this result, we show that if $α_{1}I+S_{f}$, where $S_{f}$ is the operator given by convolution with $f$, $f\in L^{1}_{v}(\mathbb{H})$, is invertible in $\B(L^{p}(\mathbb{H}))$, then (α_{1}I+S_{f})^{-1}=α_{2}I+S_{g}$, and $g\in L^{1}_{v}(\mathbb{H})$. We prove analogous results for twisted convolution operators and apply the latter results to a class of Weyl pseudodifferential operators. We briefly discuss relevance to mobile communications.
△ Less
Submitted 5 December, 2007; v1 submitted 1 December, 2006;
originally announced December 2006.
-
Quantitative Estimates for the Finite Section Method
Authors:
Karlheinz Gröchenig,
Ziemowit Rzeszotnik,
Thomas Strohmer
Abstract:
The finite section method is a classical scheme to approximate the solution of an infinite system of linear equations. We present quantitative estimates for the rate of the convergence of the finite section method on weighted $\ell ^p$-spaces. Our approach uses recent results from the theory of Banach algebras of matrices with off-diagonal decay. Furthermore, we demonstrate that Banach algebra t…
▽ More
The finite section method is a classical scheme to approximate the solution of an infinite system of linear equations. We present quantitative estimates for the rate of the convergence of the finite section method on weighted $\ell ^p$-spaces. Our approach uses recent results from the theory of Banach algebras of matrices with off-diagonal decay. Furthermore, we demonstrate that Banach algebra theory provides a natural framework for deriving a finite section method that is applicable to large classes of non-hermitian matrices. An example from digital communication illustrates the practical usefulness of the proposed theoretical framework.
△ Less
Submitted 19 October, 2006;
originally announced October 2006.
-
Pseudodifferential Operators on Locally Compact Abelian Groups and Sjoestrand's Symbol Class
Authors:
Karlheinz Grochenig,
Thomas Strohmer
Abstract:
We investigate pseudodifferential operators on arbitrary locally compact abelian groups. As symbol classes for the Kohn-Nirenberg calculus we introduce a version of Sjoestrand's class. Pseudodifferential operators with such symbols form a Banach algebra that is closed under inversion. Since "hard analysis" techniques are not available on locally compact abelian groups, a new time-frequency appro…
▽ More
We investigate pseudodifferential operators on arbitrary locally compact abelian groups. As symbol classes for the Kohn-Nirenberg calculus we introduce a version of Sjoestrand's class. Pseudodifferential operators with such symbols form a Banach algebra that is closed under inversion. Since "hard analysis" techniques are not available on locally compact abelian groups, a new time-frequency approach is used with the emphasis on modulation spaces, Gabor frames, and Banach algebras of matrices. Sjoestrand's original results are thus understood as a phenomenon of abstract harmonic analysis rather than "hard analysis" and are proved in their natural context and generality.
△ Less
Submitted 12 April, 2006;
originally announced April 2006.
-
Wilson bases for general time-frequency lattices
Authors:
Gitta Kutyniok,
Thomas Strohmer
Abstract:
Motivated by a recent generalization of the Balian-Low theorem and by new research in wireless communications we analyze the construction of Wilson bases for general time-frequency lattices. We show that orthonormal Wilson bases for $\LtR$ can be constructed for any time-frequency lattice whose volume is $\tfrac12$. We then focus on the spaces $\ell^2(\ZZ)$ and $\CC^L$ which are the preferred se…
▽ More
Motivated by a recent generalization of the Balian-Low theorem and by new research in wireless communications we analyze the construction of Wilson bases for general time-frequency lattices. We show that orthonormal Wilson bases for $\LtR$ can be constructed for any time-frequency lattice whose volume is $\tfrac12$. We then focus on the spaces $\ell^2(\ZZ)$ and $\CC^L$ which are the preferred settings for numerical and practical purposes. We demonstrate that with a properly adapted definition of Wilson bases the construction of orthonormal Wilson bases for general time-frequency lattices also holds true in these discrete settings. In our analysis we make use of certain metaplectic transforms. Finally we discuss some practical consequences of our theoretical findings.
△ Less
Submitted 15 November, 2004; v1 submitted 21 November, 2003;
originally announced November 2003.
-
Fast multi-dimensional scattered data approximation with Neumann boundary conditions
Authors:
Denis Grishin,
Thomas Strohmer
Abstract:
An important problem in applications is the approximation of a function $f$ from a finite set of randomly scattered data $f(x_j)$. A common and powerful approach is to construct a trigonometric least squares approximation based on the set of exponentials $\{e^{2πi kx}\}$. This leads to fast numerical algorithms, but suffers from disturbing boundary effects due to the underlying periodicity assum…
▽ More
An important problem in applications is the approximation of a function $f$ from a finite set of randomly scattered data $f(x_j)$. A common and powerful approach is to construct a trigonometric least squares approximation based on the set of exponentials $\{e^{2πi kx}\}$. This leads to fast numerical algorithms, but suffers from disturbing boundary effects due to the underlying periodicity assumption on the data, an assumption that is rarely satisfied in practice. To overcome this drawback we impose Neumann boundary conditions on the data. This implies the use of cosine polynomials $\cos (πkx)$ as basis functions. We show that scattered data approximation using cosine polynomials leads to a least squares problem involving certain Toeplitz+Hankel matrices. We derive estimates on the condition number of these matrices. Unlike other Toeplitz+Hankel matrices, the Toeplitz+Hankel matrices arising in our context cannot be diagonalized by the discrete cosine transform, but they still allow a fast matrix-vector multiplication via DCT which gives rise to fast conjugate gradient type algorithms. We show how the results can be generalized to higher dimensions. Finally we demonstrate the performance of the proposed method by applying it to a two-dimensional geophysical scattered data problem.
△ Less
Submitted 14 January, 2003;
originally announced January 2003.
-
Grassmannian Frames with Applications to Coding and Communication
Authors:
Thomas Strohmer,
Robert Heath
Abstract:
For a given class ${\cal F}$ of uniform frames of fixed redundancy we define a Grassmannian frame as one that minimizes the maximal correlation $|< f_k,f_l >|$ among all frames $\{f_k\}_{k \in {\cal I}} \in {\cal F}$. We first analyze finite-dimensional Grassmannian frames. Using links to packings in Grassmannian spaces and antipodal spherical codes we derive bounds on the minimal achievable cor…
▽ More
For a given class ${\cal F}$ of uniform frames of fixed redundancy we define a Grassmannian frame as one that minimizes the maximal correlation $|< f_k,f_l >|$ among all frames $\{f_k\}_{k \in {\cal I}} \in {\cal F}$. We first analyze finite-dimensional Grassmannian frames. Using links to packings in Grassmannian spaces and antipodal spherical codes we derive bounds on the minimal achievable correlation for Grassmannian frames. These bounds yield a simple condition under which Grassmannian frames coincide with uniform tight frames. We exploit connections to graph theory, equiangular line sets, and coding theory in order to derive explicit constructions of Grassmannian frames. Our findings extend recent results on uniform tight frames. We then introduce infinite-dimensional Grassmannian frames and analyze their connection to uniform tight frames for frames which are generated by group-like unitary systems. We derive an example of a Grassmannian Gabor frame by using connections to sphere packing theory. Finally we discuss the application of Grassmannian frames to wireless communication and to multiple description coding.
△ Less
Submitted 13 January, 2003;
originally announced January 2003.
-
Hyperbolic secants yield Gabor frames
Authors:
A. J. E. M. Janssen,
Thomas Strohmer
Abstract:
We show that $(g_2,a,b)$ is a Gabor frame when $a>0, b>0, ab<1$ and $g_2(t)=({1/2}πγ)^{1/2} (\cosh πγt)^{-1}$ is a hyperbolic secant with scaling parameter $γ>0$. This is accomplished by expressing the Zak transform of $g_2$ in terms of the Zak transform of the Gaussian $g_1(t)=(2γ)^{1/4} \exp (-πγt^2)$, together with an appropriate use of the Ron-Shen criterion for being a Gabor frame. As a sid…
▽ More
We show that $(g_2,a,b)$ is a Gabor frame when $a>0, b>0, ab<1$ and $g_2(t)=({1/2}πγ)^{1/2} (\cosh πγt)^{-1}$ is a hyperbolic secant with scaling parameter $γ>0$. This is accomplished by expressing the Zak transform of $g_2$ in terms of the Zak transform of the Gaussian $g_1(t)=(2γ)^{1/4} \exp (-πγt^2)$, together with an appropriate use of the Ron-Shen criterion for being a Gabor frame. As a side result it follows that the windows, generating tight Gabor frames, that are canonically associated to $g_2$ and $g_1$ are the same at critical density $a=b=1$. Also, we display the ``singular'' dual function corresponding to the hyperbolic secant at critical density.
△ Less
Submitted 13 January, 2003;
originally announced January 2003.
-
Characterization and computation of canonical tight windows for Gabor frames
Authors:
A. J. E. M Janssen,
Thomas Strohmer
Abstract:
Let $(g_{nm})_{n,m\in Z}$ be a Gabor frame for $L_2(R)$ for given window $g$. We show that the window $h^0=S^{-1/2} g$ that generates the canonically associated tight Gabor frame minimizes $\|g-h\|$ among all windows $h$ generating a normalized tight Gabor frame. We present and prove versions of this result in the time domain, the frequency domain, the time-frequency domain, and the Zak transfor…
▽ More
Let $(g_{nm})_{n,m\in Z}$ be a Gabor frame for $L_2(R)$ for given window $g$. We show that the window $h^0=S^{-1/2} g$ that generates the canonically associated tight Gabor frame minimizes $\|g-h\|$ among all windows $h$ generating a normalized tight Gabor frame. We present and prove versions of this result in the time domain, the frequency domain, the time-frequency domain, and the Zak transform domain, where in each domain the canonical $h^0$ is expressed using functional calculus for Gabor frame operators. Furthermore, we derive a Wiener-Levy type theorem for rationally oversampled Gabor frames. Finally, a Newton-type method for a fast numerical calculation of $\ho$ is presented. We analyze the convergence behavior of this method and demonstrate the efficiency of the proposed algorithm by some numerical examples.
△ Less
Submitted 25 October, 2000;
originally announced October 2000.
-
Approximation of dual Gabor frames, window decay, and wireless communications
Authors:
Thomas Strohmer
Abstract:
We consider three problems for Gabor frames that have recently received much attention. The first problem concerns the approximation of dual Gabor frames in $L_2(R)$ by finite-dimensional methods. Utilizing Wexler-Raz type duality relations we derive a method to approximate the dual Gabor frame, that is much simpler than previously proposed techniques. Furthermore it enables us to give estimates…
▽ More
We consider three problems for Gabor frames that have recently received much attention. The first problem concerns the approximation of dual Gabor frames in $L_2(R)$ by finite-dimensional methods. Utilizing Wexler-Raz type duality relations we derive a method to approximate the dual Gabor frame, that is much simpler than previously proposed techniques. Furthermore it enables us to give estimates for the approximation rate when the dimension of the finite model approaches infinity. The second problem concerns the relation between the decay of the window function $g$ and its dual $γ$. Based on results on commutative Banach algebras and Laurent operators we derive a general condition under which the dual $γ$ inherits the decay properties of $g$. The third problem concerns the design of pulse shapes for orthogonal frequency division multiplex (OFDM) systems for time- and frequency dispersive channels. In particular, we provide a theoretical foundation for a recently proposed algorithm to construct orthogonal transmission functions that are well localized in the time-frequency plane.
△ Less
Submitted 25 October, 2000;
originally announced October 2000.
-
Four short stories about Toeplitz matrix calculations
Authors:
Thomas Strohmer
Abstract:
The stories told in this paper are dealing with the solution of finite, infinite, and biinfinite Toeplitz-type systems. A crucial role plays the off-diagonal decay behavior of Toeplitz matrices and their inverses. Classical results of Gelfand et al. on commutative Banach algebras yield a general characterization of this decay behavior. We then derive estimates for the approximate solution of (bi…
▽ More
The stories told in this paper are dealing with the solution of finite, infinite, and biinfinite Toeplitz-type systems. A crucial role plays the off-diagonal decay behavior of Toeplitz matrices and their inverses. Classical results of Gelfand et al. on commutative Banach algebras yield a general characterization of this decay behavior. We then derive estimates for the approximate solution of (bi)infinite Toeplitz systems by the finite section method, showing that the approximation rate depends only on the decay of the entries of the Toeplitz matrix and its condition number. Furthermore, we give error estimates for the solution of doubly infinite convolution systems by finite circulant systems. Finally, some quantitative results on the construction of preconditioners via circulant embedding are derived, which allow to provide a theoretical explanation for numerical observations made by some researchers in connection with deconvolution problems.
△ Less
Submitted 25 October, 2000;
originally announced October 2000.
-
Numerical Analysis of the Non-uniform Sampling Problem
Authors:
Thomas Strohmer
Abstract:
We give an overview of recent developments in the problem of reconstructing a band-limited signal from non-uniform sampling from a numerical analysis view point. It is shown that the appropriate design of the finite-dimensional model plays a key role in the numerical solution of the non-uniform sampling problem. In the one approach (often proposed in the literature) the finite-dimensional model…
▽ More
We give an overview of recent developments in the problem of reconstructing a band-limited signal from non-uniform sampling from a numerical analysis view point. It is shown that the appropriate design of the finite-dimensional model plays a key role in the numerical solution of the non-uniform sampling problem. In the one approach (often proposed in the literature) the finite-dimensional model leads to an ill-posed problem even in very simple situations. The other approach that we consider leads to a well-posed problem that preserves important structural properties of the original infinite-dimensional problem and gives rise to efficient numerical algorithms. Furthermore a fast multilevel algorithm is presented that can reconstruct signals of unknown bandwidth from noisy non-uniformly spaced samples. We also discuss the design of efficient regularization methods for ill-conditioned reconstruction problems. Numerical examples from spectroscopy and exploration geophysics demonstrate the performance of the proposed methods.
△ Less
Submitted 25 October, 2000;
originally announced October 2000.
-
A Levinson-Galerkin algorithm for regularized trigonometric approximation
Authors:
Thomas Strohmer
Abstract:
Trigonometric polynomials are widely used for the approximation of a smooth function $f$ from a set of nonuniformly spaced samples $\{f(x_j)\}_{j=0}^{N-1}$. If the samples are perturbed by noise, controlling the smoothness of the trigonometric approximation becomes an essential issue to avoid overfitting and underfitting of the data. Using the polynomial degree as regularization parameter we der…
▽ More
Trigonometric polynomials are widely used for the approximation of a smooth function $f$ from a set of nonuniformly spaced samples $\{f(x_j)\}_{j=0}^{N-1}$. If the samples are perturbed by noise, controlling the smoothness of the trigonometric approximation becomes an essential issue to avoid overfitting and underfitting of the data. Using the polynomial degree as regularization parameter we derive a multi-level algorithm that iteratively adapts to the least squares solution of optimal smoothness. The proposed algorithm computes the solution in at most $\cal{O}(NM + M^2)$ operations ($M$ being the polynomial degree of the approximation) by solving a family of nested Toeplitz systems. It is shown how the presented method can be extended to multivariate trigonometric approximation. We demonstrate the performance of the algorithm by applying it in echocardiography to the recovery of the boundary of the Left Ventricle.
△ Less
Submitted 25 October, 2000; v1 submitted 26 January, 1999;
originally announced January 1999.
-
Rates of convergence for the approximation of dual shift-invariant systems in $l_2(Z)$
Authors:
Thomas Strohmer
Abstract:
A shift-invariant system is a collection of functions $\{g_{m,n}\}$ of the form $g_{m,n}(k) = g_m(k-an)$. Such systems play an important role in time-frequency analysis and digital signal processing. A principal problem is to find a dual system $γ_{m,n}(k) = γ_m(k-an)$ such that each function $f$ can be written as $f = \sum < f, γ_{m,n} > g_{m,n}$. The mathematical theory usually addresses this…
▽ More
A shift-invariant system is a collection of functions $\{g_{m,n}\}$ of the form $g_{m,n}(k) = g_m(k-an)$. Such systems play an important role in time-frequency analysis and digital signal processing. A principal problem is to find a dual system $γ_{m,n}(k) = γ_m(k-an)$ such that each function $f$ can be written as $f = \sum < f, γ_{m,n} > g_{m,n}$. The mathematical theory usually addresses this problem in infinite dimensions (typically in $L_2(R)$ or $l_2(Z)$), whereas numerical methods have to operate with a finite-dimensional model. Exploiting the link between the frame operator and Laurent operators with matrix-valued symbol, we apply the finite section method to show that the dual functions obtained by solving a finite-dimensional problem converge to the dual functions of the original infinite-dimensional problem in $l_2(Z)$. For compactly supported $g_{m,n}$ (FIR filter banks) we prove an exponential rate of convergence and derive explicit expressions for the involved constants. Further we investigate under which conditions one can replace the discrete model of the finite section method by the periodic discrete model, which is used in many numerical procedures. Again we provide explicit estimates for the speed of convergence. Some remarks on tight frames complete the paper.
△ Less
Submitted 26 January, 1999;
originally announced January 1999.
-
A multi-level algorithm for the solution of moment problems
Authors:
Otmar Scherzer,
Thomas Strohmer
Abstract:
We study numerical methods for the solution of general linear moment problems, where the solution belongs to a family of nested subspaces of a Hilbert space. Multi-level algorithms, based on the conjugate gradient method and the Landweber--Richardson method are proposed that determine the "optimal" reconstruction level a posteriori from quantities that arise during the numerical calculations. As…
▽ More
We study numerical methods for the solution of general linear moment problems, where the solution belongs to a family of nested subspaces of a Hilbert space. Multi-level algorithms, based on the conjugate gradient method and the Landweber--Richardson method are proposed that determine the "optimal" reconstruction level a posteriori from quantities that arise during the numerical calculations. As an important example we discuss the reconstruction of band-limited signals from irregularly spaced noisy samples, when the actual bandwidth of the signal is not available. Numerical examples show the usefulness of the proposed algorithms.
△ Less
Submitted 26 January, 1999;
originally announced January 1999.
