-
Performance Gaps in Multi-view Clustering under the Nested Matrix-Tensor Model
Authors:
Hugo Lebeau,
Mohamed El Amine Seddik,
José Henrique de Morais Goulart
Abstract:
We study the estimation of a planted signal hidden in a recently introduced nested matrix-tensor model, which is an extension of the classical spiked rank-one tensor model, motivated by multi-view clustering. Prior work has theoretically examined the performance of a tensor-based approach, which relies on finding a best rank-one approximation, a problem known to be computationally hard. A tractabl…
▽ More
We study the estimation of a planted signal hidden in a recently introduced nested matrix-tensor model, which is an extension of the classical spiked rank-one tensor model, motivated by multi-view clustering. Prior work has theoretically examined the performance of a tensor-based approach, which relies on finding a best rank-one approximation, a problem known to be computationally hard. A tractable alternative approach consists in computing instead the best rank-one (matrix) approximation of an unfolding of the observed tensor data, but its performance was hitherto unknown. We quantify here the performance gap between these two approaches, in particular by deriving the precise algorithmic threshold of the unfolding approach and demonstrating that it exhibits a BBP-type transition behavior. This work is therefore in line with recent contributions which deepen our understanding of why tensor-based methods surpass matrix-based methods in handling structured tensor data.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
Majorization-minimization for Sparse Nonnegative Matrix Factorization with the $β$-divergence
Authors:
Arthur Marmin,
José Henrique de Morais Goulart,
Cédric Févotte
Abstract:
This article introduces new multiplicative updates for nonnegative matrix factorization with the $β$-divergence and sparse regularization of one of the two factors (say, the activation matrix). It is well known that the norm of the other factor (the dictionary matrix) needs to be controlled in order to avoid an ill-posed formulation. Standard practice consists in constraining the columns of the di…
▽ More
This article introduces new multiplicative updates for nonnegative matrix factorization with the $β$-divergence and sparse regularization of one of the two factors (say, the activation matrix). It is well known that the norm of the other factor (the dictionary matrix) needs to be controlled in order to avoid an ill-posed formulation. Standard practice consists in constraining the columns of the dictionary to have unit norm, which leads to a nontrivial optimization problem. Our approach leverages a reparametrization of the original problem into the optimization of an equivalent scale-invariant objective function. From there, we derive block-descent majorization-minimization algorithms that result in simple multiplicative updates for either $\ell_{1}$-regularization or the more "aggressive" log-regularization. In contrast with other state-of-the-art methods, our algorithms are universal in the sense that they can be applied to any $β$-divergence (i.e., any value of $β$) and that they come with convergence guarantees. We report numerical comparisons with existing heuristic and Lagrangian methods using various datasets: face images, an audio spectrogram, hyperspectral data, and song play counts. We show that our methods obtain solutions of similar quality at convergence (similar objective values) but with significantly reduced CPU times.
△ Less
Submitted 12 March, 2024; v1 submitted 13 July, 2022;
originally announced July 2022.
-
COL0RME: Super-resolution microscopy based on sparse blinking/fluctuating fluorophore localization and intensity estimation
Authors:
Vasiliki Stergiopoulou,
Luca Calatroni,
José Henrique de Morais Goulart,
Sébastien Schaub,
Laure Blanc-Féraud
Abstract:
To overcome the physical barriers caused by light diffraction, super-resolution techniques are often applied in fluorescence microscopy. State-of-the-art approaches require specific and often demanding acquisition conditions to achieve adequate levels of both spatial and temporal resolution. Analyzing the stochastic fluctuations of the fluorescent molecules provides a solution to the aforementione…
▽ More
To overcome the physical barriers caused by light diffraction, super-resolution techniques are often applied in fluorescence microscopy. State-of-the-art approaches require specific and often demanding acquisition conditions to achieve adequate levels of both spatial and temporal resolution. Analyzing the stochastic fluctuations of the fluorescent molecules provides a solution to the aforementioned limitations, as sufficiently high spatio-temporal resolution for live-cell imaging can be achieved by using common microscopes and conventional fluorescent dyes. Based on this idea, we present COL0RME, a method for COvariance-based $\ell_0$ super-Resolution Microscopy with intensity Estimation, which achieves good spatio-temporal resolution by solving a sparse optimization problem in the covariance domain and discuss automatic parameter selection strategies. The method is composed of two steps: the former where both the emitters' independence and the sparse distribution of the fluorescent molecules are exploited to provide an accurate localization; the latter where real intensity values are estimated given the computed support. The paper is furnished with several numerical results both on synthetic and real fluorescence microscopy images and several comparisons with state-of-the art approaches are provided. Our results show that COL0RME outperforms competing methods exploiting analogously temporal fluctuations; in particular, it achieves better localization, reduces background artifacts and avoids fine parameter tuning.
△ Less
Submitted 30 March, 2022; v1 submitted 16 August, 2021;
originally announced August 2021.
-
A Random Matrix Perspective on Random Tensors
Authors:
José Henrique de Morais Goulart,
Romain Couillet,
Pierre Comon
Abstract:
Tensor models play an increasingly prominent role in many fields, notably in machine learning. In several applications, such as community detection, topic modeling and Gaussian mixture learning, one must estimate a low-rank signal from a noisy tensor. Hence, understanding the fundamental limits of estimators of that signal inevitably calls for the study of random tensors. Substantial progress has…
▽ More
Tensor models play an increasingly prominent role in many fields, notably in machine learning. In several applications, such as community detection, topic modeling and Gaussian mixture learning, one must estimate a low-rank signal from a noisy tensor. Hence, understanding the fundamental limits of estimators of that signal inevitably calls for the study of random tensors. Substantial progress has been recently achieved on this subject in the large-dimensional limit. Yet, some of the most significant among these results--in particular, a precise characterization of the abrupt phase transition (with respect to signal-to-noise ratio) that governs the performance of the maximum likelihood (ML) estimator of a symmetric rank-one model with Gaussian noise--were derived based of mean-field spin glass theory, which is not easily accessible to non-experts. In this work, we develop a sharply distinct and more elementary approach, relying on standard but powerful tools brought by years of advances in random matrix theory. The key idea is to study the spectra of random matrices arising from contractions of a given random tensor. We show how this gives access to spectral properties of the random tensor itself. For the aforementioned rank-one model, our technique yields a hitherto unknown fixed-point equation whose solution precisely matches the asymptotic performance of the ML estimator above the phase transition threshold in the third-order case. A numerical verification provides evidence that the same holds for orders 4 and 5, leading us to conjecture that, for any order, our fixed-point equation is equivalent to the known characterization of the ML estimation performance that had been obtained by relying on spin glasses. Moreover, our approach sheds light on certain properties of the ML problem landscape in large dimensions and can be extended to other models, such as asymmetric and non-Gaussian.
△ Less
Submitted 15 June, 2022; v1 submitted 2 August, 2021;
originally announced August 2021.
-
COL0RME: COvariance-based $\ell_0$ super-Resolution Microscopy with intensity Estimation
Authors:
Vasiliki Stergiopoulou,
José Henrique de Morais Goulart,
Sébastien Schaub,
Luca Calatroni,
Laure Blanc-Féraud
Abstract:
Super-resolution light microscopy overcomes the physical barriers due to light diffraction, allowing for the observation of otherwise indistinguishable subcellular entities. However, the specific acquisition conditions required by state-of-the-art super-resolution methods to achieve adequate spatio-temporal resolution are often very challenging. Exploiting molecules fluctuations allows good spatio…
▽ More
Super-resolution light microscopy overcomes the physical barriers due to light diffraction, allowing for the observation of otherwise indistinguishable subcellular entities. However, the specific acquisition conditions required by state-of-the-art super-resolution methods to achieve adequate spatio-temporal resolution are often very challenging. Exploiting molecules fluctuations allows good spatio-temporal resolution live-cell imaging by means of common microscopes and conventional fluorescent dyes. In this work, we present the method COL0RME for COvariance-based $\ell_0$ super-Resolution Microscopy with intensity Estimation. It codifies the assumption of sparse distribution of the fluorescent molecules as well as the temporal and spatial independence between emitters via a non-convex optimization problem formulated in the covariance domain. In order to deal with real data, the proposed approach also estimates background and noise statistics. It also includes a final estimation step where intensity information is retrieved, which is valuable for biological interpretation and future applications to super-resolution imaging.
△ Less
Submitted 12 July, 2022; v1 submitted 26 October, 2020;
originally announced October 2020.
-
On the minimal ranks of matrix pencils and the existence of a best approximate block-term tensor decomposition
Authors:
José Henrique de Morais Goulart,
Pierre Comon
Abstract:
Under the action of the general linear group with tensor structure, the ranks of matrices $A$ and $B$ forming an $m \times n$ pencil $A + λB$ can change, but in a restricted manner. Specifically, with every pencil one can associate a pair of minimal ranks, which is unique up to a permutation. This notion can be defined for matrix pencils and, more generally, also for matrix polynomials of arbitrar…
▽ More
Under the action of the general linear group with tensor structure, the ranks of matrices $A$ and $B$ forming an $m \times n$ pencil $A + λB$ can change, but in a restricted manner. Specifically, with every pencil one can associate a pair of minimal ranks, which is unique up to a permutation. This notion can be defined for matrix pencils and, more generally, also for matrix polynomials of arbitrary degree. In this paper, we provide a formal definition of the minimal ranks, discuss its properties and the natural hierarchy it induces in a pencil space. Then, we show how the minimal ranks of a pencil can be determined from its Kronecker canonical form. For illustration, we classify the orbits according to their minimal ranks (under the action of the general linear group) in the case of real pencils with $m, n \le 4$. Subsequently, we show that real regular $2k \times 2k$ pencils having only complex-valued eigenvalues, which form an open positive-volume set, do not admit a best approximation (in the norm topology) on the set of real pencils whose minimal ranks are bounded by $2k-1$. Our results can be interpreted from a tensor viewpoint, where the minimal ranks of a degree-$(d-1)$ matrix polynomial characterize the minimal ranks of matrices constituting a block-term decomposition of an $m \times n \times d$ tensor into a sum of matrix-vector tensor products.
△ Less
Submitted 19 June, 2018; v1 submitted 15 December, 2017;
originally announced December 2017.
-
Statistical efficiency of structured cpd estimation applied to Wiener-Hammerstein modeling
Authors:
José Henrique De Morais Goulart,
Maxime Boizard,
Rémy Boyer,
Gérard Favier,
Pierre Comon
Abstract:
The computation of a structured canonical polyadic decomposition (CPD) is useful to address several important modeling problems in real-world applications. In this paper, we consider the identification of a nonlinear system by means of a Wiener-Hammerstein model, assuming a high-order Volterra kernel of that system has been previously estimated. Such a kernel, viewed as a tensor, admits a CPD with…
▽ More
The computation of a structured canonical polyadic decomposition (CPD) is useful to address several important modeling problems in real-world applications. In this paper, we consider the identification of a nonlinear system by means of a Wiener-Hammerstein model, assuming a high-order Volterra kernel of that system has been previously estimated. Such a kernel, viewed as a tensor, admits a CPD with banded circulant factors which comprise the model parameters. To estimate them, we formulate specialized estimators based on recently proposed algorithms for the computation of structured CPDs. Then, considering the presence of additive white Gaussian noise, we derive a closed-form expression for the Cramer-Rao bound (CRB) associated with this estimation problem. Finally, we assess the statistical performance of the proposed estimators via Monte Carlo simulations, by comparing their mean-square error with the CRB.
△ Less
Submitted 24 June, 2015; v1 submitted 24 February, 2015;
originally announced February 2015.
