-
Inferring the parameters of Taylor's law in ecology
Authors:
Lionel Truquet,
Joel E. Cohen,
Paul Doukhan
Abstract:
Taylor's power law (TL) or fluctuation scaling has been verified empirically for the abundances of many species, human and non-human, and in many other fields including physics, meteorology, computer science, and finance. TL asserts that the variance is directly proportional to a power of the mean, exactly for population moments and, whether or not population moments exist, approximately for sampl…
▽ More
Taylor's power law (TL) or fluctuation scaling has been verified empirically for the abundances of many species, human and non-human, and in many other fields including physics, meteorology, computer science, and finance. TL asserts that the variance is directly proportional to a power of the mean, exactly for population moments and, whether or not population moments exist, approximately for sample moments. In many papers, linear regression of log variance as a function of log mean is used to estimate TL's parameters. We provide some statistical guarantees with large-sample asymptotics for this kind of inference under general conditions, and we derive confidence intervals for the parameters. In many ecological applications, the means and variances are estimated over time or across space from arrays of abundance data collected at different locations and time points. When the ratio between the time-series length and the number of spatial points converges to a constant as both become large, the usual normalized statistics are asymptotically biased. We provide a bias correction to get correct confidence intervals. TL, widely studied in multiple sciences, is a source of challenging new statistical problems in a nonstationary spatiotemporal framework. We illustrate our results with both simulated and real data sets.
△ Less
Submitted 27 August, 2024;
originally announced August 2024.
-
Gaps Between Consecutive Primes and the Exponential Distribution
Authors:
Joel E. Cohen
Abstract:
Based on the primes less than $4 \times 10^{18}$, Oliveira e Silva et al. (2014) conjectured an asymptotic formula for the sum of the $k$th power of the gaps between consecutive primes less than a large number $x$. We show that the conjecture of Oliveira e Silva holds if and only if the $k$th moment of the first $n$ gaps is asymptotic to the $k$th moment of an exponential distribution with mean…
▽ More
Based on the primes less than $4 \times 10^{18}$, Oliveira e Silva et al. (2014) conjectured an asymptotic formula for the sum of the $k$th power of the gaps between consecutive primes less than a large number $x$. We show that the conjecture of Oliveira e Silva holds if and only if the $k$th moment of the first $n$ gaps is asymptotic to the $k$th moment of an exponential distribution with mean $\log n$, though the distribution of gaps is not exponential. Asymptotically exponential moments imply that the gaps asymptotically obey Taylor's law of fluctuation scaling: variance of the first $n$ gaps $\sim$ (mean of the first $n$ gaps)$^2$. If the distribution of the first $n$ gaps is asymptotically exponential with mean $\log n$, then the expectation of the largest of the first $n$ gaps is asymptotic to $(\log n)^2$. The largest of the first $n$ gaps is asymptotic to $(\log n)^2$ if and only if the Cramér-Shanks conjecture holds. Numerical counts of gaps and the maximal gap $G_n$ among the first $n$ gaps test these results. While most values of $G_n$ are better approximated by $(\log n)^2$ than by other models, seven values of $n$ with $G_{n} >2 e^{-γ}(\log n)^2$ suggest that $\limsup_{n \to\infty} G_n/[2e^{-γ}(\log n)^2]$ may exceed 1.
△ Less
Submitted 12 June, 2024; v1 submitted 24 May, 2024;
originally announced May 2024.
-
Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
Authors:
Jeremy E. Cohen,
Valentin Leplat
Abstract:
Regularized nonnegative low-rank approximations, such as sparse Nonnegative Matrix Factorization or sparse Nonnegative Tucker Decomposition, form an important branch of dimensionality reduction models known for their enhanced interpretability. From a practical perspective, however, selecting appropriate regularizers and regularization coefficients, as well as designing efficient algorithms, remain…
▽ More
Regularized nonnegative low-rank approximations, such as sparse Nonnegative Matrix Factorization or sparse Nonnegative Tucker Decomposition, form an important branch of dimensionality reduction models known for their enhanced interpretability. From a practical perspective, however, selecting appropriate regularizers and regularization coefficients, as well as designing efficient algorithms, remains challenging due to the multifactor nature of these models and the limited theoretical guidance available. This paper addresses these challenges by studying a more general model, the Homogeneous Regularized Scale-Invariant model. We prove that the scale-invariance inherent to low-rank approximation models induces an implicit regularization effect that balances solutions. This insight provides a deeper understanding of the role of regularization functions in low-rank approximation models, informs the selection of regularization hyperparameters, and enables the design of balancing strategies to accelerate the empirical convergence of optimization algorithms.
Additionally, we propose a generic Majorization-Minimization (MM) algorithm capable of handling $\ell_p^p$-regularized nonnegative low-rank approximations with non-Euclidean loss functions, with convergence guarantees. Our contributions are demonstrated on sparse Nonnegative Matrix Factorization, ridge-regularized Nonnegative Canonical Polyadic Decomposition, and sparse Nonnegative Tucker Decomposition.
△ Less
Submitted 30 January, 2025; v1 submitted 27 March, 2024;
originally announced March 2024.
-
Generalizations of Bertrand's Postulate to Sums of Any Number of Primes
Authors:
Joel E. Cohen
Abstract:
In 1845, Bertrand conjectured that twice any prime strictly exceeds the next prime. Tchebichef proved Bertrand's postulate in 1850. In 1934, Ishikawa proved a stronger result: the sum of any two consecutive primes strictly exceeds the next prime, except for the only equality $2+3=5$. This observation is a special case of a more general result, perhaps not previously noticed: if $p_n$ denotes the…
▽ More
In 1845, Bertrand conjectured that twice any prime strictly exceeds the next prime. Tchebichef proved Bertrand's postulate in 1850. In 1934, Ishikawa proved a stronger result: the sum of any two consecutive primes strictly exceeds the next prime, except for the only equality $2+3=5$. This observation is a special case of a more general result, perhaps not previously noticed: if $p_n$ denotes the $n$th prime, $n=1, 2, \ldots$, with $p_1=2, p_2=3, \ldots$, and if $c_1, \ldots, c_g$ are nonnegative integers (not necessarily distinct), and $d_1, \ldots, d_h$ are positive integers (not necessarily distinct), and $g>h\ge 1$, then there exists a positive integer $N$ such that $p_{n-c_1}+p_{n-c_2}+\cdots +p_{n-c_g}>p_{n+d_1}+\cdots +p_{n+d_h}$ for all $n\ge N$. We prove this result using only the prime number theorem. For any instance of this result, we sketch a way to find the least possible $N$. We give some numerical results and unanswered questions.
△ Less
Submitted 5 May, 2023;
originally announced May 2023.
-
Taylor's Law for some infinitely divisible probabbility distributions from population models
Authors:
Joel E. Cohen,
Thierry E Huillet
Abstract:
In a family of random variables, Taylor's law or Taylor's power law offluctuation scaling is a variance function that gives the variance $σ^{2}>0$ of a random variable (rv) $X$ with expectation $μ>0$ as a powerof $μ$: $σ^{2}=Aμ^{b}$ for finite real $A>0,\ b$ that are thesame for all rvs in the family. Equivalently, TL holds when $\log σ^{2}=a+b\log μ,\ a=\log A$, for all rvs in some set. Here we a…
▽ More
In a family of random variables, Taylor's law or Taylor's power law offluctuation scaling is a variance function that gives the variance $σ^{2}>0$ of a random variable (rv) $X$ with expectation $μ>0$ as a powerof $μ$: $σ^{2}=Aμ^{b}$ for finite real $A>0,\ b$ that are thesame for all rvs in the family. Equivalently, TL holds when $\log σ^{2}=a+b\log μ,\ a=\log A$, for all rvs in some set. Here we analyze thepossible values of the TL exponent $b$ in five families of infinitelydivisible two-parameter distributions and show how the values of $b$ dependon the parameters of these distributions. The five families areTweedie-Bar-Lev-Enis, negative binomial, compound Poisson-geometric,compound geometric-Poisson (or Pólya-Aeppli), and gamma distributions.These families arise frequently in empirical data and population models, and they are limit laws of Markov processes that we exhibit in each case.
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
Nonnegative Tucker Decomposition with Beta-divergence for Music Structure Analysis of Audio Signals
Authors:
Axel Marmoret,
Florian Voorwinden,
Valentin Leplat,
Jérémy E. Cohen,
Frédéric Bimbot
Abstract:
Nonnegative Tucker decomposition (NTD), a tensor decomposition model, has received increased interest in the recent years because of its ability to blindly extract meaningful patterns, in particular in Music Information Retrieval. Nevertheless, existing algorithms to compute NTD are mostly designed for the Euclidean loss. This work proposes a multiplicative updates algorithm to compute NTD with th…
▽ More
Nonnegative Tucker decomposition (NTD), a tensor decomposition model, has received increased interest in the recent years because of its ability to blindly extract meaningful patterns, in particular in Music Information Retrieval. Nevertheless, existing algorithms to compute NTD are mostly designed for the Euclidean loss. This work proposes a multiplicative updates algorithm to compute NTD with the beta-divergence loss, often considered a better loss for audio processing. We notably show how to implement efficiently the multiplicative rules using tensor algebra. Finally, we show on a music structure analysis task that unsupervised NTD fitted with beta-divergence loss outperforms earlier results obtained with the Euclidean loss.
△ Less
Submitted 2 August, 2022; v1 submitted 27 October, 2021;
originally announced October 2021.
-
An AO-ADMM approach to constraining PARAFAC2 on all modes
Authors:
Marie Roald,
Carla Schenker,
Vince D. Calhoun,
Tülay Adalı,
Rasmus Bro,
Jeremy E. Cohen,
Evrim Acar
Abstract:
Analyzing multi-way measurements with variations across one mode of the dataset is a challenge in various fields including data mining, neuroscience and chemometrics. For example, measurements may evolve over time or have unaligned time profiles. The PARAFAC2 model has been successfully used to analyze such data by allowing the underlying factor matrices in one mode (i.e., the evolving mode) to ch…
▽ More
Analyzing multi-way measurements with variations across one mode of the dataset is a challenge in various fields including data mining, neuroscience and chemometrics. For example, measurements may evolve over time or have unaligned time profiles. The PARAFAC2 model has been successfully used to analyze such data by allowing the underlying factor matrices in one mode (i.e., the evolving mode) to change across slices. The traditional approach to fit a PARAFAC2 model is to use an alternating least squares-based algorithm, which handles the constant cross-product constraint of the PARAFAC2 model by implicitly estimating the evolving factor matrices. This approach makes imposing regularization on these factor matrices challenging. There is currently no algorithm to flexibly impose such regularization with general penalty functions and hard constraints. In order to address this challenge and to avoid the implicit estimation, in this paper, we propose an algorithm for fitting PARAFAC2 based on alternating optimization with the alternating direction method of multipliers (AO-ADMM). With numerical experiments on simulated data, we show that the proposed PARAFAC2 AO-ADMM approach allows for flexible constraints, recovers the underlying patterns accurately, and is computationally efficient compared to the state-of-the-art. We also apply our model to two real-world datasets from neuroscience and chemometrics, and show that constraining the evolving mode improves the interpretability of the extracted patterns.
△ Less
Submitted 8 July, 2022; v1 submitted 4 October, 2021;
originally announced October 2021.
-
PARAFAC2 AO-ADMM: Constraints in all modes
Authors:
Marie Roald,
Carla Schenker,
Jeremy E. Cohen,
Evrim Acar
Abstract:
The PARAFAC2 model provides a flexible alternative to the popular CANDECOMP/PARAFAC (CP) model for tensor decompositions. Unlike CP, PARAFAC2 allows factor matrices in one mode (i.e., evolving mode) to change across tensor slices, which has proven useful for applications in different domains such as chemometrics, and neuroscience. However, the evolving mode of the PARAFAC2 model is traditionally m…
▽ More
The PARAFAC2 model provides a flexible alternative to the popular CANDECOMP/PARAFAC (CP) model for tensor decompositions. Unlike CP, PARAFAC2 allows factor matrices in one mode (i.e., evolving mode) to change across tensor slices, which has proven useful for applications in different domains such as chemometrics, and neuroscience. However, the evolving mode of the PARAFAC2 model is traditionally modelled implicitly, which makes it challenging to regularise it. Currently, the only way to apply regularisation on that mode is with a flexible coupling approach, which finds the solution through regularised least-squares subproblems. In this work, we instead propose an alternating direction method of multipliers (ADMM)-based algorithm for fitting PARAFAC2 and widen the possible regularisation penalties to any proximable function. Our numerical experiments demonstrate that the proposed ADMM-based approach for PARAFAC2 can accurately recover the underlying components from simulated data while being both computationally efficient and flexible in terms of imposing constraints.
△ Less
Submitted 3 February, 2021;
originally announced February 2021.
-
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings
Authors:
Carla Schenker,
Jeremy E. Cohen,
Evrim Acar
Abstract:
Coupled matrix and tensor factorizations (CMTF) are frequently used to jointly analyze data from multiple sources, also called data fusion. However, different characteristics of datasets stemming from multiple sources pose many challenges in data fusion and require to employ various regularizations, constraints, loss functions and different types of coupling structures between datasets. In this pa…
▽ More
Coupled matrix and tensor factorizations (CMTF) are frequently used to jointly analyze data from multiple sources, also called data fusion. However, different characteristics of datasets stemming from multiple sources pose many challenges in data fusion and require to employ various regularizations, constraints, loss functions and different types of coupling structures between datasets. In this paper, we propose a flexible algorithmic framework for coupled matrix and tensor factorizations which utilizes Alternating Optimization (AO) and the Alternating Direction Method of Multipliers (ADMM). The framework facilitates the use of a variety of constraints, loss functions and couplings with linear transformations in a seamless way. Numerical experiments on simulated and real datasets demonstrate that the proposed approach is accurate, and computationally efficient with comparable or better performance than available CMTF methods for Frobenius norm loss, while being more flexible. Using Kullback-Leibler divergence on count data, we demonstrate that the algorithm yields accurate results also for other loss functions.
△ Less
Submitted 19 July, 2020;
originally announced July 2020.
-
Nonconcavity of the Spectral Radius in Levinger's Theorem
Authors:
Lee Altenberg,
Joel E. Cohen
Abstract:
Let ${\bf A} \in R^{n \times n}$ be a nonnegative irreducible square matrix and let $r({\bf A})$ be its spectral radius and Perron-Frobenius eigenvalue. Levinger asserted and several have proven that $r(t):=r((1{-}t) {\bf A} + t {\bf A}^\top)$ increases over $t \in [0,1/2]$ and decreases over $t \in [1/2,1]$. It has further been stated that $r(t)$ is concave over $t \in (0,1)$. Here we show that t…
▽ More
Let ${\bf A} \in R^{n \times n}$ be a nonnegative irreducible square matrix and let $r({\bf A})$ be its spectral radius and Perron-Frobenius eigenvalue. Levinger asserted and several have proven that $r(t):=r((1{-}t) {\bf A} + t {\bf A}^\top)$ increases over $t \in [0,1/2]$ and decreases over $t \in [1/2,1]$. It has further been stated that $r(t)$ is concave over $t \in (0,1)$. Here we show that the latter claim is false in general through a number of counterexamples, but prove it is true for ${\bf A} \in R^{2\times 2}$, weighted shift matrices (but not cyclic weighted shift matrices), tridiagonal Toeplitz matrices, and the 3-parameter Toeplitz matrices from Fiedler, but not Toeplitz matrices in general. A general characterization of the range of $t$, or the class of matrices, for which the spectral radius is concave in Levinger's homotopy remains an open problem.
△ Less
Submitted 19 August, 2020; v1 submitted 6 July, 2020;
originally announced July 2020.
-
Sparse Separable Nonnegative Matrix Factorization
Authors:
Nicolas Nadisic,
Arnaud Vandaele,
Jeremy E. Cohen,
Nicolas Gillis
Abstract:
We propose a new variant of nonnegative matrix factorization (NMF), combining separability and sparsity assumptions. Separability requires that the columns of the first NMF factor are equal to columns of the input matrix, while sparsity requires that the columns of the second NMF factor are sparse. We call this variant sparse separable NMF (SSNMF), which we prove to be NP-complete, as opposed to s…
▽ More
We propose a new variant of nonnegative matrix factorization (NMF), combining separability and sparsity assumptions. Separability requires that the columns of the first NMF factor are equal to columns of the input matrix, while sparsity requires that the columns of the second NMF factor are sparse. We call this variant sparse separable NMF (SSNMF), which we prove to be NP-complete, as opposed to separable NMF which can be solved in polynomial time. The main motivation to consider this new model is to handle underdetermined blind source separation problems, such as multispectral image unmixing. We introduce an algorithm to solve SSNMF, based on the successive nonnegative projection algorithm (SNPA, an effective algorithm for separable NMF), and an exact sparse nonnegative least squares solver. We prove that, in noiseless settings and under mild assumptions, our algorithm recovers the true underlying sources. This is illustrated by experiments on synthetic data sets and the unmixing of a multispectral image.
△ Less
Submitted 12 June, 2020;
originally announced June 2020.
-
Computing the proximal operator of the $\ell_1$ induced matrix norm
Authors:
Jeremy E. Cohen
Abstract:
In this short article, for any matrix $X\in\mathbb{R}^{n\times m}$ the proximity operator of two induced norms $ \|X\|_1 $ and $ \|X\|_{\infty}$ are derived. Although no close form expression is obtained, an algorithmic procedure is described which costs roughly $\mathcal{O}(nm)$. This algorithm relies on a bisection on a real parameter derived from the Karush-Kuhn-Tucker conditions, following the…
▽ More
In this short article, for any matrix $X\in\mathbb{R}^{n\times m}$ the proximity operator of two induced norms $ \|X\|_1 $ and $ \|X\|_{\infty}$ are derived. Although no close form expression is obtained, an algorithmic procedure is described which costs roughly $\mathcal{O}(nm)$. This algorithm relies on a bisection on a real parameter derived from the Karush-Kuhn-Tucker conditions, following the proof idea of the proximal operator of the $ \max $ function found in Parikh(2014).
△ Less
Submitted 15 January, 2021; v1 submitted 14 May, 2020;
originally announced May 2020.
-
Accelerating Block Coordinate Descent for Nonnegative Tensor Factorization
Authors:
Andersen Man Shun Ang,
Jeremy E. Cohen,
Nicolas Gillis,
Le Thi Khanh Hien
Abstract:
This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for…
▽ More
This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for dense NTF, in particular for challenging computational scenarios, while requiring a negligible additional computational budget.
△ Less
Submitted 20 November, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Spectral Unmixing with Multiple Dictionaries
Authors:
Jeremy E. Cohen,
Nicolas Gillis
Abstract:
Spectral unmixing aims at recovering the spectral signatures of materials, called endmembers, mixed in a hyperspectral or multispectral image, along with their abundances. A typical assumption is that the image contains one pure pixel per endmember, in which case spectral unmixing reduces to identifying these pixels. Many fully automated methods have been proposed in recent years, but little work…
▽ More
Spectral unmixing aims at recovering the spectral signatures of materials, called endmembers, mixed in a hyperspectral or multispectral image, along with their abundances. A typical assumption is that the image contains one pure pixel per endmember, in which case spectral unmixing reduces to identifying these pixels. Many fully automated methods have been proposed in recent years, but little work has been done to allow users to select areas where pure pixels are present manually or using a segmentation algorithm. Additionally, in a non-blind approach, several spectral libraries may be available rather than a single one, with a fixed number (or an upper or lower bound) of endmembers to chose from each. In this paper, we propose a multiple-dictionary constrained low-rank matrix approximation model that address these two problems. We propose an algorithm to compute this model, dubbed M2PALS, and its performance is discussed on both synthetic and real hyperspectral images.
△ Less
Submitted 8 November, 2017;
originally announced November 2017.
-
About Notations in Multiway Array Processing
Authors:
Jeremy E. Cohen
Abstract:
This paper gives an overview of notations used in multiway array processing. We redefine the vectorization and matricization operators to comply with some properties of the Kronecker product. The tensor product and Kronecker product are also represented with two different symbols, and it is shown how these notations lead to clearer expressions for multiway array operations. Finally, the paper reca…
▽ More
This paper gives an overview of notations used in multiway array processing. We redefine the vectorization and matricization operators to comply with some properties of the Kronecker product. The tensor product and Kronecker product are also represented with two different symbols, and it is shown how these notations lead to clearer expressions for multiway array operations. Finally, the paper recalls the useful yet widely unknown properties of the array normal law with suggested notations.
△ Less
Submitted 3 February, 2016; v1 submitted 4 November, 2015;
originally announced November 2015.
