-
From Two Sample Testing to Singular Gaussian Discrimination
Authors:
Leonardo V. Santoro,
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
We establish that testing for the equality of two probability measures on a general separable and compact metric space is equivalent to testing for the singularity between two corresponding Gaussian measures on a suitable Reproducing Kernel Hilbert Space. The corresponding Gaussians are defined via the notion of kernel mean and covariance embedding of a probability measure. Discerning two singular…
▽ More
We establish that testing for the equality of two probability measures on a general separable and compact metric space is equivalent to testing for the singularity between two corresponding Gaussian measures on a suitable Reproducing Kernel Hilbert Space. The corresponding Gaussians are defined via the notion of kernel mean and covariance embedding of a probability measure. Discerning two singular Gaussians is fundamentally simpler from an information-theoretic perspective than non-parametric two-sample testing, particularly in high-dimensional settings. Our proof leverages the Feldman-Hajek criterion for singularity/equivalence of Gaussians on Hilbert spaces, and shows that discrepancies between distributions are heavily magnified through their corresponding Gaussian embeddings: at a population level, distinct probability measures lead to essentially separated Gaussian embeddings. This appears to be a new instance of the blessing of dimensionality that can be harnessed for the design of efficient inference tools in great generality.
△ Less
Submitted 7 May, 2025;
originally announced May 2025.
-
Extreme Points of Spectrahedra
Authors:
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
We consider the problem of characterizing extreme points of the convex set of positive linear operators on a possibly infinite-dimensional Hilbert space under linear constraints. We show that even perturbations of points in such sets admit what resembles a Douglas factorization. Using this result, we prove that an operator is extreme iff a corresponding set of linear operators is dense in the spac…
▽ More
We consider the problem of characterizing extreme points of the convex set of positive linear operators on a possibly infinite-dimensional Hilbert space under linear constraints. We show that even perturbations of points in such sets admit what resembles a Douglas factorization. Using this result, we prove that an operator is extreme iff a corresponding set of linear operators is dense in the space of trace-class self-adjoint operators with range contained in the closure of the range of that operator. If the number of constraints is finite, we show that the extreme point must be of low-rank relative to the number of constraints and derive a purely rank-based characterization of the extreme points.
In the finite-dimensional setting, our results lead to a remarkably simple characterization of the elliptope, that is, the set of correlation matrices, in terms of the Hadamard product which allows us to characterize the set of matrices which constitute the equality case of the Hadamard rank inequality when the involved matrices are equal and positive semi-definite. We illustrate the importance of our results using examples from statistics and quantum mechanics.
△ Less
Submitted 29 December, 2024; v1 submitted 18 October, 2024;
originally announced October 2024.
-
Computerized Tomography and Reproducing Kernels
Authors:
Ho Yun,
Victor M. Panaretos
Abstract:
The X-ray transform is one of the most fundamental integral operators in image processing and reconstruction. In this article, we revisit the formalism of the X-ray transform by considering it as an operator between Reproducing Kernel Hilbert Spaces (RKHS). Within this framework, the X-ray transform can be viewed as a natural analogue of Euclidean projection. The RKHS framework considerably simpli…
▽ More
The X-ray transform is one of the most fundamental integral operators in image processing and reconstruction. In this article, we revisit the formalism of the X-ray transform by considering it as an operator between Reproducing Kernel Hilbert Spaces (RKHS). Within this framework, the X-ray transform can be viewed as a natural analogue of Euclidean projection. The RKHS framework considerably simplifies projection image interpolation, and leads to an analogue of the celebrated representer theorem for the problem of tomographic reconstruction. It leads to methodology that is dimension-free and stands apart from conventional filtered back-projection techniques, as it does not hinge on the Fourier transform. It also allows us to establish sharp stability results at a genuinely functional level (i.e. without recourse to discretization), but in the realistic setting where the data are discrete and noisy. The RKHS framework is versatile, accommodating any reproducing kernel on a unit ball, affording a high level of generality. When the kernel is chosen to be rotation-invariant, explicit spectral representations can be obtained, elucidating the regularity structure of the associated Hilbert spaces. Moreover, the reconstruction problem can be solved at the same computational cost as filtered back-projection.
△ Less
Submitted 24 June, 2024; v1 submitted 13 November, 2023;
originally announced November 2023.
-
Statistical Inference for Bures-Wasserstein Flows
Authors:
Leonardo V. Santoro,
Victor M. Panaretos
Abstract:
We develop a statistical framework for conducting inference on collections of time-varying covariance operators (covariance flows) over a general, possibly infinite dimensional, Hilbert space. We model the intrinsically non-linear structure of covariances by means of the Bures-Wasserstein metric geometry. We make use of the Riemmanian-like structure induced by this metric to define a notion of mea…
▽ More
We develop a statistical framework for conducting inference on collections of time-varying covariance operators (covariance flows) over a general, possibly infinite dimensional, Hilbert space. We model the intrinsically non-linear structure of covariances by means of the Bures-Wasserstein metric geometry. We make use of the Riemmanian-like structure induced by this metric to define a notion of mean and covariance of a random flow, and develop an associated Karhunen-Loève expansion. We then treat the problem of estimation and construction of functional principal components from a finite collection of covariance flows, observed fully or irregularly.
Our theoretical results are motivated by modern problems in functional data analysis, where one observes operator-valued random processes -- for instance when analysing dynamic functional connectivity and fMRI data, or when analysing multiple functional time series in the frequency domain. Nevertheless, our framework is also novel in the finite-dimensions (matrix case), and we demonstrate what simplifications can be afforded then. We illustrate our methodology by means of simulations and data analyses.
△ Less
Submitted 24 June, 2024; v1 submitted 20 October, 2023;
originally announced October 2023.
-
The Positive-Definite Completion Problem
Authors:
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
We study the positive-definite completion problem for kernels on a variety of domains and prove results concerning the existence, uniqueness, and characterization of solutions. In particular, we study a special solution called the canonical completion which is the reproducing kernel analogue of the determinant-maximizing completion known to exist for matrices. We establish several results concerni…
▽ More
We study the positive-definite completion problem for kernels on a variety of domains and prove results concerning the existence, uniqueness, and characterization of solutions. In particular, we study a special solution called the canonical completion which is the reproducing kernel analogue of the determinant-maximizing completion known to exist for matrices. We establish several results concerning its existence and uniqueness, which include algebraic and variational characterizations. Notably, we prove the existence of a canonical completion for domains which are equivalent to the band containing the diagonal. This corresponds to the existence of a canonical extension in the context of the classical extension problem of positive-definite functions, which can be understood as the solution to an abstract Cauchy problem in a certain reproducing kernel Hilbert space.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
The Functional Graphical Lasso
Authors:
Kartik G. Waghmare,
Tomas Masak,
Victor M. Panaretos
Abstract:
We consider the problem of recovering conditional independence relationships between $p$ jointly distributed Hilbertian random elements given $n$ realizations thereof. We operate in the sparse high-dimensional regime, where $n \ll p$ and no element is related to more than $d \ll p$ other elements. In this context, we propose an infinite-dimensional generalization of the graphical lasso. We prove m…
▽ More
We consider the problem of recovering conditional independence relationships between $p$ jointly distributed Hilbertian random elements given $n$ realizations thereof. We operate in the sparse high-dimensional regime, where $n \ll p$ and no element is related to more than $d \ll p$ other elements. In this context, we propose an infinite-dimensional generalization of the graphical lasso. We prove model selection consistency under natural assumptions and extend many classical results to infinite dimensions. In particular, we do not require finite truncation or additional structural restrictions. The plug-in nature of our method makes it applicable to any observational regime, whether sparse or dense, and indifferent to serial dependence. Importantly, our method can be understood as naturally arising from a coherent maximum likelihood philosophy.
△ Less
Submitted 23 June, 2023; v1 submitted 4 June, 2023;
originally announced June 2023.
-
Transportation of Measure Regression in Higher Dimensions
Authors:
Laya Ghodrati,
Victor M. Panaretos
Abstract:
We present an optimal transport framework for performing regression when both the covariate and the response are probability distributions on a compact Euclidean subset $Ω\subset\mathbb{R}^d$, where $d>1$. Extending beyond compactly supported distributions, this method also applies when both the predictor and responses are Gaussian distributions on $\mathbb{R}^d$. Our approach generalizes an exist…
▽ More
We present an optimal transport framework for performing regression when both the covariate and the response are probability distributions on a compact Euclidean subset $Ω\subset\mathbb{R}^d$, where $d>1$. Extending beyond compactly supported distributions, this method also applies when both the predictor and responses are Gaussian distributions on $\mathbb{R}^d$. Our approach generalizes an existing transportation-based regression model to higher dimensions. This model postulates that the conditional Fréchet mean of the response distribution is linked to the covariate distribution via an optimal transport map. We establish an upper bound for the rate of convergence of a plug-in estimator.
We propose an iterative algorithm for computing the estimator, which is based on DC (Difference of Convex Functions) Programming. In the Gaussian case, the estimator achieves a parametric rate of convergence, and the computation of the estimator simplifies to a finite-dimensional optimization over positive definite matrices, allowing for an efficient solution. The performance of the estimator is demonstrated in a simulation study.
△ Less
Submitted 4 March, 2024; v1 submitted 27 May, 2023;
originally announced May 2023.
-
Large Sample Theory for Bures-Wasserstein Barycentres
Authors:
Leonardo V. Santoro,
Victor M. Panaretos
Abstract:
We establish a strong law of large numbers and a central limit theorem in the Bures-Wasserstein space of covariance operators -- or equivalently centred Gaussian measures -- over a general separable Hilbert space. Specifically, we show that empirical barycentre sequences indexed by sample size are almost certainly relatively compact, with accumulation points comprising population barycentres. We g…
▽ More
We establish a strong law of large numbers and a central limit theorem in the Bures-Wasserstein space of covariance operators -- or equivalently centred Gaussian measures -- over a general separable Hilbert space. Specifically, we show that empirical barycentre sequences indexed by sample size are almost certainly relatively compact, with accumulation points comprising population barycentres. We give a sufficient regularity condition for the limit to be unique. When the limit is unique, we also establish a central limit theorem under a refined pair of moment and regularity conditions.
Finally, we prove strong operator convergence of the empirical optimal transport maps to their population counterparts. Though our results naturally extend finite-dimensional counterparts, including associated regularity conditions, our techniques are distinctly different owing to the functional nature of the problem in the general setting. A key element is the characterisation of compact sets in the Bures-Wasserstein topology that reflects an ordered Heine-Borel property of the Bures-Wasserstein space.
△ Less
Submitted 4 November, 2024; v1 submitted 24 May, 2023;
originally announced May 2023.
-
A Karhunen-Loève Theorem for Random Flows in Hilbert spaces
Authors:
Leonardo V. Santoro,
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
We develop a generalisation of Mercer's theorem to operator-valued kernels in infinite dimensional Hilbert spaces. We then apply our result to deduce a Karhunen-Loève theorem, valid for mean-square continuous Hilbertian functional data, i.e. flows in Hilbert spaces. That is, we prove a series expansion with uncorrelated coefficients for square-integrable random flows in a Hilbert space, that holds…
▽ More
We develop a generalisation of Mercer's theorem to operator-valued kernels in infinite dimensional Hilbert spaces. We then apply our result to deduce a Karhunen-Loève theorem, valid for mean-square continuous Hilbertian functional data, i.e. flows in Hilbert spaces. That is, we prove a series expansion with uncorrelated coefficients for square-integrable random flows in a Hilbert space, that holds uniformly over time.
△ Less
Submitted 2 March, 2023; v1 submitted 1 March, 2023;
originally announced March 2023.
-
Continuously Indexed Graphical Models
Authors:
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
Let $X = \{X_{u}\}_{u \in U}$ be a real-valued Gaussian process indexed by a set $U$. It can be thought of as an undirected graphical model with every random variable $X_{u}$ serving as a vertex. We characterize this graph in terms of the covariance of $X$ through its reproducing kernel property. Unlike other characterizations in the literature, our characterization does not restrict the index set…
▽ More
Let $X = \{X_{u}\}_{u \in U}$ be a real-valued Gaussian process indexed by a set $U$. It can be thought of as an undirected graphical model with every random variable $X_{u}$ serving as a vertex. We characterize this graph in terms of the covariance of $X$ through its reproducing kernel property. Unlike other characterizations in the literature, our characterization does not restrict the index set $U$ to be finite or countable, and hence can be used to model the intrinsic dependence structure of stochastic processes in continuous time/space. Consequently, this characterization is not in terms of the zero entries of an inverse covariance. This poses novel challenges for the problem of recovery of the dependence structure from a sample of independent realizations of $X$, also known as structure estimation. We propose a methodology that circumvents these issues, by targeting the recovery of the underlying graph up to a finite resolution, which can be arbitrarily fine and is limited only by the available sample size. The recovery is shown to be consistent so long as the graph is sufficiently regular in an appropriate sense. We derive corresponding convergence rates and finite sample guarantees. Our methodology is illustrated by means of a simulation study and two data analyses.
△ Less
Submitted 12 December, 2023; v1 submitted 5 February, 2023;
originally announced February 2023.
-
Minimax Rate for Optimal Transport Regression Between Distributions
Authors:
Laya Ghodrati,
Victor M. Panaretos
Abstract:
Distribution-on-distribution regression considers the problem of formulating and estimating a regression relationship where both covariate and response are probability distributions. The optimal transport distributional regression model postulates that the conditional Fréchet mean of the response distribution is linked to the covariate distribution via an optimal transport map. We establish the mi…
▽ More
Distribution-on-distribution regression considers the problem of formulating and estimating a regression relationship where both covariate and response are probability distributions. The optimal transport distributional regression model postulates that the conditional Fréchet mean of the response distribution is linked to the covariate distribution via an optimal transport map. We establish the minimax rate of estimation of such a regression function, by deriving a lower-bound that matches the convergence rate attained by the Fréchet least squares estimator.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
On the rate of convergence for the autocorrelation operator in functional autoregression
Authors:
Alessia Caponera,
Victor M. Panaretos
Abstract:
We consider the problem of estimating the autocorrelation operator of an autoregressive Hilbertian process. By means of a Tikhonov approach, we establish a general result that yields the convergence rate of the estimated autocorrelation operator as a function of the rate of convergence of the estimated lag zero and lag one autocovariance operators. The result is general in that it can accommodate…
▽ More
We consider the problem of estimating the autocorrelation operator of an autoregressive Hilbertian process. By means of a Tikhonov approach, we establish a general result that yields the convergence rate of the estimated autocorrelation operator as a function of the rate of convergence of the estimated lag zero and lag one autocovariance operators. The result is general in that it can accommodate any consistent estimators of the lagged autocovariances. Consequently it can be applied to processes under any mode of observation: complete, discrete, sparse, and/or with measurement errors. An appealing feature is that the result does not require delicate spectral decay assumptions on the autocovariances but instead rests on natural source conditions. The result is illustrated by application to important special cases.
△ Less
Submitted 8 June, 2022; v1 submitted 18 February, 2022;
originally announced February 2022.
-
Functional Estimation of Anisotropic Covariance and Autocovariance Operators on the Sphere
Authors:
Alessia Caponera,
Julien Fageot,
Matthieu Simeoni,
Victor M. Panaretos
Abstract:
We propose nonparametric estimators for the second-order central moments of possibly anisotropic spherical random fields, within a functional data analysis context. We consider a measurement framework where each random field among an identically distributed collection of spherical random fields is sampled at a few random directions, possibly subject to measurement error. The collection of random f…
▽ More
We propose nonparametric estimators for the second-order central moments of possibly anisotropic spherical random fields, within a functional data analysis context. We consider a measurement framework where each random field among an identically distributed collection of spherical random fields is sampled at a few random directions, possibly subject to measurement error. The collection of random fields could be i.i.d. or serially dependent. Though similar setups have already been explored for random functions defined on the unit interval, the nonparametric estimators proposed in the literature often rely on local polynomials, which do not readily extend to the (product) spherical setting. We therefore formulate our estimation procedure as a variational problem involving a generalized Tikhonov regularization term. The latter favours smooth covariance/autocovariance functions, where the smoothness is specified by means of suitable Sobolev-like pseudo-differential operators. Using the machinery of reproducing kernel Hilbert spaces, we establish representer theorems that fully characterize the form of our estimators. We determine their uniform rates of convergence as the number of random fields diverges, both for the dense (increasing number of spatial samples) and sparse (bounded number of spatial samples) regimes. We moreover demonstrate the computational feasibility and practical merits of our estimation procedure in a simulation setting, assuming a fixed number of samples per random field. Our numerical estimation procedure leverages the sparsity and second-order Kronecker structure of our setup to reduce the computational and memory requirements by approximately three orders of magnitude compared to a naive implementation would require.
△ Less
Submitted 25 June, 2022; v1 submitted 23 December, 2021;
originally announced December 2021.
-
Detecting Whether a Stochastic Process is Finitely Expressed in a Basis
Authors:
Neda Mohammadi,
Victor M. Panaretos
Abstract:
Is it possible to detect if the sample paths of a stochastic process almost surely admit a finite expansion with respect to some/any basis? The determination is to be made on the basis of a finite collection of discretely/noisily observed sample paths. We show that it is indeed possible to construct a hypothesis testing scheme that is almost surely guaranteed to make only finitely many incorrect d…
▽ More
Is it possible to detect if the sample paths of a stochastic process almost surely admit a finite expansion with respect to some/any basis? The determination is to be made on the basis of a finite collection of discretely/noisily observed sample paths. We show that it is indeed possible to construct a hypothesis testing scheme that is almost surely guaranteed to make only finitely many incorrect decisions as more data are collected. Said differently, our scheme almost certainly detects whether the process has a finite or infinite basis expansion for all sufficiently large sample sizes. Our approach relies on Cover's classical test for the irrationality of a mean, combined with tools for the non-parametric estimation of covariance operators.
△ Less
Submitted 2 November, 2021;
originally announced November 2021.
-
Nonparametric Estimation for SDE with Sparsely Sampled Paths: an FDA Perspective
Authors:
Neda Mohammadi,
Leonardo Santoro,
Victor M. Panaretos
Abstract:
We consider the problem of nonparametric estimation of the drift and diffusion coefficients of a Stochastic Differential Equation (SDE), based on $n$ independent replicates $\left\{X_i(t)\::\: t\in [0,1]\right\}_{1
\leq i \leq n}$, observed sparsely and irregularly on the unit interval, and subject to additive noise corruption. By sparse we intend to mean that the number of measurements per path…
▽ More
We consider the problem of nonparametric estimation of the drift and diffusion coefficients of a Stochastic Differential Equation (SDE), based on $n$ independent replicates $\left\{X_i(t)\::\: t\in [0,1]\right\}_{1
\leq i \leq n}$, observed sparsely and irregularly on the unit interval, and subject to additive noise corruption. By sparse we intend to mean that the number of measurements per path can be arbitrary (as small as two), and remain constant with respect to $n$. We focus on time-inhomogeneous SDE of the form $dX_t = μ(t)X_t^αdt + σ(t)X_t^βdW_t$, where $α\in \{0,1\}$ and $β\in \{0,1/2,1\}$, which includes prominent examples such as Brownian motion, Ornstein-Uhlenbeck process, geometric Brownian motion, and Brownian bridge. Our estimators are constructed by relating the local (drift/diffusion) parameters of the diffusion to their global parameters (mean/covariance, and their derivatives) by means of an apparently novel Partial Differential Equation (PDE). This allows us to use methods inspired by functional data analysis, and pool information across the sparsely measured paths. The methodology we develop is fully non-parametric and avoids any functional form specification on the time-dependency of either the drift function or the diffusion function. We establish almost sure uniform asymptotic convergence rates of the proposed estimators as the number of observed curves $n$ grows to infinity. Our rates are non-asymptotic in the number of measurements per path, explicitly reflecting how different sampling frequency might affect the speed of convergence. Our framework suggests possible further fruitful interactions between FDA and SDE methods in problems with replication.
△ Less
Submitted 24 November, 2023; v1 submitted 27 October, 2021;
originally announced October 2021.
-
The Completion of Covariance Kernels
Authors:
Kartik G. Waghmare,
Victor M. Panaretos
Abstract:
We consider the problem of positive-semidefinite continuation: extending a partially specified covariance kernel from a subdomain $Ω$ of a rectangular domain $I\times I$ to a covariance kernel on the entire domain $I\times I$. For a broad class of domains $Ω$ called \emph{serrated domains}, we are able to present a complete theory. Namely, we demonstrate that a canonical completion always exists a…
▽ More
We consider the problem of positive-semidefinite continuation: extending a partially specified covariance kernel from a subdomain $Ω$ of a rectangular domain $I\times I$ to a covariance kernel on the entire domain $I\times I$. For a broad class of domains $Ω$ called \emph{serrated domains}, we are able to present a complete theory. Namely, we demonstrate that a canonical completion always exists and can be explicitly constructed. We characterise all possible completions as suitable perturbations of the canonical completion, and determine necessary and sufficient conditions for a unique completion to exist. We interpret the canonical completion via the graphical model structure it induces on the associated Gaussian process. Furthermore, we show how the estimation of the canonical completion reduces to the solution of a system of linear statistical inverse problems in the space of Hilbert-Schmidt operators, and derive rates of convergence. We conclude by providing extensions of our theory to more general forms of domains, and by demonstrating how our results can be used to construct covariance estimators from sample path fragments of the associated stochastic process. Our results are illustrated numerically by way of a simulation study and a real example.
△ Less
Submitted 12 May, 2022; v1 submitted 15 July, 2021;
originally announced July 2021.
-
Functional Data Analysis with Rough Sample Paths?
Authors:
Neda Mohammadi,
Victor M. Panaretos
Abstract:
Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for the use of smoothing techniques that annihilate the noise. At the same time, imposing the smoothness assumption excludes a considerable range o…
▽ More
Functional data are typically modeled as sample paths of smooth stochastic processes in order to mitigate the fact that they are often observed discretely and noisily, occasionally irregularly and sparsely. The smoothness assumption is imposed to allow for the use of smoothing techniques that annihilate the noise. At the same time, imposing the smoothness assumption excludes a considerable range of stochastic processes, most notably diffusion processes. Under perfect observation of the sample paths, such processes would not need to be excluded from the realm of functional data analysis. In this paper, we introduce a careful modification of existing methods, dubbed the "reflected triangle estimator", and show that this allows for the functional data analysis of processes with nowhere differentiable sample paths, even when these are discretely and noisily observed, including under irregular and sparse designs. Our estimator matches the established rates of convergence for processes with smooth paths, and furthermore attains the same optimal rates as one would get under perfect observation. Thus, with reflected triangle estimation, the scope of applicability of much of the methodology developed for discretely/irregularly/noisily/sparsely sampled functional data is considerably extended. By way of simulation it is shown that the advantages furnished are reflected in practice, hinting at potential closer links with the field of diffusion inference.
△ Less
Submitted 22 December, 2021; v1 submitted 25 May, 2021;
originally announced May 2021.
-
CovNet: Covariance Networks for Functional Data on Multidimensional Domains
Authors:
Soham Sarkar,
Victor M. Panaretos
Abstract:
Covariance estimation is ubiquitous in functional data analysis. Yet, the case of functional observations over multidimensional domains introduces computational and statistical challenges, rendering the standard methods effectively inapplicable. To address this problem, we introduce "Covariance Networks" (CovNet) as a modeling and estimation tool. The CovNet model is "universal" - it can be used t…
▽ More
Covariance estimation is ubiquitous in functional data analysis. Yet, the case of functional observations over multidimensional domains introduces computational and statistical challenges, rendering the standard methods effectively inapplicable. To address this problem, we introduce "Covariance Networks" (CovNet) as a modeling and estimation tool. The CovNet model is "universal" - it can be used to approximate any covariance up to desired precision. Moreover, the model can be fitted efficiently to the data and its neural network architecture allows us to employ modern computational tools in the implementation. The CovNet model also admits a closed-form eigendecomposition, which can be computed efficiently, without constructing the covariance itself. This facilitates easy storage and subsequent manipulation of a covariance in the context of the CovNet. We establish consistency of the proposed estimator and derive its rate of convergence. The usefulness of the proposed method is demonstrated by means of an extensive simulation study and an application to resting state fMRI data.
△ Less
Submitted 4 November, 2021; v1 submitted 11 April, 2021;
originally announced April 2021.
-
Separable Expansions for Covariance Estimation
Authors:
Tomas Masak,
Soham Sarkar,
Victor M. Panaretos
Abstract:
The non-parametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are typically alleviated by assuming separability. However, separability is often questionable, sometimes even demonstrably inadequate. We propose a framework for the anal…
▽ More
The non-parametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are typically alleviated by assuming separability. However, separability is often questionable, sometimes even demonstrably inadequate. We propose a framework for the analysis of covariance operators of random surfaces that generalises separability, while retaining its major advantages. Our approach is based on the expansion of the covariance into a series of separable terms. The expansion is valid for any covariance over a two-dimensional domain. Leveraging the key notion of the partial inner product, we extend the power iteration method to general Hilbert spaces and show how the aforementioned expansion can be efficiently constructed in practice. Truncation of the expansion and retention of the leading terms automatically induces a non-parametric estimator of the covariance, whose parsimony is dictated by the truncation level. The resulting estimator can be calculated, stored and manipulated with little computational overhead relative to separability. Consistency and rates of convergence are derived under mild regularity assumptions, illustrating the trade-off between bias and variance regulated by the truncation level. The merits and practical performance of the proposed methodology are demonstrated in a comprehensive simulation study and on classification of EEG signals.
△ Less
Submitted 17 January, 2022; v1 submitted 23 July, 2020;
originally announced July 2020.
-
Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes
Authors:
Valentina Masarotto,
Victor M. Panaretos,
Yoav Zemel
Abstract:
Covariance operators are fundamental in functional data analysis, providing the canonical means to analyse functional variation via the celebrated Karhunen--Loève expansion. These operators may themselves be subject to variation, for instance in contexts where multiple functional populations are to be compared. Statistical techniques to analyse such variation are intimately linked with the choice…
▽ More
Covariance operators are fundamental in functional data analysis, providing the canonical means to analyse functional variation via the celebrated Karhunen--Loève expansion. These operators may themselves be subject to variation, for instance in contexts where multiple functional populations are to be compared. Statistical techniques to analyse such variation are intimately linked with the choice of metric on covariance operators, and the intrinsic infinite-dimensionality of these operators. In this paper, we describe the manifold geometry of the space of trace-class infinite-dimensional covariance operators and associated key statistical properties, under the recently proposed infinite-dimensional version of the Procrustes metric. We identify this space with that of centred Gaussian processes equipped with the Wasserstein metric of optimal transportation. The identification allows us to provide a complete description of those aspects of this manifold geometry that are important in terms of statistical inference, and establish key properties of the Fréchet mean of a random sample of covariances, as well as generative models that are canonical for such metrics and link with the problem of registration of functional data.
△ Less
Submitted 6 January, 2018;
originally announced January 2018.
-
Fréchet Means and Procrustes Analysis in Wasserstein Space
Authors:
Yoav Zemel,
Victor M. Panaretos
Abstract:
We consider two statistical problems at the intersection of functional and non-Euclidean data analysis: the determination of a Fréchet mean in the Wasserstein space of multivariate distributions; and the optimal registration of deformed random measures and point processes. We elucidate how the two problems are linked, each being in a sense dual to the other. We first study the finite sample versio…
▽ More
We consider two statistical problems at the intersection of functional and non-Euclidean data analysis: the determination of a Fréchet mean in the Wasserstein space of multivariate distributions; and the optimal registration of deformed random measures and point processes. We elucidate how the two problems are linked, each being in a sense dual to the other. We first study the finite sample version of the problem in the continuum. Exploiting the tangent bundle structure of Wasserstein space, we deduce the Fréchet mean via gradient descent. We show that this is equivalent to a Procrustes analysis for the registration maps, thus only requiring successive solutions to pairwise optimal coupling problems. We then study the population version of the problem, focussing on inference and stability: in practice, the data are i.i.d. realisations from a law on Wasserstein space, and indeed their observation is discrete, where one observes a proxy finite sample or point process. We construct regularised nonparametric estimators, and prove their consistency for the population mean, and uniform consistency for the population Procrustes registration maps.
△ Less
Submitted 17 January, 2018; v1 submitted 24 January, 2017;
originally announced January 2017.
-
Hybrid Regularisation of Functional Linear Models
Authors:
Anirvan Chakraborty,
Victor M. Panaretos
Abstract:
We consider the problem of estimating the slope function in a functional regression with a scalar response and a functional covariate. This central problem of functional data analysis is well known to be ill-posed, thus requiring a regularised estimation procedure. The two most commonly used approaches are based on spectral truncation or Tikhonov regularisation of the empirical covariance operator…
▽ More
We consider the problem of estimating the slope function in a functional regression with a scalar response and a functional covariate. This central problem of functional data analysis is well known to be ill-posed, thus requiring a regularised estimation procedure. The two most commonly used approaches are based on spectral truncation or Tikhonov regularisation of the empirical covariance operator. In principle, Tikhonov regularisation is the more canonical choice. Compared to spectral truncation, it is robust to eigenvalue ties, while it attains the optimal minimax rate of convergence in the mean squared sense, and not just in a concentration probability sense. In this paper, we show that, surprisingly, one can strictly improve upon the performance of the Tikhonov estimator in finite samples by means of a linear estimator, while retaining its stability and asymptotic properties by combining it with a form of spectral truncation. Specifically, we construct an estimator that additively decomposes the functional covariate by projecting it onto two orthogonal subspaces defined via functional PCA; it then applies Tikhonov regularisation to the one component, while leaving the other component unregularised. We prove that when the covariate is Gaussian, this hybrid estimator uniformly improves upon the MSE of the Tikhonov estimator in a non-asymptotic sense, effectively rendering it inadmissible. This domination is shown to also persist under discrete observation of the covariate function. The hybrid estimator is linear, straightforward to construct in practice, and with no computational overhead relative to the standard regularisation methods. By means of simulation, it is shown to furnish sizeable gains even for modest sample sizes.
△ Less
Submitted 4 October, 2016;
originally announced October 2016.
-
Functional Data Analysis by Matrix Completion
Authors:
Marie-Hélène Descary,
Victor M. Panaretos
Abstract:
Functional data analyses typically proceed by smoothing, followed by functional PCA. This paradigm implicitly assumes that rough variation is due to nuisance noise. Nevertheless, relevant functional features such as time-localised or short scale fluctuations may indeed be rough relative to the global scale, but still smooth at shorter scales. These may be confounded with the global smooth componen…
▽ More
Functional data analyses typically proceed by smoothing, followed by functional PCA. This paradigm implicitly assumes that rough variation is due to nuisance noise. Nevertheless, relevant functional features such as time-localised or short scale fluctuations may indeed be rough relative to the global scale, but still smooth at shorter scales. These may be confounded with the global smooth components of variation by the smoothing and PCA, potentially distorting the parsimony and interpretability of the analysis. The goal of this paper is to investigate how both smooth and rough variations can be recovered on the basis of discretely observed functional data. Assuming that a functional datum arises as the sum of two uncorrelated components, one smooth and one rough, we develop identifiability conditions for the recovery of the two corresponding covariance operators. The key insight is that they should possess complementary forms of parsimony: one smooth and finite rank (large scale), and the other banded and potentially infinite rank (small scale). Our conditions elucidate the precise interplay between rank, bandwidth, and grid resolution. Under these conditions, we show that the recovery problem is equivalent to rank-constrained matrix completion, and exploit this to construct estimators of the two covariances, without assuming knowledge of the true bandwidth or rank; we study their asymptotic behaviour, and then use them to recover the smooth and rough components of each functional datum by best linear prediction. As a result, we effectively produce separate functional PCAs for smooth and rough variation.
△ Less
Submitted 18 September, 2018; v1 submitted 3 September, 2016;
originally announced September 2016.
-
Amplitude and phase variation of point processes
Authors:
Victor M. Panaretos,
Yoav Zemel
Abstract:
We develop a canonical framework for the study of the problem of registration of multiple point processes subjected to warping, known as the problem of separation of amplitude and phase variation. The amplitude variation of a real random function $\{Y(x):x\in[0,1]\}$ corresponds to its random oscillations in the $y$-axis, typically encapsulated by its (co)variation around a mean level. In contrast…
▽ More
We develop a canonical framework for the study of the problem of registration of multiple point processes subjected to warping, known as the problem of separation of amplitude and phase variation. The amplitude variation of a real random function $\{Y(x):x\in[0,1]\}$ corresponds to its random oscillations in the $y$-axis, typically encapsulated by its (co)variation around a mean level. In contrast, its phase variation refers to fluctuations in the $x$-axis, often caused by random time changes. We formalise similar notions for a point process, and nonparametrically separate them based on realisations of i.i.d. copies $\{Π_i\}$ of the phase-varying point process. A key element in our approach is to demonstrate that when the classical phase variation assumptions of Functional Data Analysis (FDA) are applied to the point process case, they become equivalent to conditions interpretable through the prism of the theory of optimal transportation of measure. We demonstrate that these induce a natural Wasserstein geometry tailored to the warping problem, including a formal notion of bias expressing over-registration. Within this framework, we construct nonparametric estimators that tend to avoid over-registration in finite samples. We show that they consistently estimate the warp maps, consistently estimate the structural mean, and consistently register the warped point processes, even in a sparse sampling regime. We also establish convergence rates, and derive $\sqrt{n}$-consistency and a central limit theorem in the Cox process case under dense sampling, showing rate optimality of our structural mean estimator in that case.
△ Less
Submitted 29 March, 2016;
originally announced March 2016.
-
Fourier analysis of stationary time series in function space
Authors:
Victor M. Panaretos,
Shahin Tavakoli
Abstract:
We develop the basic building blocks of a frequency domain framework for drawing statistical inferences on the second-order structure of a stationary sequence of functional data. The key element in such a context is the spectral density operator, which generalises the notion of a spectral density matrix to the functional setting, and characterises the second-order dynamics of the process. Our main…
▽ More
We develop the basic building blocks of a frequency domain framework for drawing statistical inferences on the second-order structure of a stationary sequence of functional data. The key element in such a context is the spectral density operator, which generalises the notion of a spectral density matrix to the functional setting, and characterises the second-order dynamics of the process. Our main tool is the functional Discrete Fourier Transform (fDFT). We derive an asymptotic Gaussian representation of the fDFT, thus allowing the transformation of the original collection of dependent random functions into a collection of approximately independent complex-valued Gaussian random functions. Our results are then employed in order to construct estimators of the spectral density operator based on smoothed versions of the periodogram kernel, the functional generalisation of the periodogram matrix. The consistency and asymptotic law of these estimators are studied in detail. As immediate consequences, we obtain central limit theorems for the mean and the long-run covariance operator of a stationary functional time series. Our results do not depend on structural modelling assumptions, but only functional versions of classical cumulant mixing conditions, and are shown to be stable under discrete observation of the individual curves.
△ Less
Submitted 9 May, 2013;
originally announced May 2013.
-
On random tomography with unobservable projection angles
Authors:
Victor M. Panaretos
Abstract:
We formulate and investigate a statistical inverse problem of a random tomographic nature, where a probability density function on $\mathbb{R}^3$ is to be recovered from observation of finitely many of its two-dimensional projections in random and unobservable directions. Such a problem is distinct from the classic problem of tomography where both the projections and the unit vectors normal to t…
▽ More
We formulate and investigate a statistical inverse problem of a random tomographic nature, where a probability density function on $\mathbb{R}^3$ is to be recovered from observation of finitely many of its two-dimensional projections in random and unobservable directions. Such a problem is distinct from the classic problem of tomography where both the projections and the unit vectors normal to the projection plane are observable. The problem arises in single particle electron microscopy, a powerful method that biophysicists employ to learn the structure of biological macromolecules. Strictly speaking, the problem is unidentifiable and an appropriate reformulation is suggested hinging on ideas from Kendall's theory of shape. Within this setup, we demonstrate that a consistent solution to the problem may be derived, without attempting to estimate the unknown angles, if the density is assumed to admit a mixture representation.
△ Less
Submitted 2 September, 2009;
originally announced September 2009.
