-
Completion of Matrices with Low Description Complexity
Authors:
Erwin Riegler,
Günther Koliander,
David Stotz,
Helmut Bölcskei
Abstract:
We propose a theory for matrix completion that goes beyond the low-rank structure commonly considered in the literature and applies to general matrices of low description complexity. Specifically, complexity of the sets of matrices encompassed by the theory is measured in terms of Hausdorff and upper Minkowski dimensions. Our goal is the characterization of the number of linear measurements, with…
▽ More
We propose a theory for matrix completion that goes beyond the low-rank structure commonly considered in the literature and applies to general matrices of low description complexity. Specifically, complexity of the sets of matrices encompassed by the theory is measured in terms of Hausdorff and upper Minkowski dimensions. Our goal is the characterization of the number of linear measurements, with an emphasis on rank-$1$ measurements, needed for the existence of an algorithm that yields reconstruction, either perfect, with probability 1, or with arbitrarily small probability of error, depending on the setup. Concretely, we show that matrices taken from a set $\mathcal{U}$ such that $\mathcal{U}-\mathcal{U}$ has Hausdorff dimension $s$ can be recovered from $k>s$ measurements, and random matrices supported on a set $\mathcal{U}$ of Hausdorff dimension $s$ can be recovered with probability 1 from $k>s$ measurements. What is more, we establish the existence of recovery mappings that are robust against additive perturbations or noise in the measurements. Concretely, we show that there are $β$-Hölder continuous mappings recovering matrices taken from a set of upper Minkowski dimension $s$ from $k>2s/(1-β)$ measurements and, with arbitrarily small probability of error, random matrices supported on a set of upper Minkowski dimension $s$ from $k>s/(1-β)$ measurements. The numerous concrete examples we consider include low-rank matrices, sparse matrices, QR decompositions with sparse R-components, and matrices of fractal nature.
△ Less
Submitted 2 October, 2024; v1 submitted 7 March, 2023;
originally announced March 2023.
-
Fusion of Probability Density Functions
Authors:
Günther Koliander,
Yousef El-Laham,
Petar M. Djurić,
Franz Hlawatsch
Abstract:
Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processi…
▽ More
Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processing, statistical inference, and machine learning, a universally accepted method for pdf fusion does not exist. The diversity of approaches, perspectives, and solutions related to pdf fusion motivates a unified presentation of the theory and methodology of the field. We discuss three different approaches to fusing pdfs. In the axiomatic approach, the fusion rule is defined indirectly by a set of properties (axioms). In the optimization approach, it is the result of minimizing an objective function that involves an information-theoretic divergence or a distance measure. In the supra-Bayesian approach, the fusion center interprets the pdfs to be fused as random observations. Our work is partly a survey, reviewing in a structured and coherent fashion many of the concepts and methods that have been developed in the literature. In addition, we present new results for each of the three approaches. Our original contributions include new fusion rules, axioms, and axiomatic and optimization-based characterizations; a new formulation of supra-Bayesian fusion in terms of finite-dimensional parametrizations; and a study of supra-Bayesian fusion of posterior pdfs for linear Gaussian models.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
A Differential Entropy Estimator for Training Neural Networks
Authors:
Georg Pichler,
Pierre Colombo,
Malik Boudiaf,
Günther Koliander,
Pablo Piantanida
Abstract:
Mutual Information (MI) has been widely used as a loss regularizer for training neural networks. This has been particularly effective when learn disentangled or compressed representations of high dimensional data. However, differential entropy (DE), another fundamental measure of information, has not found widespread use in neural network training. Although DE offers a potentially wider range of a…
▽ More
Mutual Information (MI) has been widely used as a loss regularizer for training neural networks. This has been particularly effective when learn disentangled or compressed representations of high dimensional data. However, differential entropy (DE), another fundamental measure of information, has not found widespread use in neural network training. Although DE offers a potentially wider range of applications than MI, off-the-shelf DE estimators are either non differentiable, computationally intractable or fail to adapt to changes in the underlying distribution. These drawbacks prevent them from being used as regularizers in neural networks training. To address shortcomings in previously proposed estimators for DE, here we introduce KNIFE, a fully parameterized, differentiable kernel-based estimator of DE. The flexibility of our approach also allows us to construct KNIFE-based estimators for conditional (on either discrete or continuous variables) DE, as well as MI. We empirically validate our method on high-dimensional synthetic data and further apply it to guide the training of neural networks for real-world tasks. Our experiments on a large variety of tasks, including visual domain adaptation, textual fair classification, and textual fine-tuning demonstrate the effectiveness of KNIFE-based estimation. Code can be found at https://github.com/g-pichler/knife.
△ Less
Submitted 19 June, 2022; v1 submitted 14 February, 2022;
originally announced February 2022.
-
Lossy Compression of General Random Variables
Authors:
Erwin Riegler,
Helmut Bölcskei,
Günther Koliander
Abstract:
This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fract…
▽ More
This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our main contributions are bounds on the rate-distortion function and the quantization error. These bounds are very general and essentially only require the existence of reference measures satisfying certain regularity conditions in terms of small ball probabilities. To illustrate the wide applicability of our results, we particularize them to random variables taking values in i) manifolds, namely, hyperspheres and Grassmannians, and ii) self-similar sets characterized by iterated function systems satisfying the weak separation property.
△ Less
Submitted 2 June, 2023; v1 submitted 24 November, 2021;
originally announced November 2021.
-
Modelling the Utility of Group Testing for Public Health Surveillance
Authors:
Günther Koliander,
Georg Pichler
Abstract:
In epidemic or pandemic situations, resources for testing the infection status of individuals may be scarce. Although group testing can help to significantly increase testing capabilities, the (repeated) testing of entire populations can exceed the resources of any country. We thus propose an extension of the theory of group testing that takes into account the fact that definitely specifying the i…
▽ More
In epidemic or pandemic situations, resources for testing the infection status of individuals may be scarce. Although group testing can help to significantly increase testing capabilities, the (repeated) testing of entire populations can exceed the resources of any country. We thus propose an extension of the theory of group testing that takes into account the fact that definitely specifying the infection status of each individual is impossible. Our theory builds on assigning to each individual an infection status (healthy/infected), as well as an associated cost function for erroneous assignments. This cost function is versatile, e.g., it could take into account that false negative assignments are worse than false positive assignments and that false assignments in critical areas, such as health care workers, are more severe than in the general population. Based on this model, we study the optimal use of a limited number of tests to minimize the expected cost. More specifically, we utilize information-theoretic methods to give a lower bound on the expected cost and describe simple strategies that can significantly reduce the expected cost over currently known strategies. A detailed example is provided to illustrate our theory.
△ Less
Submitted 28 October, 2021; v1 submitted 11 September, 2021;
originally announced September 2021.
-
Zeros of Gaussian Weyl-Heisenberg functions and hyperuniformity of charge
Authors:
Antti Haimi,
Günther Koliander,
José Luis Romero
Abstract:
We study Gaussian random functions on the complex plane whose stochastics are invariant under the Weyl-Heisenberg group (twisted stationarity). The theory is modeled on translation invariant Gaussian entire functions, but allows for non-analytic examples, in which case winding numbers can be either positive or negative.
We calculate the first intensity of zero sets of such functions, both when c…
▽ More
We study Gaussian random functions on the complex plane whose stochastics are invariant under the Weyl-Heisenberg group (twisted stationarity). The theory is modeled on translation invariant Gaussian entire functions, but allows for non-analytic examples, in which case winding numbers can be either positive or negative.
We calculate the first intensity of zero sets of such functions, both when considered as points on the plane, or as charges according to their phase winding. In the latter case, charges are shown to be in a certain average equilibrium independently of the particular covariance structure (universal screening). We investigate the corresponding fluctuations, and show that in many cases they are suppressed at large scales (hyperuniformity). This means that universal screening is empirically observable at large scales. We also derive an asymptotic expression for the charge variance.
As a main application, we obtain statistics for the zero sets of the short-time Fourier transform of complex white noise with general windows, and also prove the following uncertainty principle: the expected number of zeros per unit area is minimized, among all window functions, exactly by generalized Gaussians. Further applications include poly-entire functions such as covariant derivatives of Gaussian entire functions.
△ Less
Submitted 3 May, 2022; v1 submitted 22 December, 2020;
originally announced December 2020.
-
On the Estimation of Information Measures of Continuous Distributions
Authors:
Georg Pichler,
Pablo Piantanida,
Günther Koliander
Abstract:
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space, computed from a finite number of samples, when the probability density function belongs to a predetermined convex family $\mathcal{P}$. First, estimating diffe…
▽ More
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space, computed from a finite number of samples, when the probability density function belongs to a predetermined convex family $\mathcal{P}$. First, estimating differential entropy to any accuracy is shown to be infeasible if the differential entropy of densities in $\mathcal{P}$ is unbounded, clearly showing the necessity of additional assumptions. Subsequently, we investigate sufficient conditions that enable confidence bounds for the estimation of differential entropy. In particular, we provide confidence bounds for simple histogram based estimation of differential entropy from a fixed number of samples, assuming that the probability density function is Lipschitz continuous with known Lipschitz constant and known, bounded support. Our focus is on differential entropy, but we provide examples that show that similar results hold for mutual information and relative entropy as well.
△ Less
Submitted 24 November, 2021; v1 submitted 7 February, 2020;
originally announced February 2020.
-
Minimal Achievable Sufficient Statistic Learning
Authors:
Milan Cvitkovic,
Günther Koliander
Abstract:
We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training method for machine learning models that attempts to produce minimal sufficient statistics with respect to a class of functions (e.g. deep networks) being optimized over. In deriving MASS Learning, we also introduce Conserved Differential Information (CDI), an information-theoretic quantity that - unlike standard mutua…
▽ More
We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training method for machine learning models that attempts to produce minimal sufficient statistics with respect to a class of functions (e.g. deep networks) being optimized over. In deriving MASS Learning, we also introduce Conserved Differential Information (CDI), an information-theoretic quantity that - unlike standard mutual information - can be usefully applied to deterministically-dependent continuous random variables like the input and output of a deep network. In a series of experiments, we show that deep networks trained with MASS Learning achieve competitive performance on supervised learning and uncertainty quantification benchmarks.
△ Less
Submitted 11 June, 2019; v1 submitted 19 May, 2019;
originally announced May 2019.
-
Rate-Distortion Theory for General Sets and Measures
Authors:
Erwin Riegler,
Günther Koliander,
Helmut Bölcskei
Abstract:
This paper is concerned with a rate-distortion theory for sequences of i.i.d. random variables with general distribution supported on general sets including manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in modeli…
▽ More
This paper is concerned with a rate-distortion theory for sequences of i.i.d. random variables with general distribution supported on general sets including manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in modeling of Ethernet traffic. We derive a lower bound on the (single-letter) rate-distortion function that applies to random variables X of general distribution and for continuous X reduces to the classical Shannon lower bound. Moreover, our lower bound is explicit up to a parameter obtained by solving a convex optimization problem in a nonnegative real variable. The only requirement for the bound to apply is the existence of a sigma-finite reference measure for X satisfying a certain subregularity condition. This condition is very general and prevents the reference measure from being highly concentrated on balls of small radii. To illustrate the wide applicability of our result, we evaluate the lower bound for a random variable distributed uniformly on a manifold, namely, the unit circle, and a random variable distributed uniformly on a self-similar set, namely, the middle third Cantor set.
△ Less
Submitted 24 April, 2018;
originally announced April 2018.
-
Lossless Analog Compression
Authors:
Giovanni Alberti,
Helmut Bölcskei,
Camillo De Lellis,
Günther Koliander,
Erwin Riegler
Abstract:
We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers…
▽ More
We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x. We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution -- with respect to the s-dimensional Hausdorff measure -- supported on countable unions of s-dimensional differentiable submanifolds of the m-dimensional real coordinate space. Countable unions of differentiable submanifolds include essentially all signal models used in the compressed sensing literature. Specifically, we prove that, for almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n>s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n<s linear measurements. This leads us to the introduction of the new class of s-analytic random vectors, which admit a strong converse in the sense of n greater than or equal to s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions.
△ Less
Submitted 2 October, 2024; v1 submitted 19 March, 2018;
originally announced March 2018.
-
Information Bottleneck on General Alphabets
Authors:
Georg Pichler,
Günther Koliander
Abstract:
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables $(Y, X)$, we show essentially that for some $n \in \mathbb{N}$, a function $f$ with rate limit $\log|f| \le nR$ and $I(Y^n; f(X^n)) \ge nS$ exists if and only if there is a random vari…
▽ More
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables $(Y, X)$, we show essentially that for some $n \in \mathbb{N}$, a function $f$ with rate limit $\log|f| \le nR$ and $I(Y^n; f(X^n)) \ge nS$ exists if and only if there is a random variable $U$ such that the Markov chain $Y - X - U$ holds, $I(U; X) \le R$ and $I(U; Y) \ge S$. The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.
△ Less
Submitted 1 May, 2018; v1 submitted 3 January, 2018;
originally announced January 2018.
-
Rate-Distortion Theory of Finite Point Processes
Authors:
Günther Koliander,
Dominic Schuhmacher,
Franz Hlawatsch
Abstract:
We study the compression of data in the case where the useful information is contained in a set rather than a vector, i.e., the ordering of the data points is irrelevant and the number of data points is unknown. Our analysis is based on rate-distortion theory and the theory of finite point processes. We introduce fundamental information-theoretic concepts and quantities for point processes and pre…
▽ More
We study the compression of data in the case where the useful information is contained in a set rather than a vector, i.e., the ordering of the data points is irrelevant and the number of data points is unknown. Our analysis is based on rate-distortion theory and the theory of finite point processes. We introduce fundamental information-theoretic concepts and quantities for point processes and present general lower and upper bounds on the rate-distortion function. To enable a comparison with the vector setting, we concretize our bounds for point processes of fixed cardinality. In particular, we analyze a fixed number of unordered Gaussian data points and show that we can significantly reduce the required rates compared to the best possible compression strategy for Gaussian vectors. As an example of point processes with variable cardinality, we study the best possible compression of Poisson point processes. For the specific case of a Poisson point process with uniform intensity on the unit square, our lower and upper bounds are separated by only a small gap and thus provide a good characterization of the rate-distortion function.
△ Less
Submitted 22 May, 2018; v1 submitted 19 April, 2017;
originally announced April 2017.
-
Lossless Linear Analog Compression
Authors:
Giovanni Alberti,
Helmut Bölcskei,
Camillo De Lellis,
Günther Koliander,
Erwin Riegler
Abstract:
We establish the fundamental limits of lossless linear analog compression by considering the recovery of random vectors ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ from the noiseless linear measurements ${\boldsymbol{\mathsf{y}}}=\boldsymbol{A}{\boldsymbol{\mathsf{x}}}$ with measurement matrix $\boldsymbol{A}\in{\mathbb R}^{n\times m}$. Specifically, for a random vector…
▽ More
We establish the fundamental limits of lossless linear analog compression by considering the recovery of random vectors ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ from the noiseless linear measurements ${\boldsymbol{\mathsf{y}}}=\boldsymbol{A}{\boldsymbol{\mathsf{x}}}$ with measurement matrix $\boldsymbol{A}\in{\mathbb R}^{n\times m}$. Specifically, for a random vector ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ of arbitrary distribution we show that ${\boldsymbol{\mathsf{x}}}$ can be recovered with zero error probability from $n>\inf\underline{\operatorname{dim}}_\mathrm{MB}(U)$ linear measurements, where $\underline{\operatorname{dim}}_\mathrm{MB}(\cdot)$ denotes the lower modified Minkowski dimension and the infimum is over all sets $U\subseteq{\mathbb R}^{m}$ with $\mathbb{P}[{\boldsymbol{\mathsf{x}}}\in U]=1$. This achievability statement holds for Lebesgue almost all measurement matrices $\boldsymbol{A}$. We then show that $s$-rectifiable random vectors---a stochastic generalization of $s$-sparse vectors---can be recovered with zero error probability from $n>s$ linear measurements. From classical compressed sensing theory we would expect $n\geq s$ to be necessary for successful recovery of ${\boldsymbol{\mathsf{x}}}$. Surprisingly, certain classes of $s$-rectifiable random vectors can be recovered from fewer than $s$ measurements. Imposing an additional regularity condition on the distribution of $s$-rectifiable random vectors ${\boldsymbol{\mathsf{x}}}$, we do get the expected converse result of $s$ measurements being necessary. The resulting class of random vectors appears to be new and will be referred to as $s$-analytic random vectors.
△ Less
Submitted 5 May, 2016; v1 submitted 3 May, 2016;
originally announced May 2016.
-
Entropy and Source Coding for Integer-Dimensional Singular Random Variables
Authors:
Günther Koliander,
Georg Pichler,
Erwin Riegler,
Franz Hlawatsch
Abstract:
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random var…
▽ More
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random variables and the differential entropy of continuous random variables as special cases. We show that it transforms in a natural manner under Lipschitz functions, and that it is invariant under unitary transformations. We define joint entropy and conditional entropy for integer-dimensional singular random variables, and we show that the proposed entropy conveys useful expressions of the mutual information. As first applications of our entropy definition, we present a result on the minimal expected codeword length of quantized integer-dimensional singular sources and a Shannon lower bound for integer-dimensional singular sources.
△ Less
Submitted 3 January, 2017; v1 submitted 13 May, 2015;
originally announced May 2015.
-
Oversampling Increases the Pre-Log of Noncoherent Rayleigh Fading Channels
Authors:
Meik Dörpinghaus,
Günther Koliander,
Giuseppe Durisi,
Erwin Riegler,
Heinrich Meyr
Abstract:
We analyze the capacity of a continuous-time, time-selective, Rayleigh block-fading channel in the high signal-to-noise ratio (SNR) regime. The fading process is assumed stationary within each block and to change independently from block to block; furthermore, its realizations are not known a priori to the transmitter and the receiver (noncoherent setting). A common approach to analyzing the capac…
▽ More
We analyze the capacity of a continuous-time, time-selective, Rayleigh block-fading channel in the high signal-to-noise ratio (SNR) regime. The fading process is assumed stationary within each block and to change independently from block to block; furthermore, its realizations are not known a priori to the transmitter and the receiver (noncoherent setting). A common approach to analyzing the capacity of this channel is to assume that the receiver performs matched filtering followed by sampling at symbol rate (symbol matched filtering). This yields a discrete-time channel in which each transmitted symbol corresponds to one output sample. Liang & Veeravalli (2004) showed that the capacity of this discrete-time channel grows logarithmically with the SNR, with a capacity pre-log equal to $1-{Q}/{N}$. Here, $N$ is the number of symbols transmitted within one fading block, and $Q$ is the rank of the covariance matrix of the discrete-time channel gains within each fading block. In this paper, we show that symbol matched filtering is not a capacity-achieving strategy for the underlying continuous-time channel. Specifically, we analyze the capacity pre-log of the discrete-time channel obtained by oversampling the continuous-time channel output, i.e., by sampling it faster than at symbol rate. We prove that by oversampling by a factor two one gets a capacity pre-log that is at least as large as $1-1/N$. Since the capacity pre-log corresponding to symbol-rate sampling is $1-Q/N$, our result implies indeed that symbol matched filtering is not capacity achieving at high SNR.
△ Less
Submitted 4 August, 2014; v1 submitted 2 May, 2014;
originally announced May 2014.
-
Degrees of Freedom of Generic Block-Fading MIMO Channels without A Priori Channel State Information
Authors:
Günther Koliander,
Erwin Riegler,
Giuseppe Durisi,
Franz Hlawatsch
Abstract:
We studynthe high-SNR capacity of generic MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information but both are aware of the channel statistics. In contrast to the well-established constant block-fading model, we allow the fading to vary within each block with a temporal correlation that is "generic" (in the sense…
▽ More
We studynthe high-SNR capacity of generic MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information but both are aware of the channel statistics. In contrast to the well-established constant block-fading model, we allow the fading to vary within each block with a temporal correlation that is "generic" (in the sense used in the interference-alignment literature). We show that the number of degrees of freedom of a generic MIMO Rayleigh block-fading channel with $T$ transmit antennas and block length $N$ is given by $T(1-1/N)$ provided that $T<N$ and the number of receive antennas is at least $T(N-1)/(N-T)$. A comparison with the constant block-fading channel (where the fading is constant within each block) shows that, for large block lengths, generic correlation increases the number of degrees of freedom by a factor of up to four.
△ Less
Submitted 5 November, 2014; v1 submitted 9 October, 2013;
originally announced October 2013.
-
Generic Correlation Increases Noncoherent MIMO Capacity
Authors:
Günther Koliander,
Erwin Riegler,
Giuseppe Durisi,
Franz Hlawatsch
Abstract:
We study the high-SNR capacity of MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information. We show that when the number of receive antennas is sufficiently large and the temporal correlation within each block is "generic" (in the sense used in the interference-alignment literature), the capacity pre-log is given b…
▽ More
We study the high-SNR capacity of MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information. We show that when the number of receive antennas is sufficiently large and the temporal correlation within each block is "generic" (in the sense used in the interference-alignment literature), the capacity pre-log is given by T(1-1/N) for T<N, where T denotes the number of transmit antennas and N denotes the block length. A comparison with the widely used constant block-fading channel (where the fading is constant within each block) shows that for a large block length, generic correlation increases the capacity pre-log by a factor of about four.
△ Less
Submitted 5 June, 2013;
originally announced June 2013.
-
A Lower Bound on the Noncoherent Capacity Pre-log for the MIMO Channel with Temporally Correlated Fading
Authors:
Günther Koliander,
Erwin Riegler,
Giuseppe Durisi,
Veniamin I. Morgenshtern,
Franz Hlawatsch
Abstract:
We derive a lower bound on the capacity pre-log of a temporally correlated Rayleigh block-fading multiple-input multiple-output (MIMO) channel with T transmit antennas and R receive antennas in the noncoherent setting (no a priori channel knowledge at the transmitter and the receiver). In this model, the fading process changes independently across blocks of length L and is temporally correlated wi…
▽ More
We derive a lower bound on the capacity pre-log of a temporally correlated Rayleigh block-fading multiple-input multiple-output (MIMO) channel with T transmit antennas and R receive antennas in the noncoherent setting (no a priori channel knowledge at the transmitter and the receiver). In this model, the fading process changes independently across blocks of length L and is temporally correlated within each block for each transmit-receive antenna pair, with a given rank Q of the corresponding correlation matrix. Our result implies that for almost all choices of the coloring matrix that models the temporal correlation, the pre-log can be lower-bounded by T(1-1/L) for T <= (L-1)/Q provided that R is sufficiently large. The widely used constant block-fading model is equivalent to the temporally correlated block-fading model with Q = 1 for the special case when the temporal correlation for each transmit-receive antenna pair is the same, which is unlikely to be observed in practice. For the constant block-fading model, the capacity pre-log is given by T(1-T/L), which is smaller than our lower bound for the case Q = 1. Thus, our result suggests that the assumptions underlying the constant block- fading model lead to a pessimistic result for the capacity pre-log.
△ Less
Submitted 11 February, 2013;
originally announced February 2013.
